Staff Software Developer, Search & Distributed Systems
About The Position
ACV is a technology company that has revolutionized how dealers buy and sell cars online, transforming the automotive industry. ACV Auctions Inc. (ACV) uses innovation and user-designed, data-driven applications and solutions to build a trusted and efficient digital marketplace with data solutions for sourcing, selling, and managing used vehicles with transparency and comprehensive insights. They are disruptors of the industry, with a network of brands including ACV Auctions, ACV Transportation, ClearCar, MAX Digital, ACV Capital within its Marketplace Products, as well as, True360 and Data Services. The company focuses on the Health, Physical, Financial, Social and Emotional Wellness of its Teammates. They are looking for a Staff Software Engineer to be accountable for their Search infrastructure, focusing on its scalability, reliability, and data resiliency. This role requires a battle-scarred Distributed Systems expert with deep knowledge of Elasticsearch internals, capable of analyzing, monitoring, alerting, and quickly resolving critical issues. The Staff Engineer will own the architectural connective tissue, ensuring flawless interaction between service layers and the event-based ecosystem with Search. As a Staff Engineer, the individual will set technical standards, drive systemic reliability, and mentor senior engineers across the organization.
Requirements
- Experience: 8+ years of software engineering experience, with at least 3+ years operating at a Senior or Staff level focusing on distributed systems and high-throughput platforms.
- Elasticsearch Mastery: Deep, authoritative knowledge of Elasticsearch internals. You have managed large-scale clusters and deeply understand mapping, analysis, query optimization, cluster state management, and split-brain mitigation.
- Full-Stack Context: Proficiency in the systems upstream and downstream of Search. You have hands-on experience with Kubernetes (EKS/GKE), API Gateway/BFF architectures, and event streams (Kafka).
- Resilience Engineering: A proven track record of implementing fault-tolerant patterns (retries, rate limiting, circuit breaking, dead letter queues) in microservice architectures.
- Observability: Expert-level ability to instrument systems and diagnose complex performance issues using modern observability stacks (Datadog, Prometheus, Grafana, OpenTelemetry).
- Leadership: Strong communication skills with a proven ability to influence cross-functional teams, build consensus around architectural decisions (the Knoster model!), and mentor mid-level and senior engineers.
Nice To Haves
- Experience with Infrastructure as Code (Terraform, Helm) for stateful applications.
- Familiarity with FinOps practices, specifically optimizing Elasticsearch compute and storage costs.
- Experience integrating AI-assisted development tools into your daily workflow.
Responsibilities
- Architect for Scale: Design, configure, and scale our Elasticsearch clusters. You will define our global strategies for shard routing, Index Lifecycle Management (ILM), heap tuning, and data tiering to support massive auction throughput.
- Master the Failure Modes: Anticipate and engineer away points of failure. You will design circuit breakers, implement backpressure mechanisms, and tune asymmetric timeouts to prevent retry storms between our BFFs, K8s services, and the Search layer.
- Expert Troubleshooting & IR: Act as the ultimate technical escalation point for complex, cross-system performance degradation. You will dive deep into JVM metrics, Garbage Collection pauses, K8s network bottlenecks, and slow logs to uncover and remediate root causes.
- Holistic System Ownership: Manage the entire data lifecycle. You will optimize the ingestion pipelines syncing our event datastreams driven by producers and consumers (Kafka) to Elasticsearch, ensuring eventual consistency and data integrity at scale.
- Drive Engineering Excellence: Draft authoritative architectural Blueprints, SOPs, and Runbooks. You will elevate the surrounding engineering culture by coaching teams on distributed systems design, observability best practices, and incident management.
- Modernize & Innovate: Scan the horizon for emerging technologies. You will help evaluate and integrate next-generation search capabilities (e.g., Vector Search, RAG architectures) to support our broader AI and machine learning initiatives.
Benefits
- Company Sponsored (paid) Healthcare
- Dental
- Vision
- Life/AD&D
- Short-Term and Long-Term Disability
- Comprehensive additional optional benefits such Critical Illness and Supplemental Life/AD&D
- All insurance benefits go into effect on your date of hire
- Generous Parental Leave Top-Up Pay
- Vacation Programs
- Employee Stock Purchase Program with additional opportunities to earn stock in the company
- Retirement planning through the Company’s RRSP
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
Number of Employees
501-1,000 employees
