Senior Engineer, Competitive Benchmarking & Performance

Elastic

1d • $133,100 - $252,900

About The Position

Elastic is seeking a Senior Software Engineer to lead competitive benchmarking initiatives within the Elasticsearch Performance team. The role focuses on developing representative workloads, automating reproducible experiments, and investigating how Elasticsearch compares with alternative systems. Findings will identify performance strengths, gaps, and tradeoffs, guide engineering improvements, and help customers make informed technical decisions.

This is a hands-on engineering role centered on building benchmarks and investigating performance. Technical publishing is a shared responsibility with engineering and content partners. The work connects deep performance analysis with product improvements and public technical work, including comparisons across concurrent ingestion and hybrid search at comparable retrieval quality and cost.

Requirements

• Solid software engineering skills, including experience building maintainable automation and analysis tools in Python or another relevant programming language.
• Hands-on experience evaluating and diagnosing performance in distributed systems, databases, or search infrastructure.
• Depth in at least one relevant area: search, vector retrieval, analytical databases, or observability systems, and ability to learn unfamiliar systems.
• Experience designing representative datasets and workloads, including query patterns, concurrency, and ingestion behavior.
• Sound experimental judgment, including repeated measurements, variability analysis, warm-up and cache effects, and distinguishing meaningful changes from measurement noise.
• Experience building reproducible test environments and documenting configurations, procedures, and results.
• Clear technical writing and communication skills, with the ability to explain complex findings and their limitations.
• Solid collaboration skills and ability to turn performance evidence into applicable engineering recommendations.

Nice To Haves

• Experience with Elasticsearch, Lucene, Rally, ClickBench, or vector search benchmark frameworks.
• Experience measuring retrieval quality alongside latency, throughput, and resource usage.
• Familiarity with cloud performance and cost analysis, including autoscaling and serverless systems.
• Public technical writing, open-source contributions, or published benchmark studies.

Responsibilities

• Own prioritized competitive benchmarking initiatives across search, vector and hybrid retrieval, and analytical and observability workloads.
• Build and maintain automated benchmark harnesses, datasets, query sets, and load profiles that reflect real-world usage, including concurrent ingestion and querying.
• Design fair comparisons with documented system versions, configurations, tuning decisions, resource budgets, and functional differences.
• Evaluate latency distributions, throughput, resource consumption, and cost efficiency; for retrieval workloads, measure performance at comparable recall or relevance targets.
• Adapt and extend workload-appropriate tools such as Elasticsearch Rally, ClickBench, and vector search benchmark frameworks.
• Investigate performance differences using profiling, system metrics, and controlled experiments; collaborate with engineering teams to identify causes and validate improvements.
• Evaluate relevant self-managed and managed cloud deployments, including serverless offerings, accounting for scaling behavior and service constraints.
• Work with engineering and technical content partners to publish reproducible benchmark artifacts and clear technical reports explaining methodology, findings, limitations, and tradeoffs.
• Partner with engineering and product management to prioritize benchmark coverage and translate findings into actionable product improvements.

Benefits

• Company-matched 401(k), with dollar-for-dollar matching up to 6% of eligible earnings.
• Stock program eligibility.
• Health coverage for employees and their families in many locations.
• Flexible locations and schedules for many roles.
• Generous number of vacation days each year.
• Matching up to $2,000 (or local currency equivalent) for financial donations and service.
• Up to 40 hours each year for volunteer projects.
• At least 16 weeks of parental leave.
• Competitive pay based on the work performed and not previous salary.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed