Inference Systems Performance Architect
About The Position
SambaNova is hiring an architect for end-to-end inference performance across its AI inference platform. The role covers how requests move through tokenization, prefill, decode, and the interconnect, as well as sizing deployments against customer service-level objectives (SLOs). Its two central areas are building reproducible workload capture and benchmarking that reflect real and increasingly agentic production traffic, and developing performance models and simulations to evaluate configurations and hardware that do not yet exist. The role will help define heterogeneous, disaggregated inference, including GPU-based prefill and RDU-based decode, and address challenges involving networking, storage, prompt caching, and tail-latency-bound data movement. The work involves ambiguous problems without established solutions and includes setting technical direction across model optimization, systems, hardware, and product. The team measures production workloads, benchmarks them, models and simulates configurations, and profiles the distributed serving pipeline across host, accelerator, and fabric. Its work informs serving optimization, capacity planning, and future system and hardware planning.
Requirements
- B.S. in Computer Science, Computer Engineering, or a related field.
- 12+ years of experience in performance engineering, with a demonstrated record of technical leadership on large-scale, complex systems.
- Deep expertise in end-to-end performance analysis of distributed systems and ability to localize difficult-to-find bottlenecks.
- Proven command of realistic workload generation and simulation, and performance modeling, including calibrating models against real, variable workloads.
- Demonstrated ability to enter an unfamiliar domain and apply transferable performance methods and discipline.
- Ability to lead cross-functional efforts, mentor senior engineers, and influence organizational direction.
- Experience representing an organization credibly to customers and partners.
- Track record of independently scoping and delivering high-complexity, high-ambiguity work with significant product or roadmap impact.
Nice To Haves
- M.S. or Ph.D. in Computer Science, Computer Engineering, or a related field.
- Direct experience with LLM inference serving, including continuous batching, prompt/KV caching, prefill/decode disaggregation, and tail-latency SLOs.
- Familiarity with inference simulation frameworks or agentic benchmarking efforts.
- A public technical voice through talks, writing, or community presence.
Responsibilities
- Define and drive technical strategy for inference-systems performance, including workload capture, benchmarking, modeling, and simulation.
- Develop architecture that enables multiple potential future system configurations.
- Build workload-capture and agentic-benchmarking capabilities using representative production traffic, and scrutinize results for artificial contention or misleading cache-hit rates.
- Own performance modeling and simulation practices that predict the effects of configuration changes and inform capacity planning against customer SLOs and next-generation system and hardware planning.
- Drive tooling that produces accurate, actionable profiles of distributed inference pipelines and localizes bottlenecks across host, accelerator, and fabric.
- Serve as a senior technical voice across model optimization, systems, hardware, and product, connecting engineering activities and weighing reliability, scalability, operational cost, and ease of adoption.
- Represent SambaNova's performance story to customers and partners.
- Mentor principal and senior engineers and build systems, tools, and patterns that improve organizational productivity.
- Lead resolution of ambiguous, novel challenges that cross organizational boundaries or lack established solutions.
Benefits
- Equity.
- 95% premium coverage for employee medical insurance.
- 77% premium coverage for dependent medical insurance.
- Health Savings Account (HSA) with employer contribution.
- Dental insurance.
- Vision insurance.
- Short- and long-term disability insurance.
- Basic life, voluntary life, and AD&D insurance.
- Flexible Spending Account (FSA) options for health care, limited purpose, and dependent care.
- Headspace subscription.
- Gympass+ membership with access to physical gyms.
- One Medical membership.
- Counseling services through an Employee Assistance Program.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Principal
