ML Model Serving Engineer

Sesame • San Francisco, CA

5d • $175,000 - $280,000

About The Position

Sesame is building a new kind of computer, focused on integrating voice agents into daily life. The ML Model Serving Engineer will improve the serving layer for a range of LLM, speech, and vision models, partnering with ML infrastructure and training engineers to build a fast, cost-effective, accurate, and reliable serving layer for a new consumer product category. The role includes extending model-serving frameworks and collaborating with the training team to make models faster without sacrificing quality.

Requirements

• Expertise in a differentiable array computing framework, preferably PyTorch.
• Expertise optimizing machine-learning models for reliable serving at high throughput and low latency.
• Significant systems programming experience, including familiarity with high-performance server systems and complex codebases such as vLLM or PyTorch.
• Significant performance engineering experience, such as bottleneck analysis in high-scale server systems or profiling low-level systems code.
• Stay up to date on the latest techniques for model-serving optimization.

Nice To Haves

• Familiarity with high-performance LLM serving, such as vLLM or SGLang deployment and internals.
• Experience with a public cloud platform such as GCP, AWS, or Azure.
• Experience deploying and scaling cloud inference workloads using Kubernetes, Ray, or similar tools.
• A track record of leading complex, multi-month projects independently.
• Excitement about learning new things and working across multiple roles.

Responsibilities

• Improve Sesame’s serving layer for LLM, speech, and vision models.
• Partner with ML infrastructure and training engineers to build a fast, cost-effective, accurate, and reliable serving layer.
• Modify and extend LLM serving frameworks such as vLLM and SGLang to use current high-performance serving techniques.
• Work with the training team to identify opportunities to produce faster models without sacrificing quality.
• Use in-flight batching, caching, and custom kernels to speed up inference.
• Find ways to reduce model initialization times without sacrificing quality.

Benefits

• 401(k) with a maximum employer match of 3.5% of compensation.
• 100% employer-paid health, vision, and dental benefits for employees and their dependents.
• Unlimited PTO and sick time.
• Flexible spending account with employer matching up to $1,650 per year for the medical FSA.
• Guardian Employee Assistance Program (EAP).
• Competitive stock options offering an opportunity to share in the company's success.

What This Job Offers

• Job Type: Full-time
• Education Level: No Education Listed