ML Systems, LLM Post-Training

ChipAgents • San Jose, CA

1d

About The Position

We are seeking an ML Systems Engineer to optimize the performance and efficiency of large language model inference powering an agentic AI platform. This is a technical role focused on low-level systems optimization. The engineer will implement performance optimizations, build evaluation harnesses, and architect multi-node clusters for training and inference to improve LLM throughput and latency. The work will affect the responsiveness and cost-efficiency of AI agents used by leading semiconductor companies to design chips.

Requirements

• B.S., M.S., or PhD in Computer Science, Electrical Engineering, or a related field, or equivalent experience.
• Experience with large-scale ML systems, GPU computing, or high-performance inference optimization.
• Strong proficiency in Python and C++/CUDA.
• Hands-on experience with SGLang, vLLM, PyTorch, or similar inference frameworks.
• Deep understanding of GPU architecture, memory hierarchies, and parallel computing paradigms.
• Experience deploying and optimizing LLMs in production, including model serving, batching strategies, distributed inference, or quantization.
• Strong systems-level debugging and profiling skills, with comfort working across multiple layers of the stack from CUDA kernels to application logic.
• Self-directed problem solver interested in ambitious optimization challenges.

Nice To Haves

• Familiarity with distributed computing frameworks such as Ray and multi-node training or inference.

Responsibilities

• Design, deploy, and optimize LLM inference systems across multi-node clusters, maximizing throughput and minimizing latency for production workloads.
• Implement and benchmark inference optimizations.
• Profile and analyze inference bottlenecks at the systems level, from GPU kernel execution to memory bandwidth constraints.
• Build evaluation harnesses and benchmarking frameworks to measure accuracy, throughput, latency, and resource utilization across parallelism strategies.
• Collaborate with research scientists to integrate new model architectures and optimizations into production inference infrastructure.
• Investigate and apply emerging techniques from research papers and open-source projects to improve inference performance.

Benefits

• Unlimited PTO.
• Full benefits, including medical, vision, dental, and 401(k).
• Two engineering-centric offices with free parking, a private gym, and free lunch, drinks, and snacks.
• Equity.

What This Job Offers

• Job Type: Full-time