Software Engineer, Trainium

OpenAI • San Francisco, CA

4d • $295,000 - $380,000

About The Position

The team builds the performance-critical systems that allow OpenAI's models to run efficiently across a diverse set of AI accelerators. We work across the inference stack, from low-level kernels and compilers through model execution, to unlock the full capabilities of the underlying hardware. As a Software Engineer, Trainium, you will help bring OpenAI's inference workloads to AWS Trainium and build the software stack required to run cutting-edge frontier models efficiently on the platform. This is a deeply technical, cross-stack role spanning kernels, compilers, and model execution. You will work on the systems needed to support OpenAI's inference stack on Trainium, including developing and optimizing high-performance kernels, improving compiler support, and enabling efficient execution of the model forward pass. You'll work closely with engineers across inference, compilers, kernels, and ML systems to identify performance bottlenecks and build the software needed to take full advantage of Trainium. The work may range from low-level hardware-specific optimization to compiler and runtime improvements to integrating new model architectures into the inference stack. If you enjoy working at the intersection of ML systems, compilers, kernels, and accelerator hardware, this role is for you. We're looking for engineers who are self-directed, comfortable operating across abstraction layers, and excited to solve challenging performance problems for frontier-scale AI systems.

Requirements

• 3+ years of relevant engineering experience, ideally in ML systems, compilers, kernels, runtimes, or performance engineering.
• Strong systems programming fundamentals and experience writing performance-critical software.
• Experience working with GPU, TPU, Trainium, or other specialized accelerator architectures.
• Ability to reason about performance across multiple layers of the stack, from hardware and kernels through compilers and ML frameworks.
• Owning technically ambiguous problems end-to-end and learning new hardware and software domains as needed.

Nice To Haves

• Experience with AWS Trainium or the AWS Neuron SDK.
• Contributions to ML frameworks such as PyTorch or JAX, compiler infrastructure such as LLVM, MLIR, XLA, or Triton, or experience developing kernels for specialized accelerators.

Responsibilities

• Build and optimize OpenAI's inference stack for AWS Trainium.
• Develop high-performance kernels for critical model operations and workloads.
• Extend and improve compiler support to efficiently target Trainium hardware.
• Build the systems necessary to execute and optimize the model forward pass on Trainium.
• Profile workloads and identify bottlenecks across kernels, compiler-generated code, runtime, and model execution.
• Partner with inference and ML systems teams to bring new models and architectures onto Trainium.
• Work across the hardware/software boundary to unlock performance and capabilities from specialized AI accelerators.
• Own complex performance and systems problems end-to-end, from investigation through production deployment.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed