AI Framework Engineer

Modular • Los Altos, CA

80d • $120,000 - $222,000 • Hybrid

About The Position

At Modular, the MAX Framework is the developer-facing layer of a next-generation AI platform. It defines the APIs developers use to express models, integrate custom kernels, orchestrate execution, improve quality, and ship systems into production. The AI Framework Engineer will work on core APIs and developer experience for AI inference and training, at the intersection of API design, systems engineering, and modern AI frameworks. The role focuses on making MAX coherent, powerful, and intuitive while preserving performance, portability, maintainability, and total cost of ownership.

The role is based at Modular’s Los Altos, California headquarters, with a minimum of three days per week onsite. US-based candidates may apply, and relocation assistance is provided for out-of-state US candidates. Travel two to four times a year is expected for all roles.

Requirements

• Passion for designing developer-facing APIs, such as SDKs, frameworks, or platforms.
• Good understanding of modern AI frameworks and their design tradeoffs, such as PyTorch, JAX, TensorFlow, vLLM, and XLA/MLIR-adjacent ecosystems.
• Proficiency in at least one systems or performance language such as C++, Rust, or Go, and at least one user-facing language such as Python.
• Good instincts for API ergonomics, including naming, composability, types, error handling, configurability, and clarity.
• Strong written communication skills and ability to write specifications engineers can implement without ambiguity.
• Commitment to engineering standards and pragmatism, with a bias toward incremental development without compromising long-term design.
• Bachelor’s degree in Engineering, Information Systems, Computer Engineering, Electrical Engineering, Computer Science, or a related field and 2+ years of software engineering or related work experience; or a master’s degree in a listed or related field and 1+ year of such experience; or a PhD in a listed or related field.
• At least 1 year of academic or work experience with a programming language such as Python, Mojo, Rust, or C++.

Nice To Haves

• Familiarity with Mojo.

Responsibilities

• Develop the MAX API surface, including namespaces, core abstractions, extension points, programming model, compatibility guarantees, and developer experience.
• Design and develop inference APIs for real-world serving needs, including model loading, distributed inference, quantization, tokenization and pipelines, configuration, batching and streaming, and deployment ergonomics.
• Develop training APIs that scale from single-device to distributed execution, with primitives for device placement, parallelism, checkpointing, and observability.
• Contribute to a coherent programming model across Python and Mojo-adjacent surfaces by aligning naming, types, and conventions, avoiding leaky abstractions, and establishing a clear path to successful use.
• Write and socialize technical RFCs and specifications, gather feedback from internal model engineers and external users, and iterate toward consensus.
• Partner with compiler/runtime, kernels, cloud/serving, and documentation/DevRel teams to ensure APIs map cleanly to underlying capabilities.
• Develop reference implementations and example materials, including golden-path examples, architecture templates, and reusable best-practice patterns.

Benefits

• Benefits package may include comprehensive healthcare coverage, retirement and savings programs, employee stock purchase opportunities, paid time off, wellbeing resources, family support programs, and learning and development opportunities; specific benefits may vary by location.
• Competitive compensation packages, including RSU grants; total compensation also includes an annual target bonus, equity, and benefits.
• Relocation assistance for out-of-state candidates based in the US.
• Regular team onsite events and local meetups.

What This Job Offers

• Job Type: Full-time
• Career Level: Entry Level