Senior Principal Model Bring-up Engineer

SambaNova Systems • San Jose, TX

7d

About The Position

SambaNova is hiring a Senior Principal Model Bring-up Engineer to bring frontier open-weight large language models—including dense, mixture-of-experts, and hybrid Mamba/attention architectures—from Hugging Face to production serving on its RDU dataflow accelerators. The work involves adapting models, mapping them across chips and hosts, compiling them, and validating numerical correctness against reference implementations.

The engineer will also develop shared inference modules, including continuous batching, prompt caching, chunked and incremental prefill, dynamic tensor shapes, speculative decoding, FP8/FP4, and tensor and expert parallelism. The team aims to make each subsequent model less costly to bring up by integrating reusable improvements into its shared stack.

SambaNova describes its RDU as a chip based on a dataflow architecture rather than the traditional GPU model. RDUs are packaged into SambaRack, rack-scale hardware for deploying AI models. The role includes debugging issues at the intersection of models, compilers, and hardware, and delivering production-serving performance measurements.

Requirements

• B.S. in Computer Science, Electrical Engineering, or a related field.
• At least 8 years of industry experience in ML systems, inference, or compiler engineering.
• Strong Python and PyTorch internals experience, plus some C++ experience.
• Deep knowledge of transformer inference, including KV caching, GQA and sliding-window attention, MoE routing, prefill versus decode, batching, and speculative decoding.
• Hands-on experience with accelerator bring-up, ML compilers (MLIR, XLA, TVM, Triton, or torch-mlir), or inference engines (vLLM, SGLang, or TensorRT-LLM).
• Rigor in numerical debugging.
• Comfort with Bazel, Slurm, and large monorepos.
• Clear writing and evidence-backed claims.

Nice To Haves

• M.S. or Ph.D. in Computer Science, Electrical Engineering, or a related field.
• Experience with dataflow or spatial compilers.
• Experience with kernel DSLs.
• Experience with FP8 and FP4 inference.
• Experience with Mamba and state-space models (SSMs).
• Experience with expert parallelism.
• Experience with multi-agent AI engineering workflows.
• Open-source inference or compiler contributions.

Responsibilities

• Bring up new LLM architectures end to end, including model adaptation, checkpoint conversion, multi-chip and multi-host mapping, compilation, strict numerical validation, and production serving with latency and throughput measurements.
• Build shared inference modules for continuous batching over a paged KV pool, prompt caching, chunked and incremental prefill, dynamic tensor shapes, speculative decoding, FP8/FP4, and tensor and expert parallelism.
• Debug issues at the intersection of models, compilers, and hardware, and land each fix in the appropriate shared component.
• Own a model family from its first compile through production serving and integration into the shared stack.
• Turn model-specific fixes into shared modules adopted by others and measurably shorten a long-running team workflow.

Benefits

• Equity.
• 95% premium coverage for employee medical insurance.
• 77% premium coverage for dependent medical insurance.
• Health Savings Account (HSA) with employer contribution.
• Dental insurance.
• Vision insurance.
• Short- and long-term disability insurance.
• Basic life, voluntary life, and AD&D insurance.
• Flexible Spending Account (FSA) options for health care, limited purpose, and dependent care.
• Headspace subscription.
• Gympass+ membership with access to physical gyms.
• One Medical membership.
• Counseling services through an Employee Assistance Program.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior