Product Manager - AI SDK / Model-Serving Product Manager - US

Majestic Labs • Los Altos, CA

4d

About The Position

Majestic Labs is building vertically integrated AI infrastructure, from custom silicon to the servers and systems that power large AI models. The Product Manager will define and deliver the software stack that makes the company’s systems seamless for developers and cloud infrastructure teams. The role owns the compiler roadmap, runtime engines, and framework integrations that convert standard deep learning model representations into optimized execution paths on the Ignite APU. The objective is to let customers use their existing models and tools while achieving strong inference performance without needing to manage the underlying hardware.

Requirements

• A Master of Science or Ph.D. in Computer Science, Electrical Engineering, Mathematics, or Physics; stated as a strict, non-negotiable requirement.
• Five to eight years of experience in product management or software systems engineering, with a track record of shipping compiler technologies, deep learning frameworks, or developer platform tools.
• Deep theoretical and practical understanding of CUDA, Triton, high-performance C++ DSP libraries, and multithreaded parallel-computation paradigms.
• Understanding of inference serving, including vLLM, TensorRT-LLM, Triton Inference Server, and SGLang, as well as batching, KV-cache management, and latency-versus-throughput tradeoffs.
• Familiarity with HPC scale-out infrastructure, distributed-memory architectures, and networking protocols designed for high-performance clustering.
• Prior experience at an accelerated-software, cloud, or startup organization building complex AI compute systems; examples include NVIDIA, Meta, Google, AWS, SambaNova, Cerebras, Groq, and Lightning AI.

Nice To Haves

• MBA or equivalent experience in technology commercialization or product strategy.
• Active contributions to open-source AI infrastructure projects, compilers such as MLIR/LLVM, or performance-optimization libraries.
• Direct experience working at a frontier AI lab or foundation-model organization, such as OpenAI, Anthropic, xAI, or Google DeepMind, developing large-scale distributed systems.

Responsibilities

• Translate customer, engineering, and business requirements into detailed product requirements documents (PRDs), technical specifications, and feature definitions.
• Develop product schedules, milestone plans, and release timelines, managing dependencies across multiple engineering organizations.
• Define requirements for the software and AI model compilation toolchain and its roadmap.
• Lead product partnerships and technical integrations with machine learning frameworks and runtimes, including PyTorch, JAX, vLLM, SGLang, and Hugging Face Optimum, enabling support without customer code modifications.
• Define the performance optimization roadmap for low-level operator libraries, including BLAS and custom OpenAI Triton kernels.
• Define requirements for clustering and multi-node scale-out configurations, including execution across setups using SLURM, Ray, and LMCache.
• Advocate for developer experience by monitoring software compilation and execution in customer sandboxes and reducing the need for manual memory-management scripts, custom compiler hacks, and API wrappers.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior