Principal AI Product Engineer

Nscale•Seattle, WA

1d•$290,000 - $520,000

About The Position

Nscale is seeking a Principal AI Engineer (Specialised) to lead the inference and post-training pillar of its AI systems engineering organization. The role defines the multi-year technical roadmap for serving, evaluating, and post-training models on Nscale’s GPU cloud, including dedicated and serverless inference and bring-your-own-model deployments. The Principal Engineer leads major architectural programs and sets engineering standards for teams of 20–50+ engineers. This role is a senior technical authority across AI systems, with decisions affecting the cost, latency, and reliability of token serving and post-training workloads. Its scope spans GPU kernel efficiency, fleet-level KV cache and serving architecture, model-quality evaluation, and reinforcement-learning workflows where inference and training share hardware. The Principal Engineer frames technical solutions across the organization, including customer-facing API contracts, and aligns engineering, research, product, and infrastructure leadership on strategy.

Requirements

10–15 years of engineering experience, with a clear track record of pillar-level impact on production AI systems.
At least 4 years of hands-on work with LLM inference, GPU performance, evaluations, or post-training and RL, in production or research.
Demonstrated ability to define multi-year technical strategy for complex, multi-team AI systems organizations.
World-class depth in production LLM inference, GPU performance, evaluations, and/or post-training and RL infrastructure, with strong working knowledge across the other areas.
Demonstrated ownership of the architecture of a large-scale production inference or training platform.
Proven ability to create architectural frameworks and engineering standards adopted across large engineering organizations.
Deep understanding of the hardware/software boundary for AI accelerators, including CUDA or ROCm, memory bandwidth and interconnect constraints, and distributed compute paradigms.
Strong history of growing technical leaders at Staff level and above and multiplying technical capability across teams.
External recognition in the AI systems community through research, open source, or industry contribution.

Nice To Haves

Prior experience at a top-tier AI lab or major hyperscaler AI infrastructure team.
Maintainer or core contributor to a foundational inference, kernel, or RL framework such as vLLM, SGLang, TensorRT-LLM, LMCache, FlashInfer, Triton, verl, OpenRLHF, TRL, DeepSpeed, or Megatron-LM.
Hands-on depth in LLM reinforcement learning, including DPO/GRPO-style methods, reward modelling, multi-turn and tool-use RL, and the interaction between inference and training infrastructure.
Experience defining developer API platforms adopted at scale by external developers.
Deep experience with control plane/data plane architecture and cell-based deployment patterns in large-scale inference infrastructure.
Published work in AI systems at MLSys, NeurIPS Systems Track, OSDI, EuroSys, SC, or equivalent.
Experience with hardware-software co-design, including custom CUDA or Triton kernels, compiler-level optimization, and AI hardware roadmap engagement.
Experience defining pricing, SLO, and capacity models for a commercial inference product.

Responsibilities

Define and own the multi-year technical roadmap for Nscale’s inference, evaluation, and post-training platform, translating it into architecture multiple teams can execute.
Lead company-scale architectural initiatives, including next-generation serving, KV cache orchestration, speculative decoding, multi-tenant scheduling, GPU kernel and model efficiency, evaluation and benchmarking frameworks, and post-training and RL infrastructure.
Establish engineering standards across AI teams for API design and compatibility, benchmarking and evaluation methodology, training stability, and performance testing.
Own the framework for measuring and making cost, latency, throughput, and model-quality trade-offs across the pillar.
Identify and resolve long-horizon systemic risks, including serving-engine and framework choices, accelerator support, and capability gaps.
Align AI engineering, research, product, and infrastructure leadership on multi-team technical strategy, and communicate trade-offs in product and commercial terms.
Mentor and develop Staff and Senior AI Engineers and grow future inference technical leaders.
Represent Nscale’s technical approach externally through open-source leadership, publications, conference talks, and partnerships with GPU vendors and AI labs.