Principal Engineer, Model Performance

Runware • San Jose, CA

15h • Remote

About The Position

Runware is hiring a Principal Engineer, Model Performance to own the speed and cost of inference across its fleet, spanning diffusion and video generation, speech, 3D, and large language models across more than 400,000 models and tens of thousands of GPUs. The role sets the performance roadmap, personally delivers difficult optimization work, and builds and leads the Model Performance team.

The successful candidate will identify and deliver performance improvements across models and hardware generations, from kernel and quantization work to serving, batching, and parallelism strategies, and bring those improvements into production on real traffic. The role partners with the Sonic Inference Engine, infrastructure, and hardware teams, as well as NVIDIA and open-source engine communities. Success includes quickly optimizing newly released open models, continually reducing cost per request, and building a team that can deliver without depending on the role holder's involvement in every task.

Runware describes itself as a remote-first collective, with in-person meetings twice a year, core hours for collaboration and calls, and flexibility to set one's schedule outside those hours. Release cycles are fast and intense, followed by downtime.

Requirements

• 10+ years of experience in GPU, HPC, or deep learning systems engineering, including several years optimizing inference performance for production models at a company operating at scale.
• A track record of shipping performance improvements, including taking a state-of-the-art model from reference implementation to the Pareto frontier of latency and cost on real hardware.
• Current, hands-on CUDA and GPU architecture expertise, including kernel design, memory hierarchy, occupancy, Tensor Cores, fused kernels, CUDA Graphs, NCCL, and Nsight Systems and Compute.
• Production experience with at least one major inference engine such as TensorRT-LLM, vLLM, or SGLang, and experience with modern serving techniques such as continuous batching, paged KV cache, speculative decoding, prefill/decode disaggregation, quantization, and tensor, pipeline, and expert parallelism.
• Fluency in Python and C++, familiarity with PyTorch internals, torch.compile, and Triton, and willingness to personally write benchmarks, kernels, and CI jobs.
• Experience leading engineers, including building or running a performance team, hiring, setting technical direction, and remaining hands-on.
• Regular use of AI to accelerate work such as kernel search, profiling analysis, benchmarking, documentation, and code review, with a clear perspective on how AI tools should change a small team's operations.
• Ability to own outcomes in a fast-moving, ambiguous environment, make decisions with incomplete information, and be accountable for latency, cost, and quality across a large model catalogue.

Nice To Haves

• Diffusion and video generation performance experience, including DiT and UNet pipelines, long-sequence attention, step distillation, caching across denoising steps, LoRA hot-swapping, and loaders or samplers in ComfyUI, Diffusers, or custom engines.
• Contributions to open-source inference projects such as TensorRT-LLM, vLLM, SGLang, FlashAttention, CUTLASS, Triton, or Dynamo, or ownership of a widely used open-source project.
• Experience with Blackwell-generation hardware such as B200, GB200/GB300 NVL72, NVLink and NVSwitch topologies, and multi-node inference over InfiniBand or RoCE.
• Mixture-of-experts serving experience, including expert parallelism, placement and load balancing, and associated memory and communication trade-offs.
• Experience evaluating accuracy of optimized models using suitable harnesses, datasets, and statistical methods to establish equivalence to original models.
• Experience with non-NVIDIA accelerators such as AMD, TPU, or custom silicon, and porting inference stacks across them.
• Speech and audio model serving experience, including streaming ASR and TTS with tight latency budgets.
• Published work, patents, or talks on inference performance.

Responsibilities

• Own inference performance end to end across kernels, serving engines, model-level optimizations, and benchmarks for image, video, audio, 3D, and LLM workloads on the GPU fleet.
• Set the model performance roadmap, prioritizing models, hardware, and techniques based on real traffic, cost, and future pod requirements.
• Personally write and tune CUDA kernels; fuse attention and MLP paths; introduce quantization formats including FP8, FP4, and INT4 while maintaining quality; and develop parallelism and batching strategies for multi-GPU nodes.
• Build a pipeline to take newly released open models from weights to production-grade optimized deployment within days.
• Select and shape the serving stack, including deciding what to adopt, fork, or build, and contribute upstream where useful.
• Own reproducible, hardware-aware benchmarking and performance regression systems for throughput, latency, cost, and quality, integrated into CI.
• Partner with infrastructure and hardware engineering on GPU memory, storage, and network bottlenecks, and relay findings into future server and pod designs.
• Build and lead the Model Performance team, including hiring principal and senior engineers, setting technical standards and direction, and mentoring team members.
• Use AI tools and agents to accelerate kernel search and tuning, benchmark generation, profiling triage, and documentation.

Benefits

• Meaningful stock options.
• Remote-first work arrangement with core collaboration hours and flexibility to set one's schedule outside those hours.
• Generous paid time off, including vacation, sick days, and public holidays.
• Paid maternity, paternity, and caregiver leave.
• Twice-yearly company retreats in inspiring locations.
• Fast and intense release cycles followed by downtime, which employees are expected to take.

What This Job Offers

• Job Type: Full-time
• Career Level: Principal
• Education Level: No Education Listed