Principal Engineer, Model Performance

Runware

12h • Remote

About The Position

Runware is hiring a principal-level engineer to own inference performance across its AI inference platform and distributed GPU fleet. The role covers diffusion, video generation, speech, 3D, and LLM workloads across more than 400,000 models and tens of thousands of GPUs. The successful candidate will set the performance roadmap, personally implement and optimize the most challenging solutions, and build and lead the Model Performance team as it grows.

The position focuses on delivering production improvements to latency, cost, and quality across models and hardware generations. This includes kernel and model optimization, quantization, serving engines, batching and parallelism strategies, and new-model deployment pipelines. The role works closely with the Sonic Inference Engine, infrastructure and hardware teams, NVIDIA, and open-source engine communities. Success includes rapidly optimizing newly released open models, lowering cost per request over time, and building a team capable of delivering these results independently.

Runware is remote-first, with in-person meetings twice a year. Core hours are maintained for collaboration and calls, while employees set their own schedules outside those hours. Release cycles are described as fast and intense, followed by downtime.

Requirements

• 10+ years of experience in GPU, HPC, or deep learning systems engineering, including several years optimizing production inference at scale.
• A record of shipping performance improvements, including taking a state-of-the-art model from reference implementation to the latency-and-cost Pareto frontier on real hardware.
• Current, hands-on CUDA and GPU architecture expertise, including kernel design, memory hierarchy, occupancy, Tensor Cores, fused kernels, CUDA Graphs, NCCL, and Nsight Systems and Nsight Compute.
• Production experience with at least one major inference engine, such as TensorRT-LLM, vLLM, or SGLang, and modern serving techniques including continuous batching, paged KV cache, speculative decoding, prefill/decode disaggregation, quantization, and tensor, pipeline, and expert parallelism.
• Fluency in Python and C++; comfort with PyTorch internals, torch.compile, and Triton; and willingness to implement benchmarks, kernels, and CI jobs directly.
• Experience building or leading an engineering performance team, hiring team members, setting technical direction, and remaining hands-on.
• Daily use of AI to accelerate work such as kernel search, profiling analysis, benchmarking, documentation, and code review, with a clear perspective on how AI tools should change a small team's operating model.
• Comfort owning outcomes in a fast-moving, ambiguous environment, making decisions with incomplete information, and being accountable for latency, cost, and quality across a large model catalogue.

Nice To Haves

• Diffusion and video generation performance experience, including DiT and UNet pipelines, long-sequence attention, step distillation, caching across denoising steps, LoRA hot-swapping, and ComfyUI, Diffusers, or custom engine loaders and samplers.
• Contributions to open-source inference projects such as TensorRT-LLM, vLLM, SGLang, FlashAttention, CUTLASS, Triton, or Dynamo, or an independently developed open-source project in use by others.
• Experience with Blackwell-generation hardware (B200, GB200/GB300 NVL72), NVLink and NVSwitch topologies, and multi-node inference over InfiniBand or RoCE.
• Mixture-of-experts serving experience, including expert parallelism, placement, load balancing, and associated memory and communication trade-offs.
• Experience evaluating optimized model accuracy using appropriate harnesses, datasets, and statistical methods.
• Experience with non-NVIDIA accelerators such as AMD, TPU, or custom silicon, and porting inference stacks across them.
• Speech and audio model-serving experience, including streaming ASR and TTS with tight latency budgets.
• Published work, patents, or talks on inference performance.

Responsibilities

• Own inference performance end to end across the fleet, including kernels, serving engines, model-level optimizations, and benchmarks for image, video, audio, 3D, and LLM workloads.
• Set the model performance roadmap, prioritizing models, hardware, and techniques according to traffic, cost, and upcoming hardware generations.
• Personally write and tune CUDA kernels, fuse attention and MLP paths, bring up quantization formats while maintaining quality, and implement parallelism and batching strategies on multi-GPU nodes.
• Build a pipeline to turn newly released open models into optimized, production-grade deployments within days.
• Select and shape the serving stack, deciding what to adopt, fork, or develop, and contribute upstream where beneficial.
• Own reproducible, hardware-aware benchmarking and performance regression systems for throughput, latency, cost, and quality, integrated into CI.
• Partner with infrastructure and hardware engineering on GPU memory, storage, and network bottlenecks, and feed findings into future servers and pods.
• Build and lead the Model Performance team by hiring, mentoring, setting technical standards and direction, and enabling a small team to support a large model catalogue.
• Use AI-based agents and automation for kernel search and tuning, benchmark generation, profiling triage, and documentation.

Benefits

• Meaningful stock options.
• Remote-first work with core collaboration hours and flexibility to set one's schedule outside those hours.
• Generous paid time off, including vacation, sick days, and public holidays.
• Paid maternity, paternity, and caregiver leave.
• Twice-yearly company retreats in inspiring locations.
• Release cycles are followed by downtime, which employees are expected to take.

What This Job Offers

• Job Type: Full-time
• Career Level: Principal
• Education Level: No Education Listed