Software Engineer Intern, Inference

Rhoda AI•Mountain View, CA

About The Position

Rhoda AI is hiring an Inference Infrastructure Engineer Intern to improve the speed and efficiency of inference for its foundation world models. The intern will work with infrastructure and research engineers on a scoped, high-impact project spanning the inference stack, from data pipelines to GPU kernels. The project will be delivered into a production or research workflow, with results presented to the team. The work focuses on enabling models to run quickly enough for real-world robot control and efficiently enough to support large-scale research. The intern will profile and optimize large video and world models, improve data movement and processing, develop or tune GPU kernels, and build reliable performance tooling and benchmarks. They will also collaborate with researchers to turn promising ideas into production-quality infrastructure.

Requirements

Currently pursuing a BS, MS, or PhD in Computer Science, Electrical or Computer Engineering, or a related field.
Strong programming skills in Python and/or C++.
Solid understanding of deep learning fundamentals and experience with at least one framework; PyTorch is preferred.
Comfort reading performance data and reasoning about trade-offs among latency, throughput, memory, and accuracy.
Depth in at least one of the following areas: ML inference optimization, data efficiency optimization, or GPU kernel optimization.
For ML inference optimization: hands-on experience improving deep learning model speed or cost, such as through quantization (INT8/FP8), pruning or distillation, operator fusion, mixed precision, or compilers and runtimes such as torch.compile, TensorRT, ONNX Runtime, or TVM.
For data efficiency optimization: experience accelerating data-loading and preprocessing pipelines, particularly video or other high-bandwidth sensor data, reducing I/O and memory overhead, or improving data and compute utilization in ML systems.
For GPU kernel optimization: experience writing or tuning GPU kernels in CUDA, Triton, CUTLASS, or similar; familiarity with GPU memory hierarchy, occupancy, and profiling tools such as Nsight Systems/Compute or PyTorch profiler.

Nice To Haves

Experience with video model inference, such as optimizing video generation, video diffusion, world models, or real-time video understanding pipelines.
Research experience in systems or ML, such as publications or substantial research projects at MLSys, OSDI/SOSP, CVPR/ICCV, NeurIPS/ICML/ICLR, or similar.
Experience with real-time or latency-critical inference, such as on robots or edge/embedded accelerators.
Experience with multi-GPU inference, including tensor, sequence, or pipeline parallelism.
Contributions to open-source ML systems projects.

Responsibilities

Profile and benchmark inference for large video and world models to identify latency, throughput, and memory bottlenecks.
Apply and evaluate inference optimization techniques, including quantization, low-precision inference, step distillation, sampling acceleration for diffusion-style models, attention and caching optimizations across frames, model compilation, and graph-level optimizations.
Improve data movement and processing, including video decoding and preprocessing, input pipelines, host–device transfer, caching, and storage and I/O paths that feed inference and training.
Write and tune GPU kernels using CUDA, Triton, or similar tools, and integrate them into the inference stack.
Build tooling and benchmarks to track performance regressions and improvements reliably.
Work with researchers to turn promising ideas into production-quality infrastructure.
Own a scoped, high-impact project end to end, ship it into a production or research workflow, and present results to the team.