Software Engineer Intern, Inference

Rhoda AI•Mountain View, CA

About The Position

At Rhoda AI, the Inference Infrastructure Engineer Intern will work with infrastructure and research engineers to make foundation world-model inference faster and more efficient across the stack, from data pipelines to GPU kernels. The intern will own a scoped, high-impact project end to end, ship it into a production or research workflow, and present results to the team. The work supports models that must run quickly enough to control robots in real-world environments and efficiently enough to support large-scale research.

Requirements

Currently pursuing a BS, MS, or PhD in Computer Science, Electrical or Computer Engineering, or a related field.
Strong programming skills in Python and/or C++.
Solid understanding of deep learning fundamentals and at least one deep learning framework; PyTorch is preferred.
Comfort reading performance data and reasoning about trade-offs between latency, throughput, memory, and accuracy.
Strong depth in at least one of the following areas: ML inference optimization, data efficiency optimization, or GPU kernel optimization.
For ML inference optimization: hands-on experience making deep learning models faster or cheaper, such as through quantization (INT8/FP8), pruning or distillation, operator fusion, mixed precision, or compilers and runtimes such as torch.compile, TensorRT, ONNX Runtime, or TVM.
For data efficiency optimization: experience speeding up data loading and preprocessing pipelines, especially for video or other high-bandwidth sensor data; reducing I/O and memory overhead; or improving data and compute utilization in ML systems.
For GPU kernel optimization: experience writing or tuning GPU kernels in CUDA, Triton, CUTLASS, or similar; familiarity with GPU memory hierarchy, occupancy, and profiling tools such as Nsight Systems/Compute or PyTorch profiler.

Nice To Haves

Experience with video-model inference, such as optimizing video generation, video diffusion, world models, or real-time video-understanding pipelines.
Research experience in systems or machine learning, such as publications or substantial projects at MLSys, OSDI/SOSP, CVPR/ICCV, NeurIPS/ICML/ICLR, or similar venues.
Experience with real-time or latency-critical inference, such as on robots or edge/embedded accelerators.
Experience with multi-GPU inference, including tensor, sequence, or pipeline parallelism.
Contributions to open-source ML systems projects.

Responsibilities

Profile and benchmark inference for large video and world models to identify latency, throughput, and memory bottlenecks.
Apply and evaluate inference optimization techniques, including quantization and low-precision inference, step distillation and sampling acceleration for diffusion-style models, attention and caching optimizations across frames, model compilation, and graph-level optimizations.
Improve data movement and processing, including video decoding and preprocessing, input pipelines, host–device transfer, caching, and storage and I/O paths feeding inference and training.
Write and tune performance-critical GPU kernels using CUDA, Triton, or similar technologies, and integrate them into the inference stack.
Build tooling and benchmarks to reliably track performance regressions and improvements.
Work with researchers to turn promising ideas into production-quality infrastructure.
Own a scoped project end to end, ship it into a production or research workflow, and present results to the team.