Backend Software Engineer (ML Infra)

Rockstar • San Francisco, CA

18h • Onsite

About The Position

The client is a fast-growing AI infrastructure startup building an inference allocation engine that combines model routing, dedicated inference, and post-training to serve each request with the best model in real time. Its routers and models allocate and serve more than 100 billion tokens per month for frontier AI companies across industries.

The Software Engineer will design, build, and scale core systems for large-scale routing, model training, and deployment. The work spans distributed training pipelines, cloud-native infrastructure, internal developer platforms, and frontend and backend features supporting fine-tuning, reinforcement learning, and inference at scale. The role sits at the intersection of product engineering and ML systems, involves close collaboration with ML scientists, and includes ownership of production-grade infrastructure operating at 99.99% uptime. It is presented as an opportunity for an early-career engineer to work with distributed systems, GPU workloads, and modern ML infrastructure.

Requirements

• 1–3 years of engineering experience, ideally working on production systems at the backend and infrastructure layer.
• Strong fundamentals in distributed systems, networking, and backend architecture.
• Comfortable writing code with and without AI and taking accountability for the code.
• Excited to work on-site in San Francisco with a fast-moving early-stage team.

Responsibilities

• Design and implement systems supporting large-scale ML workloads, including fine-tuning and reinforcement learning.
• Build efficient, fault-tolerant, and observable distributed training, inference, and logging pipelines.
• Work on cloud-native systems using containers and orchestration such as Kubernetes.
• Optimize systems for performance, reliability, and cost efficiency, particularly for GPU-heavy workloads.
• Implement monitoring, logging, and observability for long-running training jobs and production services.
• Speak with customers, identify what is slowing them down, and deliver fixes.
• Develop frontend and backend logic that enables customers to use the client’s capabilities.
• Collaborate closely with ML scientists and take ownership of production-grade infrastructure.

What This Job Offers

• Job Type: Full-time
• Career Level: Entry Level
• Education Level: No Education Listed