Member of Technical Staff, LLM Infrastructure

Fireworks • San Mateo, CA

2d

About The Position

As a Software Engineer on Fireworks AI’s Infrastructure team, you will help design and build the core systems powering its generative AI platform. The role focuses on infrastructure and tools that support system reliability, performance, quality, and availability. You will work closely with cloud infrastructure, product, performance, and training teams to bridge customer needs with Fireworks’ inference engine and deliver a reliable, user-friendly generative AI platform.

Requirements

• Bachelor’s degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.
• At least 3 years of software engineering experience focused on infrastructure or machine learning systems.
• Strong programming skills in Python, Go, or a similar language.
• Proven experience with machine-learning infrastructure and tooling, such as PyTorch, MLflow, Vertex AI, SageMaker, or Kubernetes.
• Basic understanding of LLM concepts such as context length, disaggregated prefill, and KV cache memory estimation.

Nice To Haves

• 5 or more years of software engineering experience focused on infrastructure or machine learning systems.
• Experience with open-source inference engines such as vLLM, SGLang, or TRT-LLM.
• Contributions to open-source infrastructure or machine-learning projects.
• Experience building large-scale ML/MLOps infrastructure.

Responsibilities

• Contribute to the design and development of scalable backend infrastructure for distributed training, inference, and data pipelines.
• Build and maintain core backend services, including the LLM CI/CD pipeline, control plane, and model-serving systems.
• Improve performance, cost efficiency, and reliability across compute, storage, and networking layers.
• Build frameworks and safeguards to support high model quality.
• Collaborate with performance, training, and product teams to translate research and product needs into infrastructure solutions.
• Participate in code reviews, technical discussions, and continuous integration and deployment processes.

Benefits

• Opportunities to solve challenging AI infrastructure problems and work with bleeding-edge technology.
• Ownership and direct impact in a fast-growing team.
• Collaboration with engineers and AI researchers.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level