Software Dev Engineer, EC2 Nitro

Amazon • Seattle, WA

1d

About The Position

Join the EC2 Nitro Machine Learning Systems team to revolutionize accelerated
computing in the cloud. We're seeking an exceptional Software Development
Engineer to build and optimize the performance measurement infrastructure for
some of the most computationally intensive AI/ML workloads on AWS. In this role,
you'll establish EC2 as the definitive source for best-known-configurations
across diverse ML applications including LLMs, multimodal models, and video
generation workloads. Your expertise will directly influence future platform
designs by translating performance insights from state of the art research and
customer workloads into technical requirements for upcoming accelerated platform
launches.

Your impact will extend from low-level systems (CUDA, EFA, firmware) through ML
frameworks to serving layers, requiring deep technical knowledge and the ability
to communicate complex performance data as actionable business insights. This
position offers the unique opportunity to shape the future of machine learning
infrastructure at cloud scale while working at the intersection of
high-performance computing, distributed systems, and machine learning
technologies.

Your typical day begins with reviewing performance data from overnight benchmark
runs across various ML frameworks and hardware configurations. You'll
investigate anomalies, collaborate with the team on optimization opportunities,
and join design reviews to influence future platform capabilities. You'll
balance your time between building measurement infrastructure, analyzing
performance trends, and documenting best practices to help customers optimize
their workloads.

The EC2 Nitro Machine Learning Systems team is responsible for development,
operations, and maintenance of scale-out machine learning platforms used for
training and inference workloads. We build and optimize the infrastructure that
powers some of the most computationally intensive AI/ML workloads in the cloud.
Our team is passionate about creating reliable, high-performance systems that
enable customers to push the boundaries of what's possible with machine
learning.

Working with us means having the opportunity to influence the future of
supercomputing in the cloud while solving complex technical challenges at
massive scale. We collaborate closely with customers and internal teams to
continuously improve our platforms and deliver innovations that accelerate
machine learning workflows.

Requirements

• 3+ years of non-internship professional software development experience
• 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
• Experience programming with at least one software programming language
• Knowledge of Machine Learning and LLM fundamentals, including transformer architecture, training/inference lifecycles, and optimization techniques

Nice To Haves

• 3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
• Bachelor's degree in computer science or equivalent
• Knowledge of ML frameworks including JAX, PyTorch, vLLM, SGLang, Dynamo, TorchXLA, and TensorRT
• Knowledge of machine learning model architecture and inference

Responsibilities

• Design and build foundational infrastructure for ML performance measurement that scales with business demand and operates as reliable CI/CD systems, ensuring high-quality implementations that balance customer requirements with operational excellence
• Develop comprehensive regression test coverage across all major component releases including frameworks, firmware, drivers, and networking technologies to maintain optimal platform performance
• Collaborate with cross-functional teams to establish EC2 as the definitive source for best-known-configurations across diverse ML applications including LLMs, multimodal models, and MoE architectures
• Document and communicate performance insights to influence future platform designs by translating technical findings from research and customer workloads into actionable recommendations
• Identify and resolve complex performance challenges through systematic analysis of training and inference performance KPIs across accelerated platforms, working directly with customers to improve their ML system efficiency

Benefits

• health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage)
• 401(k) matching
• paid time off
• parental leave

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level