Software Development Engineer - Testing Infrastructure & Performance, Trainium/Neuron, Annapurna Labs
About The Position
The Neuron team within Annapurna ML develops the software stack that powers AWS’s custom AI training and inference chips (Trainium, Inferentia). Our team builds and operates the testing infrastructure and performance characterization systems that ensure every release meets quality and performance bars before reaching customers. We directly enable the velocity and reliability of Neuron software releases. We are looking for a Software Development Engineer to own and evolve the testing infrastructure and performance pipeline that validates Neuron software across workloads, models, and hardware configurations. You will build data pipelines that collect, process, and surface performance metrics — and dashboards that give engineers and leadership real-time visibility into regression detection, performance trends, and release readiness. Your work directly impacts how fast we ship high-quality software to customers running ML workloads on Trainium.
Requirements
- Bachelor's degree or above in computer science or equivalent
- Experience with Python in a production or infrastructure context
- Experience building data pipelines (ETL, batch/streaming processing, data warehousing)
- Experience with testing frameworks and test automation at scale
Nice To Haves
- Master's degree in computer science, machine learning, engineering, or related fields, or experience with Machine and Deep Learning toolkits such as MXNet, TensorFlow, Caffe and PyTorch
- Experience building performance dashboards or observability tooling (Grafana, QuickSight, CloudWatch, or similar)
- Experience with distributed systems testing or hardware-in-the-loop test infrastructure
- Experience with AWS services (S3, DynamoDB, Lambda, Step Functions, ECS)
- Familiarity with statistical methods for regression detection or performance analysis
Responsibilities
- Design, build, and maintain automated testing infrastructure (Python) that validates Neuron compiler, runtime, and framework integrations across hardware targets
- Build and operate data pipelines that ingest performance benchmarks, test results, and system metrics from distributed test runs into centralized data stores
- Develop performance dashboards that surface regression detection, trend analysis, and release-readiness signals to engineering teams and leadership
- Create and maintain integration test frameworks that validate end-to-end model compilation, execution, and performance across Trainium configurations
- Build automated alerting and triage tooling that identifies performance regressions early and routes them to the right owners
- Collaborate with Compiler, Runtime, and Training teams to define performance benchmarks, SLAs, and quality gates for release criteria
- Extend testing infrastructure to support new model architectures, training workflows, and hardware generations as they come online
Benefits
- health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage)
- 401(k) matching
- paid time off
- parental leave
- sign-on payments
- restricted stock units (RSUs)
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
