Senior Software Engineer, Annapurna Labs

Amazon • Seattle, WA

2d • $168,100 - $227,400 • Onsite

About The Position

Annapurna Labs, now integrated into AWS, is a key infrastructure provider within AWS, covering disciplines like silicon engineering, hardware design and verification, software, and operations. The Annapurna Labs Neuron team focuses on the complete software stack for AWS Inferentia and Trainium machine learning accelerators. This role is for a Senior Software Engineer who will lead the development of machine learning tools for running, optimizing, and analyzing ML workloads. The ideal candidate will have experience leading ML tool projects from architecture through multiple delivery generations, with deep knowledge in profiling, optimization, resource management, scheduling, and code generation. Experience with new instruction set architectures (CPU, NPU, GPU, etc.) is preferred.

Requirements

• 5+ years of non-internship professional software development experience
• 5+ years of programming with at least one software programming language experience
• 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience
• 5+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
• Experience as a mentor, tech lead or leading an engineering team

Nice To Haves

• Experience leading machine learning tool projects, preferably starting from architecture through several generations of delivery to customers.
• Deep knowledge of profiling and optimization, resource management, scheduling, code generation.
• Worked on new instruction set architectures, which may include CPU, NPU, GPU and other forms of compute.
• Bachelor's degree in computer science or equivalent

Responsibilities

• Lead the design, development, and operation of a next-generation AI workload orchestration platform built on Kubernetes.
• Enable scalable job scheduling, multi-node distributed training, and artifact management across Annapurna Labs' custom ML silicon (Inferentia and Trainium families).
• Drive architectural decisions across the full stack, from service APIs and workflow orchestration to cluster-level scheduling and resource optimization, targeting high availability and hardware utilization.
• Collaborate with ML researchers, SDK developers, hardware design teams, and infrastructure engineers to ensure platform evolution aligns with current and next-generation AI accelerators.
• Mentor and technically guide a team of software engineers, setting standards for operational excellence, system design, and developer experience.
• Architect and implement scalable services spanning Kubernetes operators, workflow orchestration, and scheduling algorithms to maximize hardware utilization.
• Own the full lifecycle of systems, from design and implementation through deployment, monitoring, and incident response.
• Improve developer velocity by refining CLI tools, APIs, and framework integrations.
• Participate in design reviews, code reviews, and operational readiness discussions.
• Work in a fast-paced, startup-like environment driven by the AI hardware roadmap.

Benefits

• health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage)
• 401(k) matching
• paid time off
• parental leave
• sign-on payments
• restricted stock units (RSUs)

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Associate degree