Principal Software Development Engineer, AWS Mantle
About The Position
We are seeking a Principal Software Development Engineer to join the AWS Mantle team and drive the technical vision for our distributed inference engine that serves millions of customers across Amazon Bedrock. In this role, you will define and execute on large-scale, ambiguous technical challenges at the intersection of machine learning systems, distributed computing, and security—shaping how the world accesses foundation models. As a Principal SDE on the Mantle team, you will serve as the technical conscience and strategic thought leader for one of AWS's most critical AI infrastructure platforms. You will architect solutions that are reliable, scalable, and secure—operating at the cutting edge of distributed systems where millisecond-level latency and zero-trust security are non-negotiable.
Requirements
- 10+ years of non-internship professional software development experience
- 10+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience
- Bachelor's degree in Computer Science, Engineering, a related field, or equivalent experience
- 8+ years of programming experience with at least one modern language such as Java, C++, Python, Go, or Rust
- Experience driving cross-organizational technical strategy and delivering results in complex, ambiguous environments where the business problem and technical approach are not pre-defined
Nice To Haves
- Master's degree or equivalent in computer science, machine learning, engineering, or related fields, or PhD
- Experience building large-scale machine learning and AI solutions at Internet scale
- Experience working with Advanced Compute technologies including, but not limited to: Accelerated Compute, High Performance Compute, Visual/Spatial Compute, and/or IoT.
- Experience writing and publishing technical documents or equivalent
- Familiarity with inference frameworks such as vLLM, TensorRT, or Triton Inference Server
Responsibilities
- Set the long-term technical direction for a globally distributed, high-performance ML inference platform serving models from industry-leading AI providers
- Own end-to-end system design decisions that directly impact latency, reliability, and scalability for millions of customers worldwide
- Influence engineering strategy across Amazon Bedrock, partnering with senior leadership to align technical investments with business outcomes
- Raise the engineering bar through exemplary system design, mentorship, and contributions to the broader AWS engineering community
- Navigate complex trade-offs across performance, security, and cost while maintaining the highest standards for operational excellence
- Design and evolve the architecture of Mantle's distributed inference engine, including capacity management, model onboarding pipelines, and quality-of-service controls
- Drive cross-organizational initiatives spanning multiple AWS teams to deliver seamless, OpenAI-compatible API experiences with Zero Operator Access (ZOA) security guarantees
- Lead technical strategy for scaling inference to support rapid onboarding of new foundation models while maintaining global availability and performance SLAs
- Author and champion technical vision documents, influence product roadmaps, and represent the team in executive-level architectural reviews
- Mentor and develop senior engineers, fostering a culture of engineering excellence, innovation, and customer obsession
Benefits
- health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage)
- 401(k) matching
- paid time off
- parental leave
- sign-on payments
- restricted stock units (RSUs)
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Principal
