Senior and/or Principal Software Engineer - AI Infrastructure
About The Position
AI Infra builds foundational platforms, services, programming models, and developer experiences that power applications using Generative AI. Its platforms support customer AI workloads and products across Microsoft and enterprise customers. The Senior and/or Principal Software Engineer will define engineering systems and developer productivity, building high-scale distributed services for large-scale state-of-the-art model training and inference. The role focuses on machine-learning services and highly distributed platforms designed for continuous availability and tight SLAs, while delivering comprehensive solutions for customers. The engineer will design and develop core AzureML compute services and AI infrastructure for large-scale AI training and inference. They will develop, test, and maintain backend services, operate training and inference frameworks and machine-learning systems, and improve stability, efficiency, maintainability, latency, and cloud security. The role includes product operational support and direct responsible individual (DRI) duties, collaboration with engineers, data scientists, Microsoft Research teams, and external enterprises, and providing technical vision and leadership to teammates. A Microsoft Cloud Background Check is required upon hire or transfer and every two years thereafter.
Requirements
- Bachelor's degree in Computer Science or a related technical field and at least 4 years of technical engineering experience coding in languages such as C, C++, C#, Java, JavaScript, or Python, or equivalent experience.
- Ability to meet Microsoft, customer, and/or government security screening requirements, including passing the Microsoft Cloud Background Check upon hire or transfer and every two years thereafter.
Nice To Haves
- Master's degree in Computer Science or a related technical field and at least 8 years of technical engineering experience coding in languages such as C, C++, C#, Java, JavaScript, or Python; or a bachelor's degree in the field and at least 12 years of such experience; or equivalent experience.
- Knowledge of and hands-on experience with large-model training systems.
- At least 3 years of experience operating and designing systems that support large-model training.
- Experience in distributed systems and architecture.
- Knowledge of AI infrastructure and workloads.
Responsibilities
- Architect, design, and develop core compute services powering AzureML for AI workloads using state-of-the-art LLMs and open-source models.
- Architect, design, and develop core AI infrastructure services supporting large-scale AI training and inference.
- Develop, test, and maintain backend services in C#, Go, Rust, C++, and other applicable languages, hosted on Kubernetes clusters and Docker containers.
- Develop and operate training and inference infrastructure frameworks and machine-learning systems.
- Enhance systems and applications to improve stability, efficiency, maintainability, latency, and cloud security.
- Provide operational support and DRI responsibilities for the product.
- Collaborate with team engineers, data scientists, Microsoft Research teams, and external enterprises to build solutions.
- Provide vision, expertise, and technical leadership to other team members.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
