Principal Software Engineer - Model Inferencing

Microsoft • Redmond, WA

2h

About The Position

Help build the end-to-end Generative AI stack for Responsible AI at Microsoft. Lead the development of global, enterprise-scale systems for LLM hosting, inference orchestration, and agentic workflows to ensure AI systems are safe, reliable, and grounded. We are looking for experts in distributed systems and high-performance backend engineering to solve complex challenges in AI alignment, task adherence, and autonomous tool-use at scale.
Microsoft is a company where passionate innovators come together to collaborate, envision new possibilities, and advance their careers. In an AI-first world, we focus on driving meaningful innovation through openness and teamwork.
The CoreAI organization builds the fundamental AI backbone for Microsoft’s flagship products, including GitHub, Office, Teams, and Windows. Within this group, the Responsible AI team is at the forefront of Azure AI innovation, building the global infrastructure required to run the world's largest AI workloads safely. Our mission is to identify, measure, and mitigate risks across all content modalities.
As a Principal Software Engineer, you will join a talented multidisciplinary team of engineers, scientists, and product managers to build industry-leading AI services. You will architect the infrastructure required for agent workflow management and real-time safety guardrails. This role is central to hosting and orchestrating models at scale, integrating deeply with platforms such as Microsoft Foundry, Azure AI Content Safety, and Azure OpenAI Service.

Requirements

• Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.
• Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
• Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Nice To Haves

• Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
• Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.
• Proven experience in building high-scale distributed systems and high-availability services.
• Deep understanding of the AI lifecycle, specifically regarding model inference and system-level optimization.

Responsibilities

• Own and lead the architecture for complex, high-availability AI services, ensuring scalability, resiliency, and low-latency performance.
• Improve AI tools and practices across the SDLC, incorporating Responsible AI controls into the system backbone.
• Lead the integration of new AI services with existing platforms such as Microsoft Foundry, Azure AI Content Safety, and Azure OpenAI Service.
• Mentor teams in producing extensible, secure, and maintainable code. Identify best practices in GenAI coding patterns and drive high-quality validation strategies.
• Identify and manage upstream/downstream dependencies, collaborating with partner teams to ensure seamless end-to-end testing and live site coverage.
• Act as a lead for security-by-design, ensuring AI safety features are implemented and regulatory audit trails are maintained.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Number of Employees: 5,001-10,000 employees