The Principal Applied Scientist will work in Microsoft’s CoreAI Foundry observability group, advancing scientific methods for AI agents and agent-based systems. The role focuses on defining how agent quality is measured, diagnosed, and improved across offline evaluation, online signals, and production telemetry, and translating that science into practical capabilities for Foundry customers. The position leads science for agent and multi-agent systems, including offline, online, and continuous evaluation and agent-improvement methods tied to real product behavior. It develops measurement frameworks to explain regressions, detect failure modes, and support confident product and release decisions. The scientist also partners with engineering, product, and platform teams to operationalize evaluation and observability across Foundry, Azure Monitor, and agent runtimes, while setting scientific direction, mentoring colleagues, and influencing research-informed product decisions.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Job Type
Full-time
Career Level
Principal