Senior Applied Scientist - Outlook Science Team

Microsoft • Redmond, WA

4d • $119,800 - $261,000

About The Position

The Outlook team at Microsoft builds intelligent experiences across Mail and Calendar, including Copilot-powered drafting, summarization, search, inbox prioritization, meeting preparation, scheduling, and agentic workflows. Its work spans product analytics, experimentation, telemetry, machine learning, generative AI, and large language model evaluation to improve customer value, quality, trust, and adoption.

As a Senior Applied Scientist on the Outlook Science Team, you will advance Outlook Copilot in large language models, evaluation, relevance, and Responsible AI. You will develop end-to-end evaluation infrastructure and measurement frameworks, collaborate across organizations and stakeholder groups, and apply data science and AI expertise to guide decisions. The role includes leading evaluation of LLM systems and their components, shaping product experiences through findings, and contributing research on LLM evaluation and user experience.

Requirements

• Bachelor's degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or a related field and 4+ years of related experience, such as statistics, predictive analytics, or research; OR a master's degree in one of these fields and 3+ years of related experience; OR a doctorate in one of these fields and 1+ year of related experience; OR equivalent experience.
• Ability to meet Microsoft, customer, and/or government security screening requirements, including passing the Microsoft Cloud Background Check upon hire or transfer and every two years thereafter.

Nice To Haves

• Ph.D. or master's degree in a relevant field, such as Data Science, Computer Science, Social Science, or HCI.
• Experience in data science, machine learning, experimentation, and AI, with a track record of delivering impactful results.
• Experience in program management and leading cross-functional teams.
• Experience with LLM fine-tuning, reinforcement learning, evaluation techniques, RAG implementation, agentic workflows, and industry best practices.
• Development experience in Python.
• Analytical, problem-solving, and presentation skills.
• Understanding of Responsible AI principles.
• Ability to work in a fast-paced and dynamic environment.

Responsibilities

• Develop and execute a strategy for LLM evaluation covering quality, cost, model performance, user experience, prompt effectiveness, and Responsible AI, informed by emerging research and aligned with company-wide efforts.
• Propose and drive cost-effective solutions, including small language models, fine-tuned models, and ensemble models.
• Manage large-scale, cross-functional evaluation programs and align them with organizational objectives and timelines.
• Develop and maintain a measurement framework to track and report LLM performance and user impact.
• Drive the engineering product roadmap for automated evaluation pipelines integrated into product workflows.
• Design experiments, analyze data, create OKRs and metrics, and derive actionable insights to enhance LLM systems.
• Influence product and user experience decisions based on evaluation results.
• Lead assessment and improvement of language models and agents, including iterative enhancements and synthetic or manufactured data creation.
• Collaborate with User Experience teams to evaluate and optimize AI interactions and improve user satisfaction.
• Implement Responsible AI and DSB principles and guidelines in AI systems, supporting ethical and unbiased model development and deployment.
• Form partnerships and lead research initiatives in LLM evaluation and user experience optimization that contribute to scientific knowledge and understanding of user mental models and alignment with LLM-powered experiences.
• Collaborate with engineering, research, product, and other teams to integrate evaluation into the development lifecycle.
• Communicate findings and recommendations to executive leadership and foster a data-driven culture.

Benefits

• Certain roles may be eligible for benefits and other compensation; additional benefits and pay information is available through the Microsoft careers website.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior