Member of Technical Staff, Applied AI Engineer

Microsoft•Mountain View, CA

12h

About The Position

Overview We’re hiring a Applied AI Engineer to join a fast‑moving, high‑ownership team building next‑generation AI assistant and productivity capabilities. This role blends LLM product engineering, evaluation science, hillclimbing, and internal tool building with the pace and creativity of a startup. You’ll work across the entire lifecycle of features from early prototypes to production‑grade systems and help define how millions of users interact with AI. Responsibilities LLM Feature & Agent Development Design and ship LLM‑powered assistant features , including conversational flows, agentic behaviors, retrieval pipelines, and multimodal interactions. Build prompt architectures , system instructions, and orchestration logic that ensure reliability, grounding, and personality consistency. Prototype new capabilities rapidly and iterate based on user signals and evaluation data. Evaluation, Hillclimbing & Quality Systems Build and maintain evaluation frameworks for correctness, safety, grounding, and UX quality. Run hillclimbing loops across prompts, models, and tool‑use strategies to continuously improve assistant performance. Analyze failure modes, design mitigations, and drive systematic improvements across the stack. LLM Tooling & Internal Infrastructure Develop internal tools for prompt experimentation, model comparison telemetry and debugging automated eval pipelines Create reusable frameworks that accelerate the entire AI org’s ability to ship high‑quality assistant features. Applied ML & Product Integration Integrate LLMs with product surfaces, APIs, and backend systems. Build lightweight ML components (ranking, classification, summarization, personalization) that enhance assistant intelligence. Collaborate with PM, design, and research to turn ambiguous ideas into polished user experiences. High‑Velocity Teamwork Operate with startup‑founder energy : bias for action, rapid iteration, and comfort with ambiguity. Work closely with researchers, engineers, and product leaders in a fast-moving AI team where ideas ship quickly and impact is immediate. Contribute to a culture of experimentation, clarity, and high-quality execution. Build prompt architectures , system instructions, and orchestration logic that ensure reliability, grounding, and personality consistency. Prototype new capabilities rapidly and iterate based on user signals and evaluation data.

Requirements

Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 1+ year(s) data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) or consulting experience OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 2+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) OR equivalent experience.

Nice To Haves

Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 1+ year(s) data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 3+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) OR equivalent experience.
2+ years shipping production-level code, models, or data analysis.
1+ years using AI-assisted coding and analysis techniques.
Experience working on small teams and mid-stage startup environments.
Experience working on AI products.
PhD in engineering, applied math, statistics, or related analytical field.
4+ years shipping production-level code, models, or data analysis.
Deep experience building from zero-to-one.
Hands on work hillclimbing AI evaluations.

Responsibilities

LLM Feature & Agent Development Design and ship LLM‑powered assistant features , including conversational flows, agentic behaviors, retrieval pipelines, and multimodal interactions.
Build prompt architectures , system instructions, and orchestration logic that ensure reliability, grounding, and personality consistency.
Prototype new capabilities rapidly and iterate based on user signals and evaluation data.
Evaluation, Hillclimbing & Quality Systems Build and maintain evaluation frameworks for correctness, safety, grounding, and UX quality.
Run hillclimbing loops across prompts, models, and tool‑use strategies to continuously improve assistant performance.
Analyze failure modes, design mitigations, and drive systematic improvements across the stack.
LLM Tooling & Internal Infrastructure Develop internal tools for prompt experimentation, model comparison telemetry and debugging automated eval pipelines Create reusable frameworks that accelerate the entire AI org’s ability to ship high‑quality assistant features.
Applied ML & Product Integration Integrate LLMs with product surfaces, APIs, and backend systems.
Build lightweight ML components (ranking, classification, summarization, personalization) that enhance assistant intelligence.
Collaborate with PM, design, and research to turn ambiguous ideas into polished user experiences.
High‑Velocity Teamwork Operate with startup‑founder energy : bias for action, rapid iteration, and comfort with ambiguity.
Work closely with researchers, engineers, and product leaders in a fast-moving AI team where ideas ship quickly and impact is immediate.
Contribute to a culture of experimentation, clarity, and high-quality execution.
Build prompt architectures , system instructions, and orchestration logic that ensure reliability, grounding, and personality consistency.
Prototype new capabilities rapidly and iterate based on user signals and evaluation data.