Safety Post-training

Microsoft AI

9d • $119,800 - $261,000

About The Position

Microsoft AI is building AI systems and products that empower people’s lives. Our work is driven by a community of brilliant, interdisciplinary minds working across frontier model development, product engineering, and responsible AI. Within Microsoft AI, the Safety team develops the training methods, evaluations, runtime safeguards, monitoring, and infrastructure needed to make advanced AI systems safer, more reliable, and more useful. Our work spans text, multimodal, and agentic systems and is developed in close partnership with other Research teams, Production Inference, Security, Responsible AI, Microsoft product organizations, and external partners and customers.

We are looking for a Member of Technical Staff to help build the shared systems and methods for safety post-training of frontier text, coding, and agentic models. You will connect data generation and curation, annotation, model and agent evaluation, training experiments, and production feedback into reproducible workflows that improve model behavior at scale. This role is a strong fit for an AI researcher or engineer who combines rigorous evaluation with strong engineering judgment and enjoys creating reproducible experiments and production-ready code.

Requirements

• Bachelor’s degree in Computer Science, Machine Learning, Statistics, Mathematics, Engineering, a related technical field, or equivalent practical experience.
• Strong programming skills in Python.
• Experience training, fine-tuning, evaluating, or researching large language models.
• Experience designing and analyzing rigorous model evaluations, experiments, benchmarks, or measurement systems.
• Knowledge of experimental design and metric selection.
• Experience working with large datasets and building reliable data, evaluation, training, or workflow pipelines.
• Experience with data quality, annotation design, and automated graders.
• Ability to turn ambiguous safety or policy questions into measurable model behaviors and communicate results clearly across research, engineering, product, and policy teams.

Nice To Haves

• Safety post-training methods such as supervised fine-tuning, preference optimization, reinforcement learning from human or AI feedback, reward modeling, process supervision, rejection sampling, or synthetic data generation.
• Reinforcement learning infrastructure, rollout systems, agentic training environments, verifiers, trajectory-level graders, or large-scale distributed model training.
• Evaluation or mitigation of jailbreaks, prompt injection, unsafe tool use, reward hacking, deception, harmful cyber behavior, over-refusal, or other adversarial model behaviors.
• Cybersecurity expertise in areas such as application or cloud security, vulnerability research, penetration testing, secure coding, incident response, identity, or security engineering, with the ability to translate realistic threats into training data and evaluations.
• Experience building or evaluating coding and computer-use agents, instrumented sandboxes or cyber ranges, security-focused verifiers, authorization controls, or defenses against prompt injection and unsafe tool use.
• Statistical methods such as power analysis, Bayesian analysis, resampling, hierarchical models, sequential testing, multiple-comparison correction, causal inference, or measurement science.
• Human-evaluation design, automated-grader calibration, rare or high-severity event measurement, benchmark contamination analysis, subgroup analysis, distribution shift, or evaluator drift.
• Data-generation, annotation, evaluation, experiment-tracking, or model-lifecycle platforms using workflow orchestration, distributed compute, cloud infrastructure, lineage, governance, or production telemetry.
• Pretraining data and objectives, model architecture, interpretability, robustness, privacy, security, trust and safety, or other upstream model-development and responsible AI work.

Responsibilities

• Design and build shared pipelines for safety data collection and generation, evaluation, experimentation, and model feedback loops.
• Develop rigorous evaluation suites, datasets, metrics, and graders for text, coding, and agentic models.
• Define durable schemas and APIs for policy taxonomies, annotations, grader outputs, datasets, tool trajectories, and evaluation results.
• Connect production signals, incidents, red-team findings, and partner feedback to evaluations, training data, model interventions, and deployment decisions.
• Partner with partner and product teams to improve model and system safety.

Benefits

• Certain roles may be eligible for benefits and other compensation.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level