Model Policy Manager, Emerging AI Misuse

OpenAI•San Francisco, CA
•$207,000 - $335,000•Hybrid

About The Position

The Model Policy Manager, Emerging AI Misuse will work on OpenAI’s Model Policy team within the Safety Systems organization, defining how increasingly capable AI models should behave in real-world settings. The role turns research on emerging risks into model behavior policies, evaluations, and safeguards, focusing on potential misuse in both digital and physical settings. Risks include assistance with conventional weapons development, misuse of AI-enabled robotics, fraud, impersonation, and other abuse at scale using AI agents. The role involves building threat models, assessing how AI changes the expertise and resources needed to carry out harmful activities, and using evidence to determine how models should respond. The position is based in San Francisco, with office work three days a week and optional work from home on Thursdays and Fridays. Relocation support is offered.

Requirements

  • Hands-on experience building, testing, or assessing dual-use technologies, or a track record of threat modeling grounded in how people use and misuse real systems.
  • Deep practical experience in one relevant domain and judgment about when to involve experts from other domains.
  • Ability to assess whether a capability demonstrated in a test materially changes what someone could do outside the test.
  • Ability to translate technical evidence and uncertainty into clear policy boundaries and measurable evaluation criteria.
  • Experience analyzing gold sets, evaluation results, and training data to spot inconsistent human labels, assess annotator disagreement, and decide what to investigate next.
  • Comfort working hands-on with model data and evaluation results, including inspecting examples, analyzing failure patterns, assessing data quality, and distinguishing policy failures from grader, model, or system failures.
  • Ability to work across disciplines and revise judgment as capabilities and evidence change.
  • Comfort working in a fast-paced, collaborative research environment where priorities shift as models, evidence, and risks change.

Responsibilities

  • Build and update threat models as AI capabilities evolve, assessing plausible misuse and the expertise and resources required.
  • Define when models should help, limit assistance, or refuse in cases where capabilities have both legitimate and harmful uses.
  • Translate safety judgment into instructions, training examples, and evaluations that teach models when to help, limit assistance, or refuse.
  • Turn threat models into evaluation scenarios and grading guidance.
  • Analyze model outputs, red-team findings, and deployment data to identify when models enable misuse or block legitimate work.
  • Partner with research, engineering, security, and product teams to apply findings to model training, deployment decisions, and monitoring.
  • Analyze gold sets, evaluation results, and training data to identify inconsistent human labels, assess annotator disagreement, and determine what to investigate.
  • Inspect model data and evaluation examples, analyze failure patterns and data quality, and distinguish policy failures from grader, model, or system failures.
  • Use empirical evidence to develop and refine safety policies and safeguards.

Benefits

  • Relocation support
  • Optional work from home on Thursdays and Fridays
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service