Research Engineer/Scientist

Center for Artificial Intelligence Safety Inc • San Francisco, CA

29d • $140,000 - $200,000 • Onsite

About The Position

The Center for AI Safety (CAIS) is a leading research and advocacy organization focused on mitigating societal-scale risks from AI. We address the toughest challenges in AI safety through technical research, field-building initiatives, and policy engagement, along with our sister organization, Center for AI Safety Action Fund.

What distinguishes us is what we choose to work on. Our work is aimed at reducing real-world risks from advanced AI systems. We deliberately pursue research directions that the field is not yet paying attention to, and we move on once the rest of the field catches up. Our focus is on problems that are both highly important and highly neglected—and our track record is built on getting to them first.

In 2022–2023, we focused on AI honesty, robustness, transparency, and trojan/backdoor behaviors.
In 2023–2024, we turned to malicious use and weaponization capabilities, introducing the first state-of-the-art benchmarks for measuring it.
More recently, we've been working on AI value systems and the functional well-being of AI systems.

This is a research philosophy as opposed to a fixed agenda: we go where the important, unworked problems are. Because our work tends to be timely and to open up territory rather than crowd into it, our papers have repeatedly gone on to become widely cited and to set the standard for underexplored research areas. Our work is regularly used by AI safety institutes and frontier AI labs, and they have shaped real policy outcomes, including being presented directly to senators and policymakers.

About the role
As a Research Engineer (RE) or Research Scientist (RS) at CAIS, you'll lead and execute high-impact research that advances the safety and reliability of frontier AI systems. This is the general posting for both roles—if you're interested in either, apply here, and we'll determine which is the better fit based on our judgment during the process.

You will design and run experiments on large language models, build the tooling to train and evaluate models at scale, and turn results into publishable research. You'll work closely with CAIS researchers and external academic and commercial partners, using our compute cluster to run large-scale training and evaluation.

Our work centers on empirical deep learning research with large language models and/or multimodal models. If your background is primarily theoretical, this role may not be a good fit.

Research directions at CAIS are set by our Research Director, who selects and prioritizes projects for their importance, neglectedness, and timeliness — a major reason our work has been so impactful. In practice, this means that you will consistently be working on interesting, high-impact problems, with substantial freedom in how you pursue them: designing and running experiments, iterating, and chasing the threads you find most promising.

Day to day work:
Own research experiments end-to-end.
Train and fine-tune large transformer models across domains.
Build and maintain datasets and benchmarks.
Run distributed training and evaluation at scale.
Write and ship research, collaborating with co-authors, and supporting submissions of papers to top conferences.
Collaborate with researchers and external partners while contributing to shared research direction and responding quickly in research cycles.
Support research infrastructure as needed, such as internal tooling, documentation, and reproducibility practices for the team.
Mentor, guide, and support others on the team.

Requirements

• Have co-authored multiple papers published at a top ML conference venues (e.g., NeurIPS, ICML, ICLR, ACL, CVPR). Workshop papers are considered, though peer-reviewed conference publications are strongly preferred. Publications in journals such as IEEE or Springer Nature are typically given less weight. Alternatively, have made meaningful research contributions at a leading AI lab.
• Are a current PhD student or researcher in machine learning or a related field. Exceptional candidates with a strong publication record may be considered regardless of degree level.
• Have a track record of empirical research in AI or ML, particularly in AI safety-relevant areas (e.g. adversarial robustness, calibration, benchmarking). We weight empirical research heavily; candidates with primarily theoretical backgrounds are generally not a strong fit.
• Are able to read an ML paper, understand the key result, and understand how it fits into the broader literature.
• Are comfortable setting up, launching, and debugging ML experiments.
• Are familiar with relevant frameworks and libraries (e.g., PyTorch).
• Communicate clearly and promptly, and take ownership of your part of a project.

Responsibilities

• Own research experiments end-to-end.
• Train and fine-tune large transformer models across domains.
• Build and maintain datasets and benchmarks.
• Run distributed training and evaluation at scale.
• Write and ship research, collaborating with co-authors, and supporting submissions of papers to top conferences.
• Collaborate with researchers and external partners while contributing to shared research direction and responding quickly in research cycles.
• Support research infrastructure as needed, such as internal tooling, documentation, and reproducibility practices for the team.
• Mentor, guide, and support others on the team.

Benefits

• Health insurance for you and your dependents
• 401K plan + 4% matching
• Unlimited PTO
• Lunch and dinner at the office
• Annual Professional Development Stipend
• Access to some of the top talent working on technical and conceptual research in AI safety

What This Job Offers

• Job Type: Full-time
• Career Level: Entry Level