Member of Technical Staff - Research Engineer, RL Environments

Preference Model•San Francisco, CA

About The Position

Preference Model is a superintelligence data research company that builds reinforcement learning (RL) environments for training capable, better-aligned superintelligences. The company develops training objectives and environments intended to improve model capabilities and alignment. It has built RL environments for frontier labs and is backed by $16M in seed funding led by a16z. Its founding team previously worked on Anthropic’s data team, building data infrastructure and datasets behind Claude. The Research Engineer will design and build RL environments to safely advance model capabilities in machine learning research and engineering. The work focuses on teaching frontier models to perform the work of ML engineers and researchers at frontier labs. This role combines research and engineering, including following current research, developing novel approaches, and implementing them in code. The role offers ownership and autonomy over environments, from design and implementation through experiments, evaluations, and delivery into production training runs. The hire will join the small, high-ownership Capabilities organization and contribute to the data layer powering frontier LLM capabilities. This opening is for experienced ML engineers; a separate opening is available for new graduates.

Requirements

At least 5 years of experience working in machine learning or research, primarily on LLMs and transformer models.
Strong machine learning fundamentals and broad research interests.
Ability to read papers or tutorials, understand topics deeply, and creatively translate them into RLVR problems.
Proficiency in Python and systems programming.
Proficiency in at least one of PyTorch or JAX.
Ability to take ownership, solve problems, and drive solutions end to end.
Passion for staying current with the rapidly evolving ML infrastructure landscape.
Ability to meet throughput expectations and respond quickly to feedback.

Nice To Haves

Expert knowledge in an active deep learning or machine learning research area, with publications or public code to demonstrate it.
Research experience, such as a PhD or MS.
Deep understanding of transformer internals and training or inference of modern LLMs.
Experience with inference libraries such as vLLM or SGLang.
Strong expertise in kernel development, such as CUDA, Triton, or Pallas.
Experience building complex interactive RL environments.

Responsibilities

Design and build RL environments and reward functions that produce clean, learnable signals for frontier models on ML research and engineering tasks.
Build deep expertise across frontier ML research, training, and inference infrastructure.
Collaborate with others to brainstorm and create new ideas and tools to improve the environment-building process.
Conduct experiments and evaluations and deliver work into production training runs.
Stay up to date with the latest research, develop novel approaches, and implement them in code.