Member of Technical Staff - Research Engineer, Post-training

Preference Model•San Francisco, CA

About The Position

Preference Model is a superintelligence data research company that builds reinforcement-learning (RL) environments for training capable, better-aligned AI systems. The company researches how to build better RL environments and has built environments for several frontier labs. It is backed by $16 million in seed funding led by a16z; its founding team previously worked on Anthropic’s data team, building data infrastructure and datasets behind Claude. The role focuses on investigating the potential of self-directed learning and advancing post-training for large language models. It blends research and engineering: the successful candidate will implement novel approaches, shape research directions, and help push the frontier of post-training. Candidates may be Research Engineers or Research Scientists. A PhD or extensive publications are not required; the company emphasizes adaptability, communication, and collaboration.

Requirements

Experience running end-to-end LLM post-training pipelines for models of at least 7B parameters.
Proficiency in Python and PyTorch or JAX.
Experience with at least one modern RL training framework.
Experience building and operating ML infrastructure at scale.

Nice To Haves

Experience evaluating model outputs and building reward or evaluation signals.
Stay current on post-training research and be able to translate papers into running code.
Strong, flexible opinions about structuring RL training code for reproducibility and fast iteration.
Ability to balance research exploration with engineering rigor.
Strong systems design and communication skills.
Adaptability and exceptional communication and collaboration skills.

Responsibilities

Train and evaluate models on proprietary RL environments to validate data quality, identify gaps in task coverage, and connect environment design with model capability.
Architect and optimize RL training infrastructure, including training abstractions and distributed experiment management, using frameworks such as Verl, OpenRLHF, or similar.
Scale systems to support increasingly complex research workflows.
Design, implement, and test training environments, evaluations, and methodologies for RL agents.
Profile and optimize training runs end-to-end, from data loading through reward computation, to increase experiment throughput and shorten research iteration cycles.