Member of Technical Staff - Research Engineer, Post-training

Preference Model•San Francisco, CA

6d•$200,000 - $350,000

About The Position

Preference Model is a superintelligence data research company that builds reinforcement learning (RL) environments for training capable, better-aligned superintelligences. The company researches how to build better RL environments, with the goal of improving training objectives for both current and future AI systems. It has built RL environments for several frontier labs and is backed by $16 million in seed funding led by a16z. Its founding team has prior experience building data infrastructure and datasets for Anthropic’s Claude. The role focuses on investigating the boundaries of self-directed learning and advancing post-training for large language models. It blends research and engineering, involving implementation of novel approaches and contribution to research direction. The position is suited to a machine learning Research Engineer or Research Scientist with experience running large-scale LLM post-training pipelines and building ML infrastructure.

Requirements

Industry experience running end-to-end LLM post-training pipelines for models at least 7B in size.
Proficiency in Python and PyTorch or JAX.
Experience with at least one modern RL training framework.
Experience building and operating ML infrastructure at scale.

Nice To Haves

Experience evaluating model outputs and building reward or evaluation signals.
Stay current on post-training research and translate papers into running code.
Strong, adaptable opinions about structuring RL training code for reproducibility and fast iteration.
Ability to balance research exploration with engineering rigor.
Strong systems design and communication skills.
Adaptability and exceptional communication and collaboration skills.

Responsibilities

Train and evaluate models on proprietary RL environments to validate data quality, identify gaps in task coverage, and connect environment design with model capability.
Architect and optimize RL training infrastructure, including training abstractions and distributed experiment management, and help scale systems for increasingly complex research workflows.
Design, implement, and test training environments, evaluations, and methodologies for RL agents.
Profile and optimize training runs end to end, from data loading through reward computation, to improve experiment throughput and shorten research iteration cycles.