Research Scientist - RL Training

Snorkel AI • San Francisco, CA

14h • $200,000 - $350,000

About The Position

Snorkel AI is seeking a Research Scientist to advance reinforcement learning for training and aligning large language models. This foundational research role focuses on generating data, reward signals, and training procedures that steer LLM behavior reliably and generalizably, supporting a core capability of Snorkel’s data-as-a-service offering.

The role works closely with research, engineering, and delivery teams to develop RL data capabilities and translate research ideas into preference datasets, reward models, and RL-ready corpora for frontier AI labs. The scientist will also contribute to a research agenda central to Snorkel’s long-term differentiation as a provider of bespoke training data.

Snorkel AI describes itself as a frontier AI data lab that develops datasets, benchmarks, evaluations, and custom solutions for real-world AI systems. Founded out of the Stanford AI Lab in 2019, the company works with leading AI labs and enterprises.

Nice To Haves

• Deep expertise in reinforcement learning from human or AI feedback, reward modeling, and credit attribution, ideally with a clear perspective on what data makes these techniques work.
• Experience training or fine-tuning large language models with 30B+ parameters at scale, including familiarity with distributed training infrastructure.
• Strong proficiency in Python and ML frameworks, especially PyTorch and HuggingFace, and hands-on experience with RL frameworks such as Verl and SkyRL.
• Solid software engineering fundamentals, including the ability to build research prototypes that others can run, extend, and integrate into data production workflows.
• Familiarity with ML infrastructure and cloud platforms and tools such as AWS, GCP, Kubernetes, and Slurm; experience with large-scale RL training pipelines is a strong plus.
• Comfort operating in a high-iteration environment with open-ended research questions and shifting, customer-driven technical constraints.
• Ph.D. in machine learning, reinforcement learning, or a related field strongly preferred; exceptional industry experience considered.

Responsibilities

• Research and implement reinforcement learning techniques, including GRPO, RLHF, RLAIF, DPO, and reward modeling, and translate them into preference datasets, reward signals, and verifiable rewards for training and fine-tuning LLMs.
• Design and build data pipelines that generate high-quality training signals for RL workflows, including AI-assisted data annotation and curation pipelines to improve generalization to unseen benchmarks.
• Prototype and iterate on end-to-end RL training recipes that inform the data Snorkel provides through its data-as-a-service deliveries.
• Work with research scientists, ML engineers, and delivery teams to translate RL research into customer-ready data products.
• Keep current with developments in large-scale multi-node LLM training, alignment research, and scalable RL methods, including methods for complex environments such as Terminal-Bench, and bring relevant advances into Snorkel’s approach.
• Contribute to Snorkel’s research publications and internal knowledge base in RL and model training.

Benefits

• Meaningful opportunities to shape priorities and initiatives, influence strategic decisions, and directly impact company success.
• Support for building a career through opportunities to deepen technical expertise, explore leadership opportunities, and learn skills across multiple functions.
• Reasonable accommodation for applicants and employees with disabilities during application, interview, and work processes.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Ph.D. or professional degree