Software Engineer - Post-training Data

Magic AI•San Francisco, CA

13d•$225,000 - $550,000

About The Position

Magic’s mission is to build safe AGI that accelerates humanity’s progress on the world’s most important problems. We believe the most promising path to safe AGI lies in automating research and code generation to improve models and solve alignment more reliably than humans can alone. Our approach combines frontier-scale pre-training, domain-specific RL, ultra-long context, and inference-time compute to achieve this goal. About the role: As a Software Engineer working on our post-training data infrastructure, you write efficient and robust data pipelines and work with the research and Applied teams to devise strategies for gathering and maintaining specific, diverse datasets at scale.

Requirements

Extreme attention to detail and commitment to data quality.
Ability to write reliable, well-tested code and quickly learn new tools, programming languages or frameworks needed for a given job.
Versatility in end-to-end data pipelines, from scraping to processing.
High intellectual agility and grit to tackle tough challenges.

Responsibilities

Develop creative ways to obtain post-training datasets teaching the model targeted capabilities.
Develop creative ways to reliably generate synthetic datasets
Iterate on filtering and scoring heuristics for a post-training dataset
Contribute to our data infrastructure by implementing, maintaining and testing data pipelines serving workloads across scales from gigabytes to 100s of petabytes