Research Engineer

Outset • San Francisco, CA

5d • $200,000 - $240,000

About The Position

Outset builds digital twins: AI systems that represent specific real people and respond to research questions as those people would. The Digital Twins Team runs an ongoing research program to measure and improve how faithfully these systems reproduce human responses and behavior across different levels of analysis.

The Research Engineer reports to the Head Scientist and builds and maintains the systems that enable the team’s scientists to run studies correctly, reproducibly, and at scale. Scientists design studies and define measurements; the Research Engineer ensures experiments execute as designed, results can be reproduced from stored inputs, and research code stays aligned with the shipped product. The ideal candidate has supported scientists in an academic computational social science lab or comparable research setting and has hands-on experience engineering systems built on large language models.

Requirements

• Several years of software engineering experience, including experience supporting researchers in a lab or comparable setting.
• Experience translating specific scientific requirements and experiment designs into code.
• Experience building and operating large-scale data pipelines on human behavior data, and confirming that the pipelines match researcher-specified criteria and design.
• Strong Python skills, including the ability to read and audit code produced by agentic coding tools.
• Fundamentals of computing, including file systems, software libraries, databases, SQL, tabular data, and Git-based workflows.
• Experience with cloud infrastructure and running many jobs concurrently.
• Working understanding of machine learning, including transformer-based language model behavior, model APIs, and fine-tuning.
• Habit of writing deterministic, reproducible, cost-aware, and well-documented code.

Nice To Haves

• Hands-on experience engineering large language model systems, such as prompt pipelines, evaluation harnesses, embedding computation, agentic systems, or fine-tuning infrastructure.
• Experience fine-tuning open-weight language models on multi-GPU hardware.
• Experience serving open-weight models with an inference framework such as vLLM or SGLang, including batched inference and prefix caching for requests that share a common prompt.
• Experience with experiment tracking and cost monitoring for machine learning workloads.
• Experience with survey or panel data and data collection platforms.
• Experience extracting and storing model activations for downstream analysis.
• Experience with experiment tracking and cost monitoring for machine learning workloads, including GPU provisioning across cloud providers.
• Prior work on generative agents, digital twins, or simulation of human respondents.

Responsibilities

• Own evaluation infrastructure for testing competing modeling approaches, enabling tests to be set up quickly, run concurrently and automatically, and monitored.
• Implement measurement pipelines exactly as specified by scientists, producing deterministic and traceable outputs.
• Maintain versioned evaluation datasets, including held-out sets that are never used for training.
• Build and maintain data pipelines for human-participant data, external datasets, and model training data, including linking records belonging to the same participant across sources.
• Keep research code aligned with the production product so validated improvements can ship promptly.
• Detect and prevent threats to evaluation validity.
• Track compute and API spending across experiments and assess implications for production-stage costs.
• Manage repositories, code review, documentation, and reproducibility of the team’s coding workflow.
• Identify implementation choices that could be made more correctly or efficiently and propose changes.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed