Research Scientist

Outset • San Francisco, CA

16d • $200,000 - $240,000

About The Position

Outset builds digital twins: AI systems that represent specific real people and respond to research questions the way those people would. The Digital Twins Team runs an ongoing research program to measure and improve how faithfully these systems reproduce human responses and behavior across cross-sectional levels of analysis.

The Research Scientist works directly with and reports to the Head Scientist. In collaboration, they choose what to study, design studies, define how results are measured, interpret findings, and write them up for internal decision-making and, where appropriate, academic publication. The role calls for current methodological training in computational social science, data science, survey methodology, human-computer interaction, and experiment design.

Requirements

• Completed or nearly completed PhD in computational social science, communication, HCI, sociology, political science, psychology, statistics, or a related field.
• Peer-reviewed publications on which the candidate was a primary author.
• Working command of graduate-level statistical methodology common to social science, including its assumptions.
• Experience analyzing large-scale telemetry and behavioral datasets, with attention to the validity of resulting inferences.
• Familiarity with machine learning, including classical, nonparametric, and LLM-based approaches; understanding of transformer-based language models, fine-tuning, and differences among major model families.
• Fundamentals of computing, including file systems, software libraries, databases, SQL, and tabular data.
• Ability to work in an environment where most code is produced through agentic coding tools, and to read and check that code.

Nice To Haves

• Experience fine-tuning open-weight language models on survey, experimental, or behavioral data.
• Experience building or evaluating calibrated confidence or error-prediction models, including deriving decision thresholds from human judgments.
• Experience measuring similarity of open-ended text, for example with embedding-based methods.
• Experience designing or running panel-based data collection with human participants.
• Experience presenting research at academic and/or industry conferences.
• Prior work on generative agents, digital twins, or simulation of human respondents.
• Familiarity with the startup ecosystem.
• Familiarity with qualitative interviews and ethnographic methods.
• Familiarity with topics in computational linguistics.

Responsibilities

• Co-design and run studies comparing system-generated responses with human responses, using qualitative and quantitative measures.
• Design, justify, and refine measures of performance.
• Apply and adapt state-of-the-art statistical techniques common to survey methodology and large-scale social data analysis to benchmark and improve sample- and population-level simulation performance.
• Design evaluations and experiments with pre-specified analysis plans.
• Design and oversee human-rating studies, including rater training, coding rubrics, and inter-rater agreement measurement.
• Design data collection from third-party data sources and directly from human participants, including longitudinal and repeated-measures studies.
• Contribute to the research agenda for new modeling approaches, including evaluation of fine-tuned models and model calibration.
• Specify analyses precisely enough for an engineer to implement.
• Co-author papers, preprints, and presentations.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Ph.D. or professional degree