Human Data Strategist

Reflection • San Francisco, CA

3d

About The Position

The Human Data Strategist will own pipelines that turn human expertise into data for improving Reflection’s AI models. The role sits on the Data Acquisition Team and partners closely with researchers to understand their needs and source data from vendors that improves model usefulness and performance. It combines program ownership with hands-on task design, contributor workflows, data quality frameworks, and success metrics. The position requires organizational rigor, technical curiosity, and sound judgment about when to move quickly and when to investigate details.

Requirements

• At least 4 years of experience delivering a portfolio of complex programs and prioritizing across changing research needs.
• Understanding of LLM training, evaluation, and prompt design, with the ability to learn unfamiliar research concepts quickly.
• Proficiency in SQL for querying datasets and building dashboards.
• Proficiency in Python for data analysis, API integrations, and workflow automation.
• Clear written communication and strong stakeholder judgment.
• Willingness to inspect individual tasks and datasets when headline numbers do not tell the full story.
• High agency, a bias toward action, and ability to drive impact in fast-paced, dynamic, and ambiguous environments.

Nice To Haves

• Experience with annotation, expert data collection, human feedback, or AI evaluation.

Responsibilities

• Translate critical model capabilities and researchers’ plans into executable data acquisition strategies.
• Assess vendor capabilities, forecast capacity, define task instructions and acceptance criteria, and hold vendors accountable for quality, delivery, and cost.
• Build dashboards tracking throughput, turnaround time, acceptance rates, rework, and cost per accepted task.
• Use data analytics to investigate bottlenecks and guide improvements.
• Surface vulnerabilities early as experiments and priorities evolve, and turn feedback into concrete changes to subsequent sourcing.
• Work with research teams to close the loop and track how data acquisitions improve model capabilities.

Benefits

• Top-tier compensation, with salary and equity structured to recognize and retain talent globally.
• Stock options for everyone who joins and contributes to Reflection’s success.
• Comprehensive medical, dental, vision, and life insurance, plus an annual wellness allowance.
• Lunch and dinner provided in the office daily.
• 22 weeks of paid parental leave for all new birthing and non-birthing parents, including adoptive and surrogate journeys.
• Unlimited paid time off in the U.S. and 30 vacation days in the U.K.
• Visa sponsorship and support for long-term immigration pathways where applicable.
• Regular off-sites, happy hours, and team celebrations.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed