Quantitative, Head of Dataset & Quality

Build AI • San Francisco, CA

14h • Onsite

About The Position

The Head of Dataset & Quality owns whether Build AI’s dataset is effective for solving physical labor. The role focuses on maximizing the amount of new learnable information entering the dataset by deciding what to collect, how to evaluate existing data, and whether additional collection contributes information or merely volume. The person defines the ideal dataset, the objective function for the current dataset, its taxonomy, and the value of incremental data. These decisions inform collection strategy, including the trade-off between scaling simple, high-bandwidth capture and using slower, higher-fidelity setups.

The role works with marketplace incentives, software, research, operations, vendors, and the Evals Lead to put dataset objectives into practice. It establishes measurable standards for coverage, quality, and information, while helping the company scale in-the-wild collection across countries. Build AI is a fully in-person team in San Francisco’s Financial District and Shenzhen’s Nanshan, and expects technical staff to contribute across engineering and research disciplines.

Requirements

• A quantitative background, such as quantitative research, statistics, decision science, or a similar field.
• Ability to think in terms of objective functions and information, not only label-quality queues.
• Ability to use numbers to reason about scaling versus fidelity.
• Experience with dataset design, collection strategy, or large-scale data programs.
• Comfort with in-the-wild collection and working at a company that is still scaling across countries.

Nice To Haves

• Experience at a lab, quant fund, or large-scale data program.
• Experience designing a taxonomy, golden set, or coverage model used in production.
• Familiarity with in-the-wild collection, video, or pose data.
• Experience setting quality standards and audit processes that collectors or vendors meet.

Responsibilities

• Define the ideal dataset for solving physical labor, including coverage, diversity, and criteria for completion.
• Define the objective function for the dataset currently available.
• Design the taxonomy used by collectors, as well as golden sets, acceptance criteria, and audits that make quality measurable.
• Create a value function for incremental data to determine the value of the next example given the existing dataset.
• Translate dataset value into collection strategy, including where and which work to collect, how much to collect, and when to stop.
• Set standards that operations teams and vendors can execute against.
• Avoid local optimizations such as extra sensors, fidelity, or process when they reduce throughput and net information.
• Work with marketplace incentives, software, and research to incorporate the objective into scorecards for coverage, quality, and information—not only volume.
• Partner with the Evals Lead so dataset decisions and model-capability measurements inform one another.

Benefits

• Competitive pay.
• Medical, dental, and vision packages with generous premium coverage.
• $500 per month credit for waiving medical benefits.
• Housing subsidy of $2,000 per month for employees living within walking distance of the office.
• Relocation support for those moving to San Francisco or Shenzhen.
• Wellness benefits covering fitness, mental health, and more.
• Daily lunch and dinner in the office.
• Unlimited compute budget, subject to ROI justification.
• Unlimited Codex and Claude credits.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed