Head of Data Operations

Health GPT Inc•Palo Alto, CA

56d•Onsite

About The Position

We are seeking a Head of Data Operations to lead the teams and partners responsible for data generation, annotation, evaluation, and RLHF (reinforcement learning from human feedback) across HippocraticAI's healthcare agent products. This leader will own the entire data operations lifecycle - from sourcing and labeling to model evaluation and feedback - ensuring precision, scalability, and alignment with our safety and ethics principles. You will manage internal teams, global contractors, and strategic vendors to deliver high-quality data pipelines that enable continual learning and improvement of our agentic systems.

Requirements

10+ years of experience in data operations, annotation, or model evaluation, with 5+ years in management or leadership roles.
Proven success scaling data or RLHF operations across geographies and vendors.
Strong program management and process optimization skills; experience managing distributed teams.
Familiarity with LLM training and evaluation, RLHF, or human-in-the-loop systems.
Deep respect for data ethics, privacy, and quality - ideally within healthcare, life sciences, or another regulated industry.
Excellent communication and collaboration skills; able to navigate between technical, clinical, and operational stakeholders.

Nice To Haves

Experience in medical or healthcare data annotation or clinical workflow modeling.
Prior work building custom data pipelines or labeling platforms.
Understanding of LLM fine-tuning, preference modeling, and evaluation metrics.
Global vendor management experience with large-scale workforce operations.

Responsibilities

Build, lead, and scale a global data operations organization including full-time employees, contractors, and vendor partners.
Define clear roles, quality standards, and performance metrics across all data functions (evaluation, labeling, RLHF, and generation).
Partner with Legal, Compliance, and Security to ensure all global data work adheres to HIPAA and data privacy standards.
Oversee the design and execution of evaluation frameworks for LLMs and agentic behaviors - both automated and human-in-the-loop.
Lead data labeling, synthesis, and annotation operations, ensuring medical accuracy, consistency, and context-rich quality.
Manage large-scale RLHF pipelines - aligning training data with clinical and ethical objectives.
Optimize throughput, cost, and quality across in-house teams and external vendors.
Partner with engineering and product to design and improve data operations infrastructure, including labeling tools, quality assurance systems, and task routing platforms.
Implement robust QA processes and auditing frameworks to ensure data integrity and reliability.
Drive continuous improvement in efficiency, consistency, and evaluator experience.
Work closely with Research, Model, and Product teams to define data needs and feedback loops.
Collaborate with Clinical and Safety leaders to align annotation and evaluation standards with clinical guidelines.
Provide strategic input into data strategy, metrics, and operational planning.