AI Operations Lead (Human-in-the-Loop) - NYC-based

Welo Data•New York, NY

18h•Onsite

About The Position

Welo Data is seeking an AI Operations Lead (also referred to as a Data Quality Lead) to lead high-impact teams of analysts focused on AI model evaluation. This strategic leadership role is crucial for defining quality standards in a human-in-the-loop environment. The successful candidate will act as a bridge between technical stakeholders and analyst teams, translating complex AI guidelines into clear, actionable operational excellence to shape the future of AI.

Requirements

5–8 years of experience in Data Quality, QA, or AI/ML operations.
2+ years of experience managing teams in a high-scale data or AI environment.
Ability to "zoom out" to see patterns while maintaining deep attention to detail.
Comfortable navigating ambiguity and balancing speed with rigorous quality standards.
A genuine curiosity about how human feedback shapes the behavior of Large Language Models (LLMs).

Responsibilities

Lead and mentor a specialized team of Data Quality Analysts, fostering growth and managing high-performance output.
Own the end-to-end quality loop, including audits, calibrations, and the identification of model performance trends.
Turn ambiguous, evolving AI guidelines into structured direction for your teams.
Act as the primary representative for your team, aligning with global stakeholders on quality standards and program goals.
Drive continuous improvement across tooling, evaluation methods, and hiring for specialized linguistic talent.