AI Safety & Data Quality Analyst (HITL) - NYC

Welo Data • New York, NY

22h • Onsite

About The Position

Welo Data is seeking Data Quality Analysts for a Human-in-the-Loop (HITL) role in NYC. This position is for individuals who are sharp, curious, and detail-oriented, and can go beyond simple labels to help define how AI should behave in the real world. As a Data Quality Analyst, your judgment is the product, and you will be responsible for refining guidelines, identifying failure modes, and ensuring that the next generation of AI is safe, accurate, and helpful. The role involves critiquing AI outputs, providing structured feedback for model improvement, identifying bias, risks, and edge cases using GenAI safety experience, spotting patterns in model failures, communicating insights to stakeholders, and contributing to the improvement of evaluation frameworks. You will also partner with leads and cross-functional teams to establish gold standards for AI quality.

Requirements

• A University degree.
• Native-level language proficiency.
• 1–2 years of professional writing experience.
• Ability to articulate complex "why" justifications clearly.
• 2+ years of experience specifically focusing on Safety & Trust in GenAI data delivery.
• The ability to navigate "grey areas" and make sound judgment calls when guidelines are ambiguous.
• Naturally interested in how LLMs behave and aren't afraid to speak up when something feels "off."
• Sharp, curious, and obsessed with the details.

Responsibilities

• Critique AI outputs and provide structured, high-level feedback that drives model improvement.
• Use your 2+ years of GenAI safety experience to identify bias, risks, and edge cases.
• Spot patterns in how models fail and communicate these insights to stakeholders.
• Help improve the evaluation frameworks themselves.
• Partner with leads and cross-functional teams to set the gold standard for AI quality.

Benefits

• Free Gourmet Dining: Chef-prepared breakfast, lunch, and dinner with global cuisines.
• Endless Snacks: Micro-kitchens stocked with premium coffee, beverages, and healthy treats.
• Campus Life: Access to unique features like rooftop nature parks and modern collaborative spaces.
• Commuter Ease: Free transport, shuttles, and bike-to-work perks.
• Full Security: Comprehensive Medical, Dental, and Vision + 401(k) and HSA.
• Time Off: 15 days of combined Paid Sick/Holiday time + major holidays.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Number of Employees: 1,001-5,000 employees