Product Engineer - AI Data

xAI • Palo Alto, CA

97d • $180,000 - $440,000

About The Position

You will design and implement evaluation pipelines that enable xAI’s models to self-improve through end-user interactions, building features and data flows that capture human feedback to drive continuous optimization of user value, engagement, and key metrics in consumer products reaching millions of users. You’ll work with us on: Building RLHF tooling to gather and incorporate end-user preferences and feedback, enabling models to iteratively self-improve based on real-world usage. Developing data pipelines and AI evaluation frameworks to track model performance, safety, user satisfaction, and metrics like retention and engagement from an end-user lens. Work in product and work with our model training teams to optimize and productionize AI powered features.

Requirements

• Strong grasp of scaling AI products from prototype to global deployment.
• Skilled in developing robust AI evaluation methodologies, including benchmarks and metrics.
• Hands-on experience with RLHF or similar preference-based learning systems.
• Ability to thrive in ambiguous environments and iterate quickly based on feedback.

Nice To Haves

• Proven track record in RLHF implementation or AI alignment projects.
• Experience building data pipelines for metric optimization and model self-improvement.
• Familiarity with evaluation tools and frameworks (e.g., custom benchmarks, A/B testing).
• Strong portfolio demonstrating evaluation work and feature development.

Responsibilities

• Design and implement evaluation pipelines for AI models.
• Build RLHF tooling to gather and incorporate end-user preferences and feedback.
• Develop data pipelines and AI evaluation frameworks to track model performance and user satisfaction.
• Collaborate with product and model training teams to optimize AI powered features.

Benefits

• Equity
• Comprehensive medical coverage
• Vision coverage
• Dental coverage
• Access to a 401(k) retirement plan
• Short & long-term disability insurance
• Life insurance
• Various discounts and perks

What This Job Offers

• Number of Employees: 11-50 employees