AI Evaluators: Assessing A Shopping Assistant

Terac

8h • Remote

About The Position

Terac is hiring AI evaluators for a project assessing the accuracy and helpfulness of a digital shopping assistant. Evaluators will analyze real user interaction traces on a custom internal platform, identify failures in logic, inaccuracies, and unhelpful product recommendations, and create structured rubrics and verifiers to consistently assess future response quality. This is an ongoing remote engagement requiring a commitment of 20 or more hours per week. Compensation is $50 per hour.

Requirements

• Experience in data evaluation, quality assurance, or AI training.
• Strong analytical skills and ability to spot subtle errors in text.
• Familiarity with e-commerce search and digital shopping experiences.
• Ability to commit to a sustained workload of 20 or more hours per week.

Nice To Haves

• Prior experience in prompt engineering, complex data annotation, or software testing.

Responsibilities

• Review real user interaction traces with an AI shopping assistant.
• Identify logical failures, inaccuracies, or poor product recommendations in text interactions.
• Create structured rubrics and verifiers to judge response quality.
• Commit to at least 20 hours per week of evaluation work on the internal platform.

What This Job Offers

• Job Type: Full-time
• Education Level: No Education Listed