AI Chatbot Evaluator

Productive Playhouse•Minneapolis, MN

11d•$23 - $45•Remote

About The Position

Productive Playhouse is seeking individuals to evaluate the performance of leading AI chatbots on health and wellness topics. This project aims to determine if personalization enhances response quality. Participants will use their existing personal AI chatbot accounts and conversation history (e.g., Gemini, ChatGPT, Claude) to assess how effectively the chatbot uses user context to deliver high-quality, meaningful, and personalized insights.

Requirements

Must be 18 years of age or older.
Currently residing in the United States.
Must be fully proficient in reading and writing instructions in English.
Must have one of the following existing, active personal accounts: A personal Google account to use Gemini (not newly created or a test account); An active, paid ChatGPT subscription; or An active, paid Claude Pro subscription.
The selected AI chatbot account must have at least 1 month of active usage history.
Must be flexible and open to changes as the project evolves.

Nice To Haves

Enjoy using and conversing with AI chatbots.
Very comfortable following step-by-step instructions independently.
Open to feedback and comfortable collaborating on an iterative workflow.
Provide thoughtful, detailed feedback on AI interactions.

Responsibilities

Interact with your assigned AI model on designated health and wellness topics.
Use your personal AI chatbot account and existing conversation history to evaluate personalized response quality.
Assess how well the system adapts to your inputs and provides accurate, relevant responses.
Address feedback and implement revisions to support high quality submissions.
Complete a structured evaluation form based on your experience.