AI Evaluation Specialist

Weekday AI

1d • Remote

About The Position

Join a cutting-edge AI research initiative focused on improving the quality, accuracy, and reasoning capabilities of next-generation artificial intelligence systems. We are seeking analytical professionals with exceptional critical thinking and communication skills to evaluate AI-generated responses across a variety of topics. In this role, you will assess AI outputs, identify strengths and weaknesses in reasoning, and provide structured, evidence-based feedback that helps improve model performance. This opportunity is ideal for individuals who enjoy careful analysis, attention to detail, and working independently on intellectually challenging tasks. This is a fully remote, contract-based opportunity with flexible working hours.

Requirements

• Bachelor's degree from a globally recognized university (top-ranked institutions preferred).
• Excellent analytical thinking and problem-solving abilities.
• Strong written communication skills with the ability to explain complex reasoning clearly and precisely.
• Exceptional critical reading skills, including the ability to identify: Nuanced arguments, Implicit meaning, Logical inconsistencies, Missing context, Weak or unsupported reasoning
• Strong attention to detail and ability to consistently apply structured evaluation guidelines.
• Ability to work independently and manage assigned tasks efficiently.
• Native-level English fluency.

Nice To Haves

• Experience in content evaluation, research, quality assurance, editing, or analytical review.
• Familiarity with artificial intelligence, large language models, or AI evaluation methodologies.
• Experience working with structured annotation or assessment frameworks.
• Ability to produce thoughtful, objective, and well-supported written evaluations under defined quality standards.

Responsibilities

• Review AI-generated content for accuracy, logical reasoning, completeness, and clarity.
• Identify factual errors, reasoning gaps, inconsistencies, and unsupported conclusions.
• Assess responses using structured evaluation frameworks and detailed quality guidelines.
• Write clear, concise, and evidence-based rationales explaining evaluation decisions.
• Highlight both strengths and areas for improvement in AI-generated outputs.
• Apply consistent judgment across a wide range of evaluation tasks.
• Follow detailed project instructions and standardized assessment criteria.
• Ensure evaluations are objective, accurate, and reproducible.
• Complete assignments independently while maintaining high quality standards.

Benefits

• Contribute to the development of next-generation AI technologies.
• Help improve the reasoning, accuracy, and reliability of advanced AI systems.
• Work on intellectually engaging projects with real-world impact.
• Collaborate indirectly with leading AI researchers through high-quality evaluation work.
• Weekly payments processed through supported payment platforms.

What This Job Offers

• Career Level: Entry Level