hour

24-Mag•New York, NY

1d•Remote

About The Position

We are sharing a specialised part-time consulting opportunity for detail-oriented generalists based in the United States or Canada who can evaluate professional content across documents, presentations, spreadsheets, and other common workplace formats. This role supports the training and evaluation of advanced AI systems. Selected professionals will review real-world work products, compare AI-generated outputs, assess quality and correctness, and provide clear written feedback on accuracy, reasoning, clarity, formatting, and completeness.

Requirements

Bachelor's degree or higher
Based in the United States or Canada
Excellent English reading comprehension and written communication
Strong analytical judgement across varied professional subject matter
Exceptional attention to detail
Ability to identify errors, inconsistencies, formatting problems, and gaps in reasoning
Comfortable working with documents, presentations, spreadsheets, and common productivity tools
Able to move efficiently between different topics and content formats
Comfortable working independently within structured evaluation guidelines

Nice To Haves

Prior AI, machine learning, or model-evaluation experience is not required

Responsibilities

Content Quality Evaluation: Review documents, presentations, spreadsheets, and other professional materials. Assess outputs for accuracy, clarity, completeness, organisation, and overall quality. Identify factual errors, inconsistencies, omissions, and weak reasoning. Evaluate whether materials meet professional standards for real-world use. Apply sound judgement across a broad range of everyday subject areas.
AI Output Comparison: Compare and rank AI-generated outputs against defined evaluation criteria. Determine which response or work product better satisfies the task requirements. Identify meaningful differences in quality, reasoning, structure, and presentation. Flag incomplete, misleading, or poorly supported outputs. Apply evaluation guidelines consistently across varied tasks.
Documents, Slides & Spreadsheets Review: Review written documents for structure, readability, and accuracy. Evaluate presentation slides for clarity, coherence, and effective communication. Assess spreadsheets for logical consistency, organisation, and correctness. Identify formatting issues that reduce usability or professional quality. Work comfortably across common productivity tools and file formats.
Written Feedback & Quality Assurance: Provide clear, concise, and well-reasoned written feedback. Explain ratings and preference decisions using specific evidence. Highlight the most important issues rather than relying on general impressions. Follow structured rubrics and project guidelines. Maintain consistent quality standards across diverse evaluation tasks.