hour

24-Mag•New York, NY

6d•Remote

About The Position

We are sharing a specialised consulting opportunity for experienced Educator/ Assessor Writers with strong expertise in assessment, grading, rubric-based evaluation, academic writing, instructional design, and critical content review to contribute to an advanced AI training and written-response evaluation project. Selected professionals will evaluate AI-generated written responses, apply structured assessment criteria, identify factual and reasoning issues, and produce clear written rationales supporting consistent quality judgements. The opportunity is fully remote and suitable for experienced educators, assessors, academic reviewers, and instructional professionals. No prior experience in AI is required.

Requirements

First-language proficiency or fully equivalent fluency in written English
Completed degree from an accredited institution in United States, Canada, United Kingdom, Ireland, Australia, or New Zealand
Experience marking, grading, examining, or reviewing written work against defined criteria
Background in assessment design, instructional design, peer review, thesis supervision, or academic writing support
Strong critical-analysis and content-evaluation skills
Excellent written communication and editing ability
Exceptional attention to detail and consistency
Ability to make objective, well-articulated assessment judgements
Comfortable interpreting and applying structured rubrics
Able to identify factual, logical, and usability issues in written content
Comfortable working independently in a remote environment

Nice To Haves

Familiarity with digital content-review platforms is advantageous
Experience with model evaluation, RLHF, annotation, or similar workflows is beneficial but not required

Responsibilities

Evaluate and compare multiple AI-generated written responses
Assess clarity, tone, helpfulness, relevance, and adherence to instructions
Identify strengths and weaknesses across competing responses
Evaluate whether content is appropriate for its intended audience
Apply consistent professional judgement across repeated review tasks
Apply provided rubrics and evaluation guidelines to written content
Assign analytical quality scores using defined assessment criteria
Distinguish between fluent, well-written responses and genuinely high-quality outputs
Identify responses that are comprehensive but impractical or insufficiently supported
Maintain consistency and objectivity across scoring decisions
Detect factual errors, unsupported claims, and internal inconsistencies
Identify weaknesses affecting reliability, usability, or instructional value
Assess reasoning quality and whether conclusions are adequately supported
Review written material with strong attention to detail and analytical precision
Flag ambiguous or borderline cases requiring additional consideration
Produce detailed written rationales explaining assessment decisions
Clearly justify scores, rankings, and evaluation outcomes
Document findings, observations, and recurring quality issues
Maintain thorough records of review decisions
Communicate complex assessment judgements in concise, professional language
Apply experience from grading, marking, instructional design, or academic review
Contribute insight on rubric interpretation and effective assessment practices
Collaborate asynchronously with project managers and reviewers
Use digital content-review platforms to complete and document assignments
Work independently while meeting project deadlines and quality expectations