Remote | Professional Services Evaluation Consultant — $60–$75/hour

24-Mag • New York, NY

3d • Remote

About The Position

We are sharing a specialised part-time consulting opportunity for experienced professionals across project management, law, accounting and audit, software development, and IT management to evaluate realistic professional work products within their own field. This role focuses on reviewing completed AI-generated work rather than producing the underlying deliverables. Selected professionals will assess whether tasks are realistic, domain content is accurate, grading decisions are justified, and the resulting work meets the standards expected of an experienced practitioner.

Requirements

• Substantial hands-on experience in one of the following professional areas: Project Management (5+ years of experience running real project plans, schedules, and budgets, Experience managing deliverables that teams actively worked against, Strong understanding of professional project-management documentation and controls), Law (US-qualified lawyer in active practice, 4+ years of post-admission experience, Hands-on experience drafting legal documents rather than exclusively reviewing others' work, Strong ability to evaluate legal accuracy, terminology, and professional work product), Accounting & Audit (4+ years of professional experience, Hands-on responsibility for financial close processes, audit files, or financial statements, Strong knowledge of accounting and audit documentation), Software Development (3+ years of professional software development experience, Experience shipping production code subsequently maintained by other engineers, Strong ability to assess code quality, implementation decisions, and realistic development practices), IT & Information Systems Management (4+ years of professional experience owning an IT function, Hands-on responsibility for budgets, vendors, systems, and operational decisions, Practical experience producing or reviewing professional IT work products, Strong understanding of real-world technology operations and management)
• Experience reviewing junior or peer work is particularly valuable
• Near-native business English
• Ability to provide concise written explanations for professional judgements
• Comfortable reviewing spreadsheets, PDFs, and related professional documents
• Able to verify figures, calculations, and internal consistency
• Candidates whose experience is primarily certification-based without substantial hands-on delivery experience are less suited to this engagement

Nice To Haves

• PMP certification is advantageous but not required
• CPA or CA qualification is strongly preferred
• Previous experience conducting substantive code reviews is highly relevant

Responsibilities

• Review complete work products representing realistic tasks from your own professional field
• Examine the task instructions, completed deliverables, supporting files, and evaluation results
• Assess whether the work reflects realistic professional practice
• Identify substantive issues that would be apparent to an experienced practitioner
• Evaluate whether completed outputs would be usable or acceptable in a real professional setting
• Determine whether assigned tasks accurately reflect work performed in the relevant profession
• Assess whether task descriptions, constraints, and expected deliverables are realistic
• Distinguish authentic professional scenarios from borderline or contrived tasks
• Evaluate whether requirements align with normal industry practice
• Provide concise explanations supporting each judgement
• Evaluate technical, legal, financial, operational, or professional content within your area of expertise
• Identify inaccurate standards, figures, regulations, assumptions, terminology, or constraints
• Detect subtle domain-specific problems that may not be obvious to non-specialists
• Review consistency between instructions, supporting information, and completed outputs
• Apply current professional judgement when assessing quality and correctness
• Review existing yes/no grading decisions against the completed work
• Assess issues such as formatting, requirement compliance, fabricated information, terminology, and content accuracy
• Identify grading decisions that are unsupported or inconsistent with the evidence
• Explain clearly where and why you agree or disagree with an evaluation
• Apply consistent standards across reviewed work products
• Independently evaluate the overall quality of each completed work product
• Determine whether a practitioner could realistically use or accept the output
• Assign an appropriate quality band based on professional standards
• Compare your assessment with the reported evaluation score
• Identify cases where reported scores do not accurately reflect work-product quality
• Provide short, clear rationales for individual review decisions
• Explain professional concerns in precise business English
• Distinguish material problems from minor presentation issues
• Support judgements with relevant professional standards or practical experience
• Maintain consistency and attention to detail throughout each review

Benefits

• Part-time independent contractor engagement
• Fully remote
• Flexible scheduling based on project requirements
• Compensation: $60–$75/hour
• Approximately one hour of review work per item
• Open to qualified professionals based in Australia, Canada, Ireland, New Zealand, the United Kingdom, and the United States
• The role involves reviewing completed work rather than independently performing or recreating the underlying task
• Projects may be extended, shortened, or concluded based on project needs and performance
• Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party
• H1-B and STEM OPT support is unavailable for this engagement

What This Job Offers

• Job Type: Part-time
• Career Level: Mid Level
• Education Level: No Education Listed