Remote | AI Agent Power User — $30–$90/hour

24-Mag•New York, NY

2d•Remote

About The Position

We are sharing a specialised part-time consulting opportunity for experienced professionals who use AI assistants extensively in real-world business workflows to contribute to an advanced AI training and evaluation project. Selected professionals will run realistic, multi-step business scenarios using tools such as ChatGPT and Claude, evaluate the quality and usefulness of AI-generated outputs, document model behaviour, and provide structured feedback that can support improvements in advanced AI systems. The work is designed around authentic professional use cases rather than experimental or casual AI usage.

Requirements

Minimum of 5 years of professional experience in a business function involving technology-enabled problem solving
Completed Bachelor's degree or higher in any discipline
Consistent, hands-on use of ChatGPT and/or Claude for real professional work rather than occasional experimentation
Strong familiarity with AI-assisted workflows, prompt-based work, and practical business use cases
Experience applying, creating, or reviewing evaluation rubrics, QA scorecards, grading criteria, or content-review guidelines
Comfortable integrating AI tools with workplace software and productivity platforms
Excellent written English with the ability to produce clear, precise, and actionable documentation
Strong attention to detail, analytical judgement, and process-improvement skills
Ability to work independently within structured remote and asynchronous project environments
Based in the United States, Canada, United Kingdom, Ireland, Australia, or New Zealand, with U.S.-based applicants strongly preferred
Authorised to undertake contract work in the relevant country
Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party

Nice To Haves

Academic evaluation, quality assurance, hiring assessment, annotation, or similar structured-review experience is advantageous

Responsibilities

Run complex, multi-step professional scenarios using AI assistants such as ChatGPT and Claude
Recreate realistic business workflows involving analysis, communication, documentation, research, and operational tasks
Assess how effectively AI systems perform across each stage of a workflow
Identify where AI-generated outputs align with or diverge from genuine professional requirements
Evaluate whether AI assistance produces useful, complete, and practically actionable results
Evaluate and score AI-generated outputs using defined rubrics, quality criteria, and assessment standards
Review responses for completeness, relevance, accuracy, clarity, and professional usefulness
Apply evaluation standards consistently across varied business scenarios
Identify outputs that appear strong at surface level but fail important workflow or user requirements
Document quality issues, inconsistencies, and recurring model-performance patterns
Write clear, constructive, and actionable feedback based on evaluation findings
Document each step of tested workflows to support transparency and reproducibility
Explain where and why AI performance succeeds or breaks down
Identify recurring strengths, weaknesses, and opportunities for process improvement
Maintain detailed written records that can be used by AI research and product teams
Connect AI assistants with commonly used workplace platforms where appropriate
Test workflows involving tools such as Google Drive, Gmail, Slack, Notion, and related productivity systems
Evaluate how AI behaves across connected, multi-tool business processes
Apply practical knowledge of AI adoption to assess whether workflows reflect realistic professional usage
Help identify opportunities to improve the reliability and usability of AI-assisted work