Remote | Philosophy Research & AI Benchmark Specialist — $40–$55/hour

24-Mag • New York, NY

2d • Remote

About The Position

We are sharing a specialised part-time consulting opportunity for philosophers and academic researchers with advanced expertise in philosophical reasoning, formal logic, epistemology, ethics, philosophy of science, or philosophy of technology. This role supports an AI research initiative focused on developing rigorous academic benchmarks in philosophy. Selected experts will author or verify challenging multiple-choice assessment content, evaluate conceptual accuracy and difficulty, develop well-supported solution rationales, and help establish high-quality reference standards for advanced AI systems.

Requirements

• PhD or current doctoral candidacy in Philosophy or a closely related discipline
• Master's degree may be considered for candidates with exceptional expertise in a relevant philosophical subdomain
• Strong command of philosophical argumentation and analytical reasoning
• Expertise in one or more areas including Formal Ontology, Knowledge Representation, AI Ethics, Applied Epistemology, Philosophy of Technology, Robotics, or Philosophy of Science
• Strong knowledge of formal logic and relevant philosophical literature
• Ability to construct rigorous, discriminating multiple-choice assessments
• Excellent written English and ability to communicate complex ideas clearly and precisely

Nice To Haves

• Academic research publications or university-level teaching experience is highly valued
• Experience with assessment design, examination development, or academic content review is advantageous
• Strong attention to conceptual precision, ambiguity, and argumentative validity

Responsibilities

• Create original multiple-choice questions that test deep conceptual understanding rather than surface-level recall
• Develop challenging problems within relevant areas of philosophical expertise
• Ensure questions are self-contained, precise, unambiguous, and independently solvable
• Design questions appropriate for undergraduate, advanced undergraduate, and postgraduate difficulty levels
• Produce one correct answer alongside plausible and carefully constructed distractors
• Review pre-written philosophy questions for accuracy, clarity, rigour, and completeness
• Identify ambiguities, conceptual errors, unsupported assumptions, or solvability issues
• Edit questions and answer choices where required
• Verify that the designated correct answer is defensible under the stated assumptions
• Document substantive changes and the reasoning behind them
• Develop clear, structured solution rationales explaining why the correct answer follows
• Apply rigorous philosophical argumentation and formal reasoning where appropriate
• Explain distinctions between the correct response and plausible alternatives
• Ensure reasoning is concise, academically defensible, and suitable for expert review
• Apply appropriate conceptual frameworks and canonical philosophical sources
• Develop content relating to AI ethics and responsible technology
• Apply expertise in applied epistemology and questions involving knowledge, evidence, and justification
• Create problems involving philosophy of technology, robotics, and artificial intelligence
• Explore philosophical issues arising from technological systems and their social implications
• Evaluate arguments involving agency, responsibility, uncertainty, and decision-making
• Create rigorous questions involving formal ontology and knowledge representation
• Evaluate conceptual relationships between entities, categories, properties, and representation systems
• Develop problems in philosophy of science involving explanation, methodology, evidence, and scientific reasoning
• Apply formal logic where relevant to philosophical assessment tasks
• Distinguish subtle conceptual differences through carefully designed answer alternatives
• Provide 1–5 academic references per question where required
• Use reputable sources such as peer-reviewed publications, academic books, and university repositories
• Rate question difficulty using defined assessment standards
• Help maintain consistent academic quality across benchmark content
• Contribute to gold-standard evaluation materials used to assess advanced AI capabilities

Benefits

• Part-time independent contractor engagement
• Fully remote and asynchronous
• Expected availability of at least 10 hours per week
• Flexible scheduling based on project requirements
• Compensation: $40–$55/hour
• Contributors may be assigned either question-authoring or question-verification workstreams
• Projects may be extended, shortened, or concluded based on project needs and performance
• Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party
• H1-B and STEM OPT support is unavailable for this engagement

What This Job Offers

• Job Type: Part-time
• Career Level: Mid Level