Remote | History & Political Science AI Benchmark Specialist — $35–$50/hour

24-Mag•New York, NY

2d•Remote

About The Position

We are sharing a specialised part-time consulting opportunity for historians, political scientists, policy researchers, and academic experts with advanced knowledge of historical analysis, political institutions, public policy, and comparative research. This role supports an AI research initiative focused on developing rigorous academic benchmarks in history and political science. Selected experts will author or verify challenging multiple-choice assessment content, evaluate factual and analytical accuracy, develop well-supported solution explanations, and help establish high-quality reference standards for advanced AI systems.

Requirements

PhD or current doctoral candidacy in History, Political Science, International Relations, Public Policy, or a closely related discipline
Master's degree may be considered for candidates with exceptional expertise in a relevant subdomain
Strong command of historiographical methods, political analysis, and comparative reasoning
Expertise in one or more areas including National Security, Public Policy, Business History, Environmental History, or Latin American History
Strong ability to evaluate evidence, competing interpretations, and complex causal arguments
Excellent written English and ability to communicate sophisticated ideas clearly and precisely
Strong attention to factual precision, ambiguity, source quality, and analytical rigour

Nice To Haves

Academic research publications are highly valued
Professional policy, government, think-tank, or national-security experience is advantageous
University-level teaching or academic assessment experience is highly valued

Responsibilities

Create original multiple-choice questions that test deep historical and political understanding rather than surface-level recall
Develop challenging problems within relevant areas of academic or professional expertise
Ensure questions are self-contained, precise, unambiguous, and independently solvable
Design questions appropriate for undergraduate, advanced undergraduate, and postgraduate difficulty levels
Produce one correct answer alongside nine plausible and carefully constructed distractors
Review pre-written questions for factual accuracy, clarity, rigour, and completeness
Identify ambiguity, historical inaccuracies, unsupported assumptions, or solvability issues
Edit questions and answer choices where required
Verify that designated correct answers are well supported by historical or political-science evidence
Document substantive changes and explain the reasoning behind them
Develop problems requiring interpretation of historical evidence, context, causation, and change over time
Apply appropriate historiographical methods and source criticism
Distinguish between competing historical interpretations
Evaluate claims using relevant primary and secondary evidence
Design questions that reward analytical judgement rather than memorisation alone
Create assessment content involving political institutions, governance, public policy, and political decision-making
Evaluate competing policy frameworks and institutional explanations
Develop questions requiring comparative political analysis
Apply political theory and empirical reasoning where appropriate
Assess the relationship between institutions, incentives, actors, and policy outcomes
Develop rigorous questions involving national security strategy and policy
Evaluate historical and contemporary security frameworks
Apply political and historical analysis to strategic decision-making
Create problems involving institutions, state behaviour, conflict, and policy trade-offs
Distinguish plausible alternatives through careful use of evidence and context
Develop or review content involving Business History
Create questions in Environmental History
Apply regional expertise to Latin American History
Evaluate economic, institutional, political, and social developments across relevant periods
Ensure specialised content reflects established scholarship and appropriate historical context
Provide 1–5 academic references per question where required
Use reputable sources such as peer-reviewed publications, academic books, and university repositories
Develop clear, structured solution explanations supporting the correct answer
Rate question difficulty according to defined Medium, Hard, and Expert standards
Help maintain consistent academic quality across benchmark content
Contribute to gold-standard evaluation materials used to assess advanced AI capabilities