🌍 Intelligence Analyst, Remote- Contract

Xperteez Technology • ,

4d • $30 - $80 • Remote

About The Position

The Intelligence Analyst will apply domain expertise to help train next-generation AI systems. The work involves developing and mapping analytic cases to defined taxonomies across evaluation areas, authoring complex test cases, assessing and grading AI-generated responses, identifying over-refusals, and documenting edge cases and analytic disagreements. The analyst will also collaborate with project leads to refine source-evaluation and confidence-level procedures and contribute to calibration of analytic standards and AI system benchmarks. No prior AI experience is required.

Requirements

• Analytic rigor.
• Source handling.
• Over-refusal detection.
• All-source intelligence analysis.

Nice To Haves

• Demonstrated all-source analytic experience or deep expertise in a specific analytic discipline, ideally with a civilian-IC background.
• Strong proficiency in source evaluation, confidence-level assessment, and established analytic standards.
• Experience teaching, mentoring, or leading analytic reviews, with a record of fostering analytic rigor.
• Exceptional written and verbal communication skills, with a focus on clarity and precision.
• Ability to identify and articulate nuanced model limitations and over-refusal scenarios.
• Familiarity with structured analytic techniques and intelligence tradecraft standards.
• Previous participation in AI evaluation or model assessment initiatives.

Responsibilities

• Develop and map analytic cases to defined taxonomies across evaluation areas to provide comprehensive coverage of relevant intelligence scenarios.
• Author complex test cases evaluating model performance in source handling, analytic rigor, and nuanced uncertainty expression.
• Assess and grade AI-generated responses for reasoning, source use, recognition of limitations, and practical utility.
• Identify and flag cases where the model over-refuses legitimate analytic exploration, documenting the issue and providing feedback.
• Document edge cases and analytic disagreements to support continuous improvement of evaluation methodologies.
• Collaborate with project leads to refine procedures for source evaluation and confidence-level assignment.
• Contribute insights to calibrate analytic standards and improve AI system benchmarks.

What This Job Offers

• Education Level: No Education Listed