Web Research Task Author (Contractor)

Careerflow.ai•,

2d•Remote

About The Position

Careerflow Human Data Labs partners with AI companies to bring real-world professional expertise into their products. You'll create research tasks that evaluate how well AI systems can find verifiable answers across the open web. Each task requires synthesizing multiple independent clues from public English sources, each supported by exact quotes. Your work is tested against frontier models and reviewed for rigor—only tasks with bulletproof sources and appropriate difficulty count. This is a remote, contract position, taking approximately four hours per completed task.

Requirements

3+ yrs of experience with Masters Degree
Advanced web research: search operators, reverse lookups, direct access to institutional catalogues and databases.
Familiarity with archives and catalogues: libraries, museums, legislative and court records, registers, specialized databases (sports, film), Wayback Machine, preprint servers.
Strong source evaluation: distinguishing primary from secondary sources, open from restricted, original from republished, stated facts from implications.
Precise citation: pulling verbatim text from web pages, PDFs, scans, and tables—typos included.
Clue architecture: designing constraints that are individually searchable yet jointly pinpoint the answer; identifying logical chains, information leaks, and undefined terms.
Clear, fluent written English.
Comfort using search engines and LLM tools—with a commitment to fact-checking every AI-generated claim against the actual source.
Demonstrated sourced research background: journalism or fact-checking, library/archival work, genealogy, due diligence, graduate-level primary-source research, or citation-heavy editing.
Availability: 20+ hours per week, any time zone; NDA required (materials are confidential client IP).

Nice To Haves

Deep expertise in history, science/technology, sports, music, politics, geography, film, or television.
OSINT or cataloguing experience.
Prior work creating benchmarks, exams, or puzzle content.

Responsibilities

Craft a 60–120 word prompt that feels like a genuine question—original phrasing, no copied content, no search hints.
Research four to six clues across different reputable open English sources (excluding Wikipedia and paywalled content), each with a direct quote, URL, and verification date.
Identify a definitive answer (name, date, catalogue number, or documented title) and clearly explain why alternatives don't fit.
Document plausible wrong answers and which clue eliminates each; articulate your search path.
Verify all sources while logged out; refine tasks that models solve too easily or that reviewers find unclear.