Accounting Expert: AI Model Evaluation

Huzzle

2d • $60 - $90

About The Position

Huzzle Labs is seeking experienced accountants to create realistic accounting tasks that evaluate how well frontier AI systems handle accounting work. The role involves designing challenging prompts, running them against a frontier model, carefully reviewing the model’s output, and writing objective scoring rubrics that let other reviewers assess results consistently. Typical deliverables include spreadsheets, journal entry schedules, or short memos. Training is provided, and an NDA must be signed before the engagement begins. The initial engagement is a one-week trial, with potential extension.

Requirements

• At least 5 years of experience in accounting, bookkeeping, or finance operations.
• Prior AI training or data-annotation experience.
• Hands-on knowledge of month-end close, reconciliations, AR/AP, accruals and adjusting entries, sales tax, and job costing.
• Excellent written English and precision with numbers.
• Careful, detail-oriented work, including accurate figures and objective, checkable criteria.
• Availability for 20 to 40 hours per week during an initial one-week trial.

Responsibilities

• Create realistic accounting tasks covering month-end close, bank and credit card reconciliations, AR/AP ageing, accruals and adjusting entries, sales tax, job costing, and cleanup of incomplete or inconsistent books.
• Review deal-room files, select relevant documents, and use them to develop prompts and scoring rubrics.
• Design requests that call for usable deliverables, usually a spreadsheet, journal entry schedule, or short memo.
• Run each request against a frontier AI model and review its output carefully, as when signing off accounting work.
• Increase task difficulty when the model succeeds until a genuine performance gap appears.
• Write objective, checkable scoring criteria that another reviewer can apply consistently.
• Follow task and rubric guidelines, flag ambiguous cases, and help refine the criteria.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed