Hybrid | Finance AI Evaluation Specialist — $40–$80/hour

24-Mag • New York, NY

1d • Hybrid

About The Position

We are sharing a specialised full-time opportunity for experienced finance professionals with deep expertise in a core financial discipline and the ability to evaluate, structure, and improve advanced AI systems working on real-world finance tasks. This role works closely with AI research and programme teams to improve how advanced models reason about professional financial work. Selected experts will review finance tasks and model outputs, develop instruction specifications and reference solutions, design challenging benchmarks, and translate senior-level financial judgement into clear evaluation standards.

Requirements

• 5+ years of substantive professional experience in finance at a recognised financial institution, investment firm, advisory firm, corporate finance organisation, accounting firm, or financial regulator
• Deep specialisation in at least one core finance discipline rather than primarily generalist exposure
• Relevant expertise may include corporate finance, FP&A, investment banking, M&A, asset management, wealth management, private equity, private credit, quantitative finance, risk management, treasury, accounting, or audit
• Demonstrated progression to a senior individual-contributor or leadership position such as Vice President, Director, Principal, Portfolio Manager, Controller, CFO, or comparable level
• Direct ownership of significant financial analysis, investment, reporting, risk, or strategic decisions
• Hands-on professional experience using large language models and AI tools
• Strong ability to distinguish technically rigorous financial reasoning from plausible but incorrect analysis
• Excellent written communication and ability to provide precise, structured feedback
• Must live in the San Francisco Bay Area and be able to attend occasional onsite sessions when required

Nice To Haves

• Advanced degree such as an MBA, MS, or PhD in Finance, Economics, Accounting, or a quantitative discipline is strongly preferred
• Professional credentials such as CFA, CPA, FRM, actuarial designation, or comparable qualifications are highly valued

Responsibilities

• Review finance knowledge-work tasks and AI-generated outputs for professional accuracy and analytical quality
• Identify flawed assumptions, missing considerations, weak reasoning, and superficially plausible conclusions
• Evaluate whether financial analyses would withstand professional review in real-world settings
• Assess model performance across complex finance scenarios
• Provide precise written feedback explaining errors and required improvements
• Develop detailed instruction specifications for finance-related AI tasks
• Produce high-quality reference solutions that establish clear standards for correct responses
• Design tasks that accurately reflect how financial work is performed in professional environments
• Translate tacit professional judgement into explicit and reproducible evaluation criteria
• Ensure reference materials demonstrate strong technical depth and practical financial reasoning
• Design challenging finance-specific benchmarks and evaluation sets
• Create tasks that test analytical depth, professional judgement, and domain-specific reasoning
• Develop evaluation criteria that differentiate rigorous financial work from plausible but incomplete responses
• Refine benchmarks based on model performance and research requirements
• Help expand finance-specific capabilities, workflows, and evaluation tools
• Apply specialist knowledge from one or more areas such as corporate finance and FP&A, investment banking and M&A, asset or wealth management, private equity, private credit, quantitative finance, risk management, treasury, accounting, or audit
• Evaluate complex financial problems using professional methodologies and industry standards
• Identify nuanced domain-specific issues that may be missed by generalist reviewers
• Contribute senior-level judgement grounded in substantial professional finance experience
• Maintain high standards of analytical rigour across financial evaluation workflows
• Collaborate with research teams and specialists across adjacent professional domains
• Participate in calibration processes to maintain consistent evaluation standards
• Apply practical experience using large language models to professional finance workflows
• Distinguish genuinely well-reasoned model outputs from responses that are persuasive but technically incorrect
• Communicate complex finance concepts clearly to technical and non-finance stakeholders

Benefits

• Full-time W-2 employment position
• Initial engagement of approximately 6 months
• 40 hours per week
• Day-to-day work is remote with occasional onsite collaboration in the San Francisco Bay Area
• Compensation: $40–$80/hour
• Client-issued accounts and equipment will be provided as required for the role
• Work will be conducted within the assigned organisation's internal tools and workflows

What This Job Offers

• Job Type: Full-time
• Career Level: Senior