Senior Consultant - AI Training & Evaluation (MBB & Top-Tier Firms)

Mindrift•Austin, TX

7d•$60•Remote

About The Position

Toloka AI, through its Mindrift initiative, is launching a Management Consulting domain focused on translating real-world consulting engagements into structured learning environments for advanced AI systems. This role is for strategy consultants from top-tier firms who can convert authentic project experience into end-to-end examples, covering problem structuring, work planning, analysis, synthesis, and client-ready recommendations. You will join a growing team of consultants from leading strategy firms shaping how AI learns high-level business reasoning. This role is exclusively for consultants with direct experience at a top-tier strategy consulting firm, ensuring the domain is built by practitioners trained to the highest standards of structured problem-solving and client delivery.

Requirements

3+ years at McKinsey, BCG, Bain, Oliver Wyman, Roland Berger, Monitor Deloitte, EY-Parthenon, Kearney, or Strategy&
Hands-on project experience in structuring ambiguous client problems into workable analytical plans
Hands-on project experience in building financial models, market analyses, or synthesized findings from messy inputs
Hands-on project experience in producing client-ready deliverables under time pressure
Hands-on project experience in forming and defending recommendations under uncertainty
Strong structured problem-solving and hypothesis-driven thinking
Ability to translate vague problems into clear analytical steps and deliverables
High attention to logical consistency and output quality
Independent, self-directed working style
Clear written English (B2+)

Nice To Haves

No deep technical background is required — we will onboard you on the lightweight tools involved.

Responsibilities

Build realistic consulting project environments — create detailed project scenarios grounded in real engagement dynamics: industry context, financials, constraints, conflicting inputs, and incomplete information.
Design structured consulting tasks for AI agents — break projects into discrete tasks that mirror real consulting work: market sizing, commercial due diligence, cost optimization, growth strategy, operational diagnosis, benchmarking, and more.
Define evaluation criteria and quality standards — develop grading frameworks, evaluation rubrics, and golden-answer solutions for each task, used to train and calibrate an LLM-based grading system that evaluates AI outputs at scale.