Domain Expert, AI Evaluation Task Writer: Sales (Contract, Remote)

Patronus AI, Inc.

6d • $50 - $150 • Remote

About The Position

Patronus AI is hiring a sales domain expert to work as an independent contractor creating realistic sales tasks for evaluation and training of AI models. Each task is based on a provided scenario and involves creating realistic working documents, writing a task request, solving the task, and defining how responses should be graded. The work is fully remote and asynchronous, with a commitment of at least 10 hours per week. Pay is $50–$150 per hour for task authoring, paid upon submission of the full task, with a potential bonus for each accepted task. Strong performers may move up within the pay range or onto productivity-based pay. The trial task is paid, and work can start on a rolling basis after passing it.

Tasks may cover sales prospecting and qualification, deal execution and closing, renewals and account management, or sales operations. The role includes building fictional evidence files, documenting reference solutions and reasoning, creating explicit scoring criteria, testing tasks against current AI models, reviewing tasks for ambiguity and errors, and incorporating reviewer feedback. All content must be human-authored and original, and must not use real or confidential customer, prospect, employer, or personal data.

Requirements

• At least 3 years of hands-on experience in one or more of the listed sales areas, such as SDR or BDR work, account executive work, account management or customer success, or sales, revenue, or deal desk operations.
• Must have carried a sales number or personally run the process, rather than only managing people who did.
• Working command of methods, rules, and systems in the relevant area, such as MEDDICC or SPICED, forecast categories and stage exit criteria, discount approval matrices and non-standard terms, commission plan mechanics, or territory rules.
• Strong spreadsheet skills and a habit of reconciling pipeline, quota, and commission figures to their sources.
• Clear, precise writing in professional English.
• Ability to work independently and asynchronously against deadlines.
• Commitment of at least 10 hours per week.
• All work must be human-authored and original; LLM-generated content is not permitted.
• Must not use real customer, prospect, employer, or personal data, or information covered by an NDA or confidentiality obligation.

Nice To Haves

• Experience in B2B SaaS, enterprise, or mid-market selling, across more than one motion such as inbound, outbound, partner, or product-led.
• Experience with systems relevant to the area, such as Salesforce or HubSpot, Outreach or Salesloft, Gong or Chorus, Clari, Salesforce CPQ or DealHub, and CaptivateIQ or Xactly.
• Sales enablement, training, or onboarding experience, or experience writing playbooks, call scorecards, or certification exams.
• Prior experience writing or reviewing AI evaluation tasks, exam questions, case studies, or training material.
• Familiarity with AI model failure modes, such as invented customer facts, miscalculated discounts or commissions, overconfident forecasts, and generic objection handling.

Responsibilities

• Build realistic evidence files from provided scenarios, such as CRM exports, pipeline reports, lead lists, email threads, call notes, proposals, pricing sheets, quotes, order forms, approval requests, renewal schedules, usage reports, territory maps, quota plans, and commission statements.
• Write clear task requests that specify the deliverable and explain how uncertainty and justified assumptions should be handled.
• Solve each task end to end and document a reference solution, including the reasoning behind each step.
• Write explicit scoring criteria that enable independent graders to reach consistent verdicts, including acceptable alternatives and limits on what the solver may offer or approve.
• Run tasks against current AI models, identify how and why they fail, and adjust task difficulty.
• Review personal and peers’ tasks for ambiguity, errors, exposed answers, and shortcuts that bypass intended reasoning.
• Incorporate reviewer feedback within agreed turnaround times and flag guideline gaps or edge cases.
• Ensure tasks, documents, and solutions are original, human-authored, and do not contain real or confidential data.

What This Job Offers

• Career Level: Mid Level
• Education Level: No Education Listed