Senior Software Engineer, AI Training - United States
About The Position
This is a contract, remote role for a highly experienced software engineer to evaluate the quality of interactions with modern AI coding agents, including OpenAI Codex, Claude Code, and Cursor. It is not a traditional engineering role and does not involve writing production code. The focus is on assessing engineering judgment, the usefulness of explanations and reasoning, and whether an interaction feels appropriate and trustworthy to an experienced developer. The engagement is ongoing, with individual projects lasting about two weeks to a few months. Evaluators who perform well may be offered new projects. The role offers flexible hours of 10–20 per week and a rate of $100–$200 per hour, depending on experience and location. The start date is as soon as the candidate clears a take-home evaluation and a project has an open seat. The selection process consists of one take-home evaluation exercise with a recorded Loom walkthrough and no interview. Eligibility may vary by location, as some regions may not be supported by the client.
Requirements
- Senior, Staff, or Principal-level engineering experience, or equivalent experience.
- Strong background in TypeScript/JavaScript or Python.
- Hands-on experience with at least one of OpenAI Codex, Claude Code, or Cursor.
- Deep familiarity with modern AI-assisted development workflows.
- Ability to evaluate code without executing it or reviewing every line.
- Strong written and spoken English at B2 level or above.
- Comfort giving direct, opinionated feedback.
- High standards for engineering quality.
- Ability to make subjective but rigorous judgments and explain them clearly.
Nice To Haves
- Prior exposure to prompt design or evaluation workflows.
- Experience mentoring senior engineers or defining engineering standards.
Responsibilities
- Evaluate AI-generated coding interactions end to end.
- Judge whether outputs are useful, correct at a high level, and aligned with how a strong engineer would think.
- Assess the quality of explanations and reasoning, not just the code.
- Distinguish between different levels of response quality and explain what makes a response stronger or weaker.
- Provide clear, opinionated written feedback on what worked, what did not, and what felt off or misleading.
- Help define what excellent interactions look like when working with tools such as Cursor, Codex, and Claude Code.
- Record short video explanations of evaluations.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Part-time
Career Level
Senior
Education Level
No Education Listed
