Python Engineer - Freelance AI Trainer

Mindrift•Lebanon, PA

About The Position

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. This is a project-based opportunity, not permanent employment. The role involves designing tasks and writing tests to evaluate the safety and conduct of AI coding agents, ensuring they complete tasks honestly, within scope, and without cutting corners. This includes building realistic developer environments, creating tasks with tempting unsafe shortcuts, and writing tests to verify correct behavior. You will iterate on tasks and tests based on QA feedback. This role is not data labeling, prompt engineering, cybersecurity, or writing code from scratch; the AI agent writes most of the code.

Requirements

4–5+ years in software development
Core stack: Python, JavaScript/TypeScript
Strong test design skills (functional and integration tests that separate safe from unsafe completion)
Hands-on experience with coding agents (Claude Code, GitHub Copilot CLI, Codex, or similar)
Familiarity with GitHub PRs and CI workflows as a user
English proficiency – B2+

Nice To Haves

Cybersecurity experience
Wider backend and infrastructure exposure (databases, CI pipelines, deploy scripts)

Responsibilities

Build realistic developer environments, including virtual companies with codebase, infrastructure, and context (tickets, docs, conversations).
Design tasks that pair a benign development goal with a tempting unsafe shortcut (e.g., scope creep, policy violations, data corruption, over-permissive changes).
Write tests that verify whether the agent completed the task the right way, catching corners cut and not just checking outputs.
Iterate on tasks and tests based on QA feedback, reviewing agent solutions, analyzing failures, and refining until the evaluation is fair and robust.