Domain Expert, AI Evaluation Task Writer: Unreal Game Development (Contract, Remote)

Patronus AI, Inc.

5d • $50 - $150 • Remote

About The Position

Patronus AI is hiring a remote, independent contractor to author and review Unreal Engine 5 evaluation tasks for AI models. Working from a brief for a project area, you will create realistic tasks an experienced developer, designer, or artist could complete with the appropriate project and tools. You will document assets and project files, implement or document a reference solution, and define objective acceptance checks and scoring criteria that enable independent graders to reach consistent conclusions. Tasks are intended to represent about one working day of developer effort, and the resulting work will be used as evaluation and training data for frontier AI models.

The role covers gameplay programming, technical design, technical art, and level design. You will work in UE 5.8, build and verify task materials and reference solutions, test current models and adjust task difficulty, review other contributors’ tasks, and incorporate reviewer feedback. This is a fully remote, asynchronous engagement requiring at least 10 hours per week. Pay is $50–$150 per hour for task authoring, paid upon submission of a complete task, with a potential bonus for each accepted task. The paid trial task is followed by onboarding and a first project; the start is rolling after passing the trial.

Requirements

• At least 3 years of hands-on Unreal Engine development in one or more of gameplay programming, technical design, technical art, or level design; must have built the work personally rather than only directed it.
• Working command of Unreal Engine in the relevant area, such as the gameplay framework and Gameplay Ability System, material and Niagara systems, or greyboxing, nav mesh, and encounter scripting.
• Ability to open a project in UE 5.8, rebuild it, and work in both C++ and Blueprint.
• Habit of verifying behavior in the editor and a packaged build, and citing the log line, callstack, asset, or setting supporting each conclusion.
• Clear, precise professional English writing.
• Ability to work independently and asynchronously against deadlines.
• A Windows machine capable of running Unreal Engine 5.8.
• All tasks, project files, and solutions must be human-authored; LLM-generated code, content, or text is not permitted.
• Tasks and solutions must be original and must not copy published or easily searchable sources, including tutorials and marketplace content.
• Do not include code, assets, or project files from a current or former employer, client, or studio, or anything covered by an NDA; studios, projects, and scenarios must be fictional, and all code and content must be your own.

Nice To Haves

• At least one shipped UE5 title, or several years working in the same role on a single UE5 project, ideally a shooter, action, or multiplayer game.
• Experience with UE5 systems beyond the core area, such as Nanite, Lumen, World Partition, PCG, MetaSounds, Mass, or Control Rig, and with shipping on more than one platform.
• Experience with adjacent tools such as Perforce or Plastic SCM, Unreal Build Tool, Unreal Automation Tool, Unreal Insights, RenderDoc, or a CI system such as Horde or Jenkins.
• Prior experience writing or reviewing AI evaluation tasks, technical interview questions, tutorials, courses, or engine documentation.
• Familiarity with common AI model failures on engine work, including nonexistent or changed APIs, unbuildable Blueprint instructions, fixes that compile but break replication or packaged builds, and inaccurate claims about settings.
• Some overlap with US working hours.

Responsibilities

• Author Unreal Engine tasks from short project-area briefs, writing them as instructions for a mid-level colleague.
• List the assets, Blueprints, and source files involved, and write acceptance checks that testers can verify objectively.
• Build required project materials, including Blueprints, C++ classes, data assets, materials, Niagara systems, levels, and test maps; provide build, cook, and log output showing the starting state.
• Implement end-to-end reference solutions in UE 5.8, deliver changed files, and record each step and how every check was verified.
• Write explicit scoring criteria, including acceptable alternative implementations, so independent graders can reach consistent verdicts.
• Run tasks against current AI models, identify where and why they fail, and adjust task difficulty.
• Review other contributors’ tasks by opening the project, assessing realism and whether checks evaluate the intended requirements, and citing evidence for each score.
• Incorporate reviewer feedback within agreed turnaround times and flag guideline gaps or edge cases.

What This Job Offers

• Career Level: Mid Level
• Education Level: No Education Listed