AI Evaluation Engineer – Reinforcement Learning & Agents

Max Corporate Group • San Francisco, CA

19h • Onsite

About The Position

We are seeking an AI Evaluation Engineer – Reinforcement Learning & Agents to build the environments, evaluation systems, and supporting infrastructure used to train and assess long-horizon enterprise AI agents. You will work on the engineering and research problems behind realistic agent environments, post-training systems, and reliable evaluation of complex multi-step workflows.

Requirements

• Hands-on experience with AI environments, evaluations, reinforcement learning infrastructure, or related agent-training systems.
• Strong software engineering fundamentals.
• Demonstrated ability to build and ship technical infrastructure.
• Understanding of evaluation methodology, reward design, graders, and agent trajectories.
• Ability to work across languages and technology stacks based on system requirements.

Nice To Haves

• A PhD is not required. Strong engineering and shipped environment or evaluation systems are more important than academic credentials or publication history.

Responsibilities

• Design evaluation environments for long-horizon enterprise agent workflows.
• Define tasks, state, tools, graders, and reward signals used to evaluate and improve agents.
• Build high-fidelity representations of complex enterprise software environments.
• Develop infrastructure for rollouts, orchestration, trajectory inspection, and grader pipelines.
• Measure both correctness and efficiency across multi-step agent behavior.
• Investigate evaluation failures, reward-quality issues, and agent behavior.
• Build production-quality systems rather than notebook-only research prototypes.

What This Job Offers

• Job Type: Full-time
• Career Level: Entry Level
• Education Level: No Education Listed