RL Environment Engineer

Replay•New York, NY

9d•$220,000 - $320,000

About The Position

Replay is hiring an RL Environment Engineer to build production-grade reinforcement learning environments from real enterprise workflows and de-identified business data. The role involves creating realistic, verifiable training environments—including tasks, tools, and graders—for frontier AI labs, and working directly with lab researchers to identify capabilities they want to improve. This is a hands-on engineering position with substantial ownership. The engineer will help establish standards and patterns for how Replay builds tasks and environments.

Requirements

3–5+ years of ML or software engineering experience shipping production systems.
Personally built production-grade RL environments used in real training or evaluation, not only prototypes or research demos.
Ability to turn ambiguous researcher goals into well-scoped, working systems.
Clear understanding of environment design tradeoffs, including task specification, state and tool design, determinism, difficulty calibration, and solvability.
Experience building evaluation or agentic harnesses, with familiarity with evaluation rigor including precision, recall, calibration, imperfect labels, and distribution shift.
High-ownership, action-oriented mindset and ability to take problems from idea to shipped.
Startup experience, broad ownership, and ability to work effectively as requirements evolve.
Clear written and verbal communication with researchers, engineers, and non-technical partners.

Nice To Haves

Background at or working with a frontier AI lab, or research-adjacent engineering experience.
Experience in forward-deployed or customer-facing engineering roles.
Experience with sensitive enterprise data or other high-trust production systems.
Previously served as a founder or early employee at a startup.

Responsibilities

Design and build production-grade RL environments grounded in real enterprise workflows, from task specification through stable, versioned release.
Write reward functions and programmatic verifiers that reliably score outcomes and resist reward hacking and shortcut solutions.
Build high-fidelity evaluation environments with seeded failure modes to expose subtle regressions, along with reliable model and agent harnesses that have bounded behavior and explicit output verification.
Turn Replay's de-identified data into realistic agent states, tools, files, and context; maintain reproducibility, observability, safe rollouts, and clear records of changes between versions.
Work with frontier lab researchers to understand capability gaps and design task suites to expose and train against them.
Measure task difficulty and solvability by running frontier models against environments, then tune environments to provide a usable training signal.
Ship quickly and help set standards and patterns for building tasks and environments at Replay.