Principal Architect - Reinforcement Learning (Physical AI)

Caterpillar•Irving, TX

2d•Onsite

About The Position

The Principal Architect – Reinforcement Learning (RL) Systems owns the end-to-end learning system that turns simulation, digital twins, and fleet data into high-performance, field-proven Physical AI policies and models. The role defines learning pipelines for reinforcement learning, imitation learning, and foundation-model post-training to train perception, prediction, planning, and control policies for Caterpillar machines, including excavators, dozers, loaders, and haul trucks. The architect defines how learning tasks are formulated through observations, actions, rewards, and curricula; selects policy and model architectures; structures large-scale closed-loop training in simulation; develops and measures sim-to-real transfer; and establishes how learned policies are evaluated, made safe, and deployed onto machines. The reference architecture includes Gym-style task and environment specifications using Isaac Lab, Isaac Sim, Omniverse, and hybrid neural simulation; GPU-based RL and imitation learning at fleet-data scale; vision-language-action and foundation-model post-training with RL; and domain-randomization, curriculum, and system-identification strategies. The role also establishes reward and metric design tied to productivity, safety, machine health, and operator-quality outcomes, along with policy evaluation, regression, and safety-case methods that gate deployment. It sets standards for reproducibility, experiment rigor, and benchmark hygiene, partners with the Experimentation & Operations architect on training and evaluation at scale, and turns research into repeatable engineering. Success is measured by field task performance of deployed policies, not simulation benchmarks alone. The position is full-time and requires working onsite five days a week at Caterpillar’s Irving, Texas office. Domestic relocation assistance and visa sponsorship are available for eligible applicants.

Requirements

Knowledge of requirements-analysis tools, methods, and techniques, with the ability to elicit, analyze, and record business and non-functional requirements.
Knowledge of analytical techniques and tools, with the ability to identify root causes and develop alternative solutions.
Knowledge of platform-architecture technologies and methods, with the ability to design processing mechanisms and roadmaps for business application systems.
Knowledge of target architecture, with the ability to develop IT blueprints and roadmaps aligned with business strategies and objectives.
Effective communication skills, including the ability to transmit, receive, and accurately interpret ideas, information, and needs.
The position requires full-time, onsite work five days a week at the Irving, Texas office.

Nice To Haves

Typically 8+ years in robot learning with reinforcement learning taken from algorithm development through deployment on real hardware, or a PhD plus 5+ years.
Experience with PPO, SAC, GRPO, model-based or world-model RL, reward shaping, curriculum, and compounding-error handling in closed-loop training.
Experience with behavior cloning, offline RL (including IQL and GAIL), distillation, diffusion policies, or foundation-model post-training with RL (including RLHF, DPO, or GRPO).
Experience architecting large-scale closed-loop RL frameworks on GPU-parallel simulators such as NVIDIA Isaac Lab/Isaac Sim, MuJoCo, or Brax, with Ray, Kubernetes, or Slurm.
Experience using neural or hybrid simulation, such as NVIDIA Cosmos or neural reconstruction, for training or augmentation.
Experience closing sim-to-real gaps using domain randomization, system identification, actuator and latency modeling, and real-to-sim reconstruction, measured by field task success; hydraulic or heavy-machinery dynamics experience is highly valuable.
Experience with policy safety, progress, productivity, and comfort metrics; statistically rigorous policy comparison; continuous evaluation and regression; adversarial and long-tail scenario testing; runtime safeguards; and safety-case approaches such as ISO 26262, SOTIF, or ISO 13849.
Foundations in MPC, impedance control, and trajectory optimization, with sound judgment about learned versus classical approaches and perception-to-action pipelines under real-time constraints.
Experience shaping demonstration and fleet-data collection, auto-labeling, and synthetic-data strategies for imitation and offline RL.
Strong Python and PyTorch skills; JAX is a plus. Working C++ for real-time integration, CUDA familiarity, experiment tracking, and Bazel or monorepo tooling and CI/CD experience are also relevant.
Architecture leadership translating research into repeatable engineering, setting reproducibility and benchmark standards, mentoring ML engineers and researchers, and having led a shipped learning-based capability.
Exposure to off-highway, construction, or mining autonomy, or robot learning at organizations such as NVIDIA (Isaac/GEAR), Waabi, Wayve, Zoox, Figure, 1X, Agility, Boston Dynamics, or Tesla Optimus.
Master’s or doctoral degree in Computer Science, Robotics, Machine Learning, Controls, or a related field; research or publication record is a plus.

Responsibilities

Develop detailed architecture deliverables to solve business problems.
Design application technical infrastructure, including databases, programming languages, utilities, and testing approaches.
Lead evaluation and deployment of new technologies that add to or enhance existing digital technical capabilities.
Address business requirements for applications and collaborate with cross-functional teams to deliver digital solutions that meet business results.
Define the reference architecture for learning-based autonomy, including task formulation, policy and model architectures, simulation training, sim-to-real transfer, and policy evaluation and deployment.
Set standards for reproducibility, experiment rigor, and benchmark hygiene.
Partner with the Experimentation & Operations architect to run training and evaluation at scale.
Establish policy evaluation, regression, and safety-case methodologies that gate deployment.
Turn research into repeatable engineering and deliver policies measured by field task performance.