Principal Architect - Reinforcement Learning (Physical AI)

Caterpillar Inc.•Irving, TX

3d•$147,760 - $240,110•Onsite

About The Position

The Principal Architect – Reinforcement Learning (RL) Systems owns the architecture of Caterpillar’s learning systems for Physical AI. The role defines how simulation, digital twins, and fleet data are converted into high-performance, field-proven policies and models for perception, prediction, planning, and control on excavators, dozers, loaders, and haul trucks. It owns the learning problem, including task formulation, policy and model architecture, closed-loop training, sim-to-real transfer, evaluation, safety, and deployment onto machines. The architect establishes the reference architecture for learning-based autonomy at Caterpillar. This includes Gym-style task and environment specifications using Isaac Lab / Isaac Sim, Omniverse, and hybrid neural simulation; GPU-parallel reinforcement and imitation learning; foundation-model post-training; domain randomization, curricula, and system identification; reward and metric design; and policy evaluation, regression, and safety-case practices that gate deployment. The role sets standards for reproducibility, experiment rigor, and benchmark hygiene, and partners with the Experimentation & Operations architect to conduct training and evaluation at scale. Success is measured by field task performance of deployed policies, not simulation benchmarks alone. The position develops architecture deliverables, designs application technical infrastructure, leads evaluation and deployment of new technologies, and collaborates with cross-functional teams to deliver digital solutions aligned with business requirements. It is based onsite at Caterpillar’s Irving, Texas office five days per week. Domestic relocation assistance and visa sponsorship are available.

Requirements

Knowledge of requirements-analysis tools, methods, and techniques, with the ability to elicit, analyze, and record business functionality and non-functional requirements.
Knowledge of analytical techniques and tools, with the ability to identify root causes and create alternative solutions.
Knowledge of platform architecture technologies and methods, with the ability to design processing mechanisms, roadmaps, and supportive end-user interfaces in accordance with standards and processes.
Knowledge of target architecture and the ability to develop IT blueprints and roadmaps aligned with business strategies and objectives.
Effective communication skills, including the ability to transmit, receive, and accurately interpret ideas, information, and needs.
Typically 8+ years in robot learning with reinforcement learning taken from algorithm development through deployment on real hardware, or a PhD plus 5+ years.
Experience with reinforcement learning methods such as PPO, SAC, GRPO, model-based or world-model RL, reward shaping, curricula, and managing compounding error in closed-loop training.
Experience architecting large-scale closed-loop training frameworks on GPU-parallel simulators, with tools such as Isaac Lab / Isaac Sim, MuJoCo, or Brax, and orchestration such as Ray, Kubernetes, or Slurm.
Experience with sim-to-real transfer, including domain randomization, system identification, actuator and latency modeling, and real-to-sim reconstruction.
Strong Python and PyTorch skills; working C++ for real-time integration; CUDA familiarity; experiment tracking; and Bazel or monorepo tooling and CI/CD.
Architecture leadership experience turning research into repeatable engineering, setting reproducibility and benchmark standards, mentoring ML engineers and researchers, and leading a shipped learning-based capability.
Ability to work full-time onsite at the Irving, TX office five days a week.

Nice To Haves

Experience with imitation learning, including behavior cloning, offline RL (such as IQL or GAIL), distillation, and diffusion policies.
Experience with vision-language-action or robot foundation models and RL-based post-training such as RLHF, DPO, or GRPO.
Experience using neural or hybrid simulation, including NVIDIA Cosmos or neural reconstruction, for training or augmentation.
Experience with hydraulic or heavy-machinery dynamics.
Experience designing policy metrics for safety, progress, productivity, and comfort; statistically rigorous policy comparisons; continuous evaluation and regression; adversarial and long-tail scenario testing; runtime safeguards; and safety-case approaches such as ISO 26262, SOTIF, or ISO 13849.
Foundations in controls and robotics, including MPC, impedance control, trajectory optimization, learned-versus-classical approach selection, and real-time perception-to-action pipelines.
Experience shaping demonstration and fleet-data collection, auto-labeling, and synthetic-data strategies for imitation and offline RL.
JAX experience.
Familiarity with off-highway, construction, or mining autonomy, or robot learning at organizations listed in the posting, including NVIDIA (Isaac / GEAR), Waabi, Wayve, Zoox, Figure, 1X, Agility, Boston Dynamics, or Tesla Optimus.
Master’s or doctoral degree in Computer Science, Robotics, Machine Learning, Controls, or a related field.
Research or publication record.

Responsibilities

Develop detailed architecture deliverables to solve business problems.
Design application technical infrastructure, including databases, programming languages, utilities, and testing approaches.
Lead the evaluation and deployment of new technologies that add to or enhance existing digital technical capabilities.
Address business requirements for applications and collaborate with cross-functional teams to deliver digital solutions that meet business results.
Define the reference architecture for learning-based autonomy and the end-to-end learning system that produces Physical AI models.
Formulate learning tasks as observations, actions, rewards, and curricula; select policy and model architectures; and structure large-scale closed-loop training in simulation.
Establish methods for sim-to-real transfer and measure their effectiveness.
Set standards for reproducibility, experiment rigor, and benchmark hygiene.
Define policy evaluation, regression, and safety-case methodology that gates deployment.
Partner with the Experimentation & Operations architect to run training and evaluation at scale.
Turn research into repeatable engineering and deliver learning-based capabilities whose success is measured by field task performance.