Robot Learning Engineer

Lightspeed•Northbrook, IL

13h•Onsite

About The Position

As a Robot Learning Engineer at LightSpeed Build Technologies, you will be responsible for teaching robots to perform complex manipulation tasks by learning from human demonstrations. This role involves owning the entire process from data capture to deployed behavior, including building teleoperation systems, training policies using collected data, and validating these policies on physical robots. You will also contribute to learning from video data of human work. This is a hands-on, fast-paced role focused on ensuring robots can reliably perform their tasks.

Requirements

3+ years of hands-on experience in robot learning, ML for robotics, or closely related production ML.
Experience training policies from demonstrations on physical robots using imitation learning (e.g., ACT, Diffusion Policy, or similar).
Experience collecting teleoperation or demonstration data for robot learning.
Strong PyTorch skills and experience running training, evaluation, and debugging end-to-end.
Track record of evaluating models in the real world and diagnosing failures.
Strong Python; working C++.
Experience with ROS or ROS2 and real robot hardware.
Understanding of manipulator kinematics and control fundamentals.
Proficiency with Linux, Docker, and Git.

Nice To Haves

MS or PhD in Robotics, Machine Learning, Computer Science, or a related field.
Experience fine-tuning VLA or foundation models for robot control.
Experience with dexterous hands, VR hand tracking, or motion retargeting.
Experience building or using leader-follower teleoperation systems.
Experience learning from human video or egocentric data.
Experience building and maintaining large-scale robotics datasets.
Robotics simulation experience with MuJoCo, Isaac Sim, or similar.
Experience with robot learning datasets and tooling (e.g., LeRobot, rosbag, MCAP).
Experience managing data vendors or labeling workflows.
Background in manufacturing, industrial automation, or construction technology.

Responsibilities

Build and improve teleoperation pipelines using leader arms for bulk arm motion and VR hand tracking for fine, finger-level interaction.
Retarget human hand motion to a dexterous robot hand, developing in simulation first and validating on hardware.
Design data collection protocols to ensure demonstrations are consistent, well-structured, and train effectively.
Define valuable data collection metrics and guide technicians in data collection sessions.
Train and fine-tune manipulation policies using imitation learning and vision-language-action (VLA) models.
Select appropriate policy architectures and action representations for multi-step, contact-rich tasks.
Integrate teleoperation data with human demonstration video to enhance policy performance.
Iterate rapidly from data collection to trained policy to real-robot testing.
Define and execute evaluation protocols on physical robots, measuring success rates for each task step.
Diagnose and resolve failures across data, models, and hardware.
Deploy policies on robots using ROS2-based control and optimize inference for real-time performance.
Collaborate with integration engineers to execute policies across full task sequences.
Define dataset formats and data quality standards for training.
Supervise external partners in building data pipeline infrastructure and labeling human demonstration video, including setting specifications and accepting deliverables.
Prototype the extraction of useful training signals from egocentric (head-mounted camera) video in-house before scaling.