Senior World Sensing Engineer

Lightspeed•Northbrook, IL

11d

About The Position

As Senior World Sensing Engineer at LightSpeed Build Technologies, you will own how the company’s robots sense and understand the world. You will architect sensor arrays and the pipeline that combines sensor signals into a trusted world model describing object locations, states, and uncertainty across factory cells, mobile DEX robots, and roof configurations. You will define and maintain sensor suites spanning vision, lidar, contact and tactile sensing, proprioception, tool and process sensing, and site safety and positioning. You will lead sensor selection and placement, time synchronization, calibration, data capture and delivery, and sensor-health monitoring. You will design a BIM-linked world model and its interfaces for planning, robot primitives, teleoperation, learning, and use across multiple robots. The role also sets direction for generalized perception, measurement of material state, object detection and pose estimation, task-related accuracy targets, benchmarks, and edge deployment within latency constraints. You will co-own foundation interfaces with robot learning and agentic planning principals, help add sensing capabilities for new trades, and work with the robot learning principal on data collection and failure labeling. You will set architecture across both robot families, mentor engineers, and remain hands-on by writing production code and testing on real robots weekly.

Requirements

At least 5 years building perception, sensing, or state estimation for physical robots or autonomous systems, with systems deployed in production.
Experience architecting a multi-sensor suite end to end, including sensor selection, placement, synchronization, calibration, and data pipeline.
Deep expertise in 3D vision and geometry, including multi-view geometry, point-cloud registration, SLAM, and sensor fusion across cameras, lidar, and IMU.
Experience building a world or scene representation consumed by planning or control, not only detectors.
Deep-learning perception experience in PyTorch, from dataset development through deployed model.
Strong C++ and Python skills.
Experience with ROS2, Linux, Docker, and CI/CD workflows.
Record of setting technical direction across teams and mentoring senior engineers.

Nice To Haves

MS or PhD in Robotics, Computer Vision, Computer Science, or a related field.
Experience with scan-to-CAD or scan-to-BIM comparison, as-built verification, or digital twins.
World modeling, mapping, or localization experience on an autonomous vehicle or field robotics team.
Perception for manipulation, including deformable objects or surface measurement.
Experience with vision foundation models such as SAM, DINOv2, or FoundationPose.
Experience with observation spaces for imitation learning or vision-language-action policies.
Experience with sensor data for robot-learning datasets, such as MCAP or rosbag.
Experience with hardware time synchronization, including PTP or hardware triggers, CUDA, TensorRT, and edge deployment on NVIDIA Jetson.
Simulation and synthetic-data experience with Isaac Sim, MuJoCo, or similar tools.
Experience with a shared world model across multiple robots.
Background in construction, manufacturing, or industrial automation.

Responsibilities

Define sensor suites for each robot family and upgrade them as new trades require additional signals.
Own sensor selection, position, and orientation to address coverage, occlusion, and changing lighting.
Own time synchronization across sensors, robot state, and teleoperation streams.
Develop and maintain intrinsic, extrinsic, hand-eye, and camera-lidar calibration strategies that remain reliable through tool changes and daily operation.
Build sensor-data capture, streaming, compression, and logging pipelines for the world model, teleoperation, and robot learning.
Monitor sensor health and detect drift before it affects task performance.
Design a BIM-keyed world model representing element poses with uncertainty and element states, including planned, placed, fastened, verified, and failed.
Register BIM to as-built conditions at tool tolerance for factory panels and on-site rooms.
Build sensor fusion and localization that lets mobile robots determine their position relative to a building.
Keep element state current using perception and verification results from robot primitives.
Own the world-model interface used by planning, primitives, and teaching, with support for sharing across robots.
Set direction for a generalized perception model that learns new geometry and category features from a few examples.
Build measurement capabilities for material state, including cable and tape position, mud and surface flatness, foam depth, and fastener seating.
Build detection, segmentation, and 6-DoF pose estimation for parts, tools, and fasteners in cluttered, confined, and poorly lit environments.
Set task-success-based accuracy targets and build benchmarks to measure perception performance.
Deploy perception on edge hardware within a defined latency budget.
Co-own foundation interfaces with robot learning and agentic planning principals and add the sensing and features required as new trades move through the skill pipeline.
Measure success by how quickly a new trade can be taught.
Decide with the robot learning principal what data to collect and how to label failures.
Set sensing and perception architecture across both robot families.
Mentor perception and sensing engineers and build systems that engineers can use to teach new trades independently.
Write production code and test on real robots every week.