Senior World Sensing Engineer

Lightspeed•Northbrook, IL

About The Position

LightSpeed Build Technologies is hiring a Senior World Sensing Engineer to own how its robots sense and understand the world. The engineer will architect sensor arrays across robot families—including factory cells, mobile DEX robots, and roof configurations—and build the pipelines and shared world model that turn sensor signals into reliable estimates of object locations, states, and uncertainty. The role covers sensor selection and placement, time synchronization, calibration, data capture and streaming, sensor-health monitoring, world modeling, sensor fusion, localization, and perception. The engineer will register BIM to as-built environments, maintain element states using perception and task verification, and provide interfaces used by planning, robot primitives, teaching, and learning systems. They will also set the direction for generalized perception, develop material-state measurement and pose-estimation capabilities, establish task-linked accuracy targets and benchmarks, and deploy perception models at the edge within latency budgets. The Senior World Sensing Engineer will set sensing and perception architecture across both robot families, mentor engineers, and work closely with robot learning and agentic planning principals. The position is hands-on, including writing production code and testing on real robots every week. Success includes enabling new trades to be taught through the skill pipeline without depending on the engineer’s direct involvement.

Requirements

At least 5 years of experience building perception, sensing, or state estimation for physical robots or autonomous systems, with systems deployed in production.
Experience architecting a multi-sensor suite end to end, including selection, placement, synchronization, calibration, and the data pipeline.
Deep expertise in 3D vision and geometry, including multi-view geometry, point-cloud registration, SLAM, and sensor fusion across cameras, lidar, and IMU.
Experience building a world or scene representation consumed by planning or control, not only detectors.
Deep-learning experience for perception in PyTorch, from dataset development through deployed model.
Strong C++ and Python skills.
Experience with ROS2, Linux, Docker, and CI/CD workflows.
A record of setting technical direction across teams and mentoring senior engineers.

Nice To Haves

MS or PhD in Robotics, Computer Vision, Computer Science, or a related field.
Experience with scan-to-CAD or scan-to-BIM comparison, as-built verification, or digital twins.
World modeling, mapping, or localization experience on an autonomous-vehicle or field-robotics team.
Perception experience for manipulation, including deformable objects or surface measurement.
Experience with vision foundation models such as SAM, DINOv2, or FoundationPose.
Experience with observation spaces for imitation learning or vision-language-action policies.
Experience with sensor data for robot-learning datasets, such as MCAP or rosbag.
Experience with hardware time synchronization (PTP or hardware triggers), CUDA, TensorRT, and edge deployment on NVIDIA Jetson.
Experience with simulation and synthetic data using Isaac Sim, MuJoCo, or similar.
Experience with a shared world model across multiple robots.
Background in construction, manufacturing, or industrial automation.

Responsibilities

Define and evolve sensor suites across robot families, including factory cells, mobile DEX robots, and roof configurations.
Select and position sensors to provide suitable coverage while accounting for occlusion and changing light.
Own time synchronization across sensors, robot state, and teleoperation streams.
Develop and maintain intrinsic, extrinsic, hand-eye, and camera-lidar calibration strategies, including reliability through tool changes and daily operation.
Build sensor-data capture, streaming, compression, and logging pipelines for the world model, teleoperation, and robot learning.
Monitor sensor health and detect drift before it affects tasks.
Design a world model with an element graph keyed to BIM, poses with uncertainty, and element states such as planned, placed, fastened, verified, or failed.
Register BIM to as-built environments to tool tolerance, for factory panels and on-site rooms.
Build sensor fusion and localization that enable mobile robots to determine their position relative to a building.
Keep element states current using perception and verification results from robot primitives.
Own the world-model interface used by planning, primitives, and teaching, with support for sharing across robots.
Set the direction for a generalized perception model for geometry and categories that can learn new features from a few examples.
Develop material-state measurement for cable and tape position, mud and surface flatness, foam depth, and fastener seating.
Build object detection, segmentation, and 6-DoF pose estimation for parts, tools, and fasteners in cluttered, confined, and poorly lit environments.
Set accuracy targets tied to task success and build benchmarks to measure performance.
Deploy perception at the edge within a defined latency budget.
Co-own foundation interfaces with robot learning and agentic planning principals, and own the world-model interface.
Add features and sensing required by each new trade as it advances through the skill pipeline.
Work with the robot learning principal to decide what data is collected and how failures are labeled.
Set sensing and perception architecture across both robot families.
Mentor perception and sensing engineers and build systems that enable engineers to teach new trades without direct assistance.
Write production code and test on real robots every week.