AI Automation Engineer, Real-World Test Lab

Niantic Spatial•San Francisco, CA

9d•$158,400 - $210,000•Hybrid

About The Position

Niantic Spatial is building the future of physical AI with groundbreaking mapping technology that unlocks a new dimension of interaction and spatial intelligence. The Real-World Test Lab plays a crucial role in bridging the gap between benchmarks and real-world application by simulating customer environments and conditions to rigorously test the company's technology. This role is for an AI Automation Engineer who will design and own the end-to-end automation system for lab evaluations, transforming manual, inconsistent, and slow processes into a reliable, always-on service. The focus is on owning the evidence produced by the system and ensuring its trustworthiness, rather than just executing tests. The ideal candidate believes evaluation is engineering and has experience building robust testing infrastructure.

Requirements

Built and maintained production-grade automation or test infrastructure that other engineers relied on daily.
Experience evaluating systems where correctness is graded rather than binary, meaning quality, accuracy, or latency thresholds rather than pass/fail assertions.
Strong Python, with fluency in orchestration, CI-style pipelines, cloud storage, and reproducible environments.
Worked with backend services and APIs you did not own, integrating against them without becoming a bottleneck for their team.
Written up a technical finding clearly enough that a non-author could act on it without a meeting.
A bachelor's degree in a relevant field, or equivalent experience.

Nice To Haves

Built agent-based or LLM-driven automation for a task that previously required human judgment.
Worked on evaluation or benchmarking for computer vision, 3D reconstruction, or spatial systems.
Instrumented dashboards or scorecards that leadership used for release decisions.
Operated data provisioning or registry infrastructure across multiple environments and access models.

Responsibilities

Build the Evaluation Machine: Own the automation that executes Lab evaluations end to end, including environment setup, run orchestration, artifact capture, and result collection, to enable constant testing.
Make Results Comparable: Instrument scorecards to allow comparison of results across product versions, devices, and capture conditions, ensuring results have proper lineage.
Automate the Agent-Driven Layer: Build agent workflows to exercise priority customer use cases at realistic scale and across a spectrum of difficulty, acknowledging limitations where human judgment is still necessary.
Kill Manual Work Permanently: Convert one-off experiments into standing protocols that run on every relevant release, automating recurring inspections and routing genuinely non-automatable tasks to scalable human review.
Make Failures Actionable: Produce precise diagnostics that enable findings to be routed to the correct owners with reproducible cases and attached data.
Keep Data From Being the Bottleneck: Collaborate with the AI Data Manager to ensure every run is reproducible from a known dataset state without manual data handling.