Staff Research Scientist

Ironsite AI•San Francisco, CA

9h•$250,000 - $350,000•Onsite

About The Position

Ironsite is a spatial intelligence construction technology company building the intelligence layer for the physical world. We are accelerating the speed, efficiency, and predictability of construction, especially for complex, mission-critical infrastructure projects including data centers, LNG facilities, sports stadiums, hospitals, and other large-scale developments, by training AI models on egocentric construction footage and labor productivity data. Ironsite is the productivity system built for the people who build America. We design our own wearable hardware, deploy it alongside craft workers, and transform a shift's footage into a next-morning report. We label the data overnight and deliver actionable insights to superintendents by 5 AM every day. We are built with a pro-worker philosophy at our core. We believe technology should empower the workforce, not replace it, and we give craft workers and project leaders real visibility into what is happening on-site, so the reality of each day is finally available to the people running the project. By capturing and labeling the world's largest dataset of first-person video from active construction sites, we are building a general-purpose Vision-Language Model with the spatial intelligence to understand the most dynamic and complex physical environment in the world: the construction site. Our mission is to create general models that deeply understand dynamic, unstructured environments to supercharge human productivity and eventually enable robots to fill critical labor gaps. Our culture is one of radical transparency, intellectual honesty, low ego, rapid experimentation, and curiosity. We are relentlessly focused on the long-term goal of building truly generalizable intelligence that benefits everyone, starting with the construction workers responsible for our built world. Ironsite is deployed across several of the top ten largest active construction projects in the country. We currently collect ~1,000 hours of new video every day from 8 states and expect to reach ~10,000 hours per day by the end of the year, growing further from there, all while maintaining a worker opt-out rate below two percent. This is enabled by a workforce-first architecture that anonymizes devices, captures no audio, and never releases raw video. Ironsite is backed by leading investors (8VC, South Park Commons, Saga Ventures) and prominent operators across technology and construction, including Eric Schmidt, Jeff Dean, Jeff Rothschild, Mark Leslie, Scott Wu, Eric Glyman, Karim Atiyeh, Russell Kaplan, and others, alongside 12 construction industry operators who have joined us as partners in building this. THE ROLE As a Principal Applied ML Researcher, you will report directly to the Chief Science Officer and own high-risk, high-impact research: training, benchmarking, and deploying state-of-the-art VLMs that can interpret the complexity of a real-world construction site, built on data no other lab has. Open problems you could own in your first year: Long-video understanding. Reasoning over multi-hour egocentric footage where the events that matter are sparse. This requires temporal grounding, long-context modeling, and memory beyond what current VLMs offer. Post-training for expert-level perception. Using SFT and RL (e.g., GRPO) to push VLM labeling of fine-grained construction activity to parity with expert human taggers, across every trade. Inference at fleet scale. We will soon collect ~10,000 hours of new video per day, expanding quickly from there. Making frontier-quality inference cheap enough to run on all of it, through distillation, quantization, and model routing, is partially unsolved. Evals that predict reality. Building benchmarks that track real field performance.

Requirements

6+ years of hands-on experience designing and training large-scale deep learning models, particularly transformer-based architectures.
A background in Computer Science, Machine Learning, AI, Robotics, or a related field.
Demonstrated experience with major deep learning frameworks (e.g., PyTorch, JAX).
Strongly proficient in Python, with a solid foundation in software engineering principles.
Experience working with and creating large-scale vision and/or language datasets.
Enjoy rapid iteration on immediate blockers in service of long-term research goals.

Nice To Haves

A track record of publications in top-tier AI/ML/CV conferences.
Deep expertise in fine-tuning and post-training large language or vision-language models (SFT, GRPO and other RL methods, parameter-efficient tuning such as LoRA).
Hands-on experience with the challenges of video data, such as temporal reasoning, long-context modeling, and efficient processing.
Experience optimizing inference, including quantization, distillation, sparsity, and efficient serving.
Familiarity with MLOps tools for scalable model training and deployment.
A strong interest in vision-language models and applying AI to real-world physical problems, including understanding the day-to-day lives of construction workers.

Responsibilities

Architect & Train Novel VLMs: Design, train, and iterate on general-purpose Vision-Language Models fine-tuned for spatial intelligence in the construction site. Model quality is the single most important output of this role.
Drive the Research Roadmap: Take a leading role in executing our research goals, from establishing baselines with state-of-the-art models to developing post-training recipes (SFT and RL), long-context architectures, and visual reasoning techniques.
Own the Construction Intelligence Benchmark: Build and expand our benchmark suite: video question answering, temporal reasoning, activity recognition, and site-level analytical reasoning.
Build Scalable Pipelines: Develop and own the model training and evaluation pipelines, ensuring we can rapidly experiment, measure performance, and deploy models into production.
Optimize Inference at Scale: Apply distillation, quantization, and model routing so state-of-the-art understanding runs affordably across thousands of hours of daily footage, working with the hardware and data teams on system design.