Software Engineer, AI Infrastructure - LVM Inference & Evaluation

Ambient.ai • Redwood City, CA

2d • $168,000 - $205,000 • Hybrid

About The Position

Ambient.ai is a leader in Agentic Physical Security, utilizing its Ambient Pulsar reasoning Vision-Language Model to enhance security systems. The platform integrates with existing security cameras and access control to improve monitoring, response, and investigations, resulting in fewer false alarms and faster response times. The company has experienced significant growth and is recognized on the Forbes best startup employers list. Founded in 2017 and backed by prominent investors, Ambient.ai aims to prevent security incidents. This role reports to Raghu Nallamothu and involves designing, building, and optimizing AI infrastructure for real-time intelligence, focusing on systems for inference, evaluation, and continuous model improvement across various AI models and large video datasets. The ideal candidate will have a strong background in infrastructure engineering, production ML systems, LLM/LVM inference, evaluation, and inference optimization, and will collaborate with research scientists and product engineering teams.

Requirements

• 2+ years of industry experience building infrastructure, distributed systems, machine learning platforms, or production AI systems.
• BS/MS in Computer Science or a related technical field, or equivalent practical experience.
• Strong programming background, especially in Python, with solid software engineering fundamentals.
• Experience designing and building scalable machine learning infrastructure for training, inference, evaluation, and deployment.
• Hands-on experience running deep learning models in production, ideally including LLMs, LVMs, vision-language models, or multimodal models.
• Strong understanding of inference optimization techniques, including batching, caching, quantization, parallelism, memory optimization, GPU utilization, and latency reduction.
• Experience with model-serving frameworks or systems such as vLLM, Triton Inference Server or similar technologies.
• Experience building evaluation frameworks, test harnesses, benchmarks, regression tests, or model-quality measurement systems.
• Strong background in machine learning and deep learning; computer vision experience is a strong plus.
• Experience designing data engines or pipelines for collecting, managing, and curating training and evaluation data.
• Familiarity with integrating advanced AI systems such as LLMs, LVMs, RAG pipelines, embedding models, or multimodal models into production applications.
• Experience with cloud infrastructure, containers, orchestration, distributed systems, and GPU-based workloads.
• Strong collaboration and communication skills, with the ability to work effectively with research scientists, product teams, infrastructure teams, and stakeholders.
• Proactive problem-solving ability, a strong ownership mindset, and adaptability to incorporate new AI technologies and methodologies.

Nice To Haves

• Experience operating large-scale GPU infrastructure or distributed inference systems.
• Experience with CUDA, NCCL, PyTorch, TensorRT, ONNX, or similar ML systems technologies.
• Experience with video understanding, real-time computer vision, multimodal AI, or physical-world AI systems.
• Experience with model compression, speculative decoding, distillation, pruning, or low-latency serving techniques.
• Experience with prompt evaluation, model regression testing, human-in-the-loop evaluation, or automated quality gates.
• Familiarity with retrieval-augmented generation, vector databases, embedding models, re-rankers, or search infrastructure.
• Experience building internal ML platforms or tools used by researchers and applied ML teams.

Responsibilities

• Design, build, and maintain cutting-edge AI infrastructure for real-time computer vision, LLM, LVM, and multimodal inference workloads.
• Build scalable systems for running state-of-the-art models across large volumes of video and sensor data.
• Optimize inference performance across latency, throughput, GPU utilization, reliability, and cost.
• Develop robust evaluation harnesses and benchmarking systems to measure model quality, system performance, regressions, and production readiness.
• Build infrastructure for continuous model evaluation, experimentation, and deployment.
• Partner with research scientists to productionize the latest advances in computer vision, LLMs, LVMs, RAG, and multimodal AI.
• Improve model-serving architecture, including batching, caching, routing, quantization, model parallelism, and hardware utilization.
• Develop data engines and feedback loops for collecting training data, evaluating model behavior, and continuously improving AI performance.
• Create reliable observability, monitoring, and debugging tools for production AI systems.
• Help define best practices for deploying, evaluating, and operating AI systems in real-world enterprise environments.

Benefits

• Stock options
• Comprehensive health + welfare package (Medical, Dental, Vision, Life, EAP, Legal Services, 401k plan)
• Flexible time off
• Winter Break (time off between Christmas and New Year’s for most roles, depending on customer demand)
• The latest tech and awesome swag
• Opportunities to connect with co-workers

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: Associate degree