Senior AI Engineer, Agentic Evaluation & V&V

Slingshot Aerospace

17d • Remote

About The Position

As a Senior AI Engineer focused on Agentic Evaluation and Verification and Validation (V&V), you will join the AI and Data Science team within Slingshot’s Research and Development organization. You will contribute to advancing how intelligent systems are evaluated and validated for mission-critical space operations. This role focuses on building and scaling evaluation frameworks, benchmarks, and simulation-backed validation systems for agentic AI systems, including multi-step, tool-using, and autonomous decision-making workflows powered by LLMs and reinforcement learning. Your work will directly support the development of reliable and trustworthy autonomous mission planning systems. You will partner closely with AI researchers and domain experts to translate real-world mission concepts into structured, testable evaluation systems.

Requirements

• 6+ years of experience in software engineering, machine learning engineering, applied AI, or equivalent experience
• Strong Python engineering skills with experience building SDKs, libraries, or evaluation tooling
• Experience designing evaluation frameworks, benchmarks, metrics, or test harnesses for AI/ML systems
• Ability to analyze system behavior, identify failure modes, and evaluate performance in complex autonomous or semi-autonomous systems
• Familiarity with modern agent frameworks, orchestration patterns, or protocol-based integrations
• Experience working in cross-functional, multidisciplinary teams
• Strong written and verbal communication skills
• Bachelor’s degree in a relevant science or engineering field, or equivalent experience
• Must be a U.S. citizen and eligible to obtain and maintain a government security clearance

Nice To Haves

• Experience in autonomous systems such as self-driving or ADAS, including perception, planning, simulation, or safety validation
• Experience developing or evaluating agentic AI systems, including multi-step, tool-using, or autonomous workflows (e.g., LLM-based agents, planning agents, or reinforcement learning approaches)
• Experience with reinforcement learning systems and simulation-based evaluation
• Familiarity with benchmark design, experiment tracking, and trace-based evaluation workflows
• Experience with orchestration frameworks such as LangGraph or similar tools
• Knowledge of astrodynamics, orbital mechanics, or spacecraft mission planning
• Experience translating mission or operational concepts into measurable evaluation scenarios
• Familiarity with physics-based simulation, trajectory analysis, or space-domain modeling
• Experience with observability and experiment tooling such as MLflow, Opik, or similar platforms
• Experience transitioning advanced research systems into production environments

Responsibilities

• Extend and maintain Slingshot’s V&V SDK and evaluation framework for simulation-backed validation of agentic AI systems
• Design and implement agent-level and end-to-end evaluations, including benchmark scenarios, scoring logic, and experiment harnesses
• Build benchmark scenarios and tooling that measure planning, reasoning, and operational performance for autonomous mission planning systems
• Translate astrodynamics and mission-domain concepts into executable evaluation scenarios and simulation configurations
• Develop reusable SDK interfaces, adapters, and evaluation utilities that connect V&V systems, TALOS benchmarks, and agent workflows
• Define and apply metrics for capability evaluation, failure analysis, regression detection, and comparative benchmarking
• Partner with cross-functional teams to identify evaluation needs and contribute to improving coverage of critical capabilities
• Contribute to best practices for evaluating complex, autonomous AI systems
• Uphold strong engineering standards through testing, documentation, reproducibility, and maintainable system design

Benefits

• Equity, Diversity & Inclusion are key to our success. We are an Equal Opportunity Employer and our employees are people with different strengths, experiences, and backgrounds, who share a passion for creating a safer, more connected world. Diversity not only includes race and gender identity, but also national origin, citizenship, sex, color, veteran status, disability, genetic information, or any other protected characteristic that is part of one’s identity. All of our employees’ points of view are key to our success, and we embrace individuality.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Number of Employees: 11-50 employees