Staff Software Engineer in Test (Modeling and Simulation)

Code Metal • San Francisco, CA

2d • $145,000 - $185,000

About The Position

Code Metal is seeking a founding Staff Quality Engineer (SDET) for Modeling & Simulation within its centralized Quality Engineering function. The Modeling & Simulation team builds a wargaming and decision-support platform that helps the U.S. Department of War evaluate mission- and campaign-level force-design decisions. The platform combines backend APIs, asynchronous workers, geospatial and sensing models, authenticated multi-team play, external simulation integrations, and emerging AI-assisted capabilities.

The role will design and build the platform’s automated validation system, connecting existing unit and integration coverage into reproducible quality evidence across services, models, and end-to-end workflows. The engineer will determine which behaviors need deterministic verification, statistical or model-based evaluation, or human review, and how limitations and residual risk are communicated as release evidence. This is a hands-on software engineering role focused on foundational test infrastructure, critical tests, and helping engineers build quality into their work.

The position typically calls for 8+ years of relevant software engineering, test-infrastructure, or quality-engineering experience, with Staff-level scope and impact. Equivalent depth of experience is valued over a rigid year count. The successful candidate has translated quality strategy into test architecture for a product area, led work through design documents and influence rather than people management, and personally shipped infrastructure and tests.

Requirements

• Significant experience as an SDET, test-infrastructure engineer, quality architect, or software engineer responsible for automated validation of production systems.
• Strong production programming skills in Python.
• Deep experience building Python test infrastructure using pytest or comparable frameworks, including asynchronous testing, fixtures and factories, parallel execution, and CI integration.
• Demonstrated experience designing risk-based test strategies across multiple layers of a distributed system.
• Experience validating complex systems with partially nondeterministic behavior, such as simulation, statistical models, ML, or AI-assisted workflows, and sound judgment about deterministic checks, reference evidence, statistical evaluation, and human oversight.
• Hands-on experience testing Docker-based or comparable containerized, multi-service applications, including APIs, databases, messaging, authentication, authorization, and data isolation.
• Experience building and improving CI workflows with GitHub Actions or a comparable platform, including test isolation, failure diagnosis, and flaky-test control.
• Ability to set technical direction, write design documents, mentor engineers, and lead multi-team initiatives through influence.
• Typically 8+ years of relevant software engineering, test infrastructure, or quality engineering experience, with demonstrated Staff-level scope and impact; equivalent depth of experience is valued over a rigid year count.
• Eligible to obtain and maintain a U.S. Secret security clearance.

Nice To Haves

• Experience validating simulation, geospatial, event-driven, or other stateful systems, including reproducible deterministic and probabilistic behavior.
• Familiarity with testing or debugging C++ simulation or systems code.
• Experience testing TypeScript and React applications with Playwright or equivalent tools.
• Experience with performance testing, generated test data, and API or schema validation.
• Experience testing cloud-based, containerized systems using logs, metrics, traces, and retained test artifacts.
• Familiarity with constructive simulation, distributed simulation standards, or mission-analysis software.
• Hands-on experience evaluating AI-assisted or agentic systems, including tool-calling or multi-step workflows, and understanding risks such as nondeterminism, unsupported outputs, prompt or context manipulation, unsafe tool use, hidden state, and silent failure.
• Experience using AI coding assistants or agents throughout the software development lifecycle and defining quality controls for AI-assisted changes, such as scoped workflows, independently verifiable tests, human code review, CI checks, and risk-based release evidence.
• Experience in a regulated, high-assurance, or mission-critical domain.

Responsibilities

• Translate Quality Engineering strategy into a risk-based validation approach across unit, integration, contract, end-to-end, performance, resilience, and security-critical workflows.
• Build reliable test infrastructure, including deterministic fixtures, isolated test data, service and event-contract tests, integration harnesses, versioned reference scenarios, deterministic replay, and focused browser automation for critical user journeys.
• Improve CI feedback through test partitioning, parallel execution, caching, failure attribution, and flaky-test control in partnership with shared platform owners.
• Write behavior-focused tests for high-risk planning, sensing, simulation, authorization, state-transition, and recovery workflows.
• Partner with simulation and model owners to turn intended behavior, assumptions, and known limitations into reusable reference scenarios, regression baselines, characterization or differential tests, and reproducible quality evidence.
• Define validation and release criteria for simulation and emerging AI-assisted workflows, combining deterministic assertions, contracts, reference checks, statistical or model-based evaluation, failure-recovery testing, and appropriate human oversight.
• Establish repeatable performance and reliability validation with measurable objectives and clear release evidence.
• Participate in design and code reviews, identify testability risks early, and mentor engineers in testing complex simulation, distributed-system, and AI-assisted changes without becoming an approval bottleneck.
• Turn lessons from Modeling & Simulation into reusable quality bars, reporting practices, testing standards, and validation infrastructure for the centralized Quality Engineering function.
• Set technical direction, write design documents, and lead multi-team initiatives through influence.

Benefits

• Health care plan with 100% premium coverage, including medical, dental, and vision.
• 401k with 5% matching.
• Paid Time Off (uncapped vacation, plus sick and public holidays).
• Flexible hybrid or remote work arrangement.
• Relocation assistance for qualifying employees.
• Pay depends on experience, and the company strives to be at the upper end of the salary range.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed