Senior QA Automation Engineer

Sustainment•Austin, TX

About The Position

Sustainment builds B2B SaaS for manufacturing supply chains, serving both commercial customers and federal/defense programs. As we move to a daily release cadence, we need a senior, hands-on QA automation engineer to lead the charge toward a fully automated playwright test suite while keeping our releases fast and trustworthy. This is a player-coach role - roughly 80% hands-on, 20% leadership. You'll spend most of your time testing and building automation, with a light leadership layer: mentoring a small QA team and owning our QA process. It is a testing-and-automation role first, and a leadership role second.

Requirements

Senior-level experience in QA and test automation (6+ years).
Deep hands-on skill with modern test-automation frameworks (Playwright preferred, Cypress, or similar) and API testing.
Experience integrating automated tests into CI/CD pipelines (GitLab a plus).
A track record of building an automated suite up from partial or manual coverage.
Genuinely comfortable staying hands-on. You want to test and build, not step out of the tools.
Some experience mentoring engineers or owning QA processes for a small team.
Comfort with Jira for test and defect management.
Candidates must be eligible to obtain and maintain a U.S. Secret or Top Secret security clearance.
Applicants must be authorized to work for ANY employer in the U.S. We are unable to sponsor or take over sponsorship of an employment Visa at this time.

Nice To Haves

Experience with AI-assisted testing tools and workflows.
Delivery into regulated/federal environments (FedRAMP, CUI, DoD); we ship into government spaces.
B2B SaaS and/or manufacturing / supply-chain domain experience.
Exposure to data or analytics testing.
Experience testing agentic / multi-agent systems and prompt-driven workflows.
AI-assisted test generation - using LLMs or agents to write and maintain tests (exactly where we're taking QA).
ML / data-quality testing - model-output validation, extraction/classification accuracy metrics.
Testing and evaluating GenAI / LLM-powered features; building evaluation harnesses, ground-truth datasets, and scoring for non-deterministic model outputs (accuracy, precision/recall, LLM-as-judge).

Responsibilities

Drive the shift from mostly-manual testing to a comprehensive, fully automated playwright test suite.
Build, maintain, and scale automated tests (UI and API) and integrate them into our CI/CD pipeline so verification keeps pace with a daily release cadence.
Contribute to manual and exploratory testing whenever it's needed, especially on releases and wherever automated coverage doesn't exist yet.
Stand up QA quality instrumentation (e.g., escaped-defect tracking) so we can measure the quality of what we ship, not just the speed.
Champion AI-assisted testing as we scale toward AI-native development.
Mentor two QA engineers and raise the team's automation capability.
Own and standardize QA process and standards; test plans, entry/exit criteria, a clear bounce-back vs. hold guideline, and QA's definition of done.
Act as the QA operational lead and a trusted second to the Director of Engineering, so day-to-day QA runs without needing to route everything through leadership.