Senior Test Automation Engineer

Rad AI•,

3d•$150,000 - $185,000

About The Position

Rad AI is hiring a Senior Test Automation Engineer to own test automation strategy for a product domain and help establish a consistent quality operating model across its healthcare AI products. The role partners with product and engineering teams to ensure testing reflects radiologists’ real workflows and to make release risk explicit. Engineering owns product quality, while Quality owns the quality system. The engineer will define what to test and at what level, design scalable automation frameworks and utilities, and help developers build and maintain their own tests. The role covers release risk assessments and planning, customer-matched workflow validation, trustworthy and efficient test suites, non-functional quality, responsible evaluation of AI systems, defect prevention, quality metrics, and mentoring across teams. This position supports products used in clinical radiology workflows, where regressions can affect reports and patient care.

Requirements

At least 6 years of software engineering or test automation experience, including at least 3 years designing and owning automation frameworks.
Strong programming skills in TypeScript and/or Python.
Deep experience with end-to-end testing frameworks such as Playwright, Selenium, or Cypress.
API testing experience across REST and GraphQL services, including how tests interact with data and downstream dependencies.
Experience building tests into CI/CD pipelines such as GitHub Actions or CircleCI, including PR-level gating, parallelization, and reporting.
A data-backed track record of reducing test flakiness and suite runtime.
Experience with risk-based test strategy for integration-heavy systems operating across many customer configurations.
Strong debugging skills across services, data, and environments, with working knowledge of SQL, logs, and observability tools.
A shift-left mindset and experience helping developers own their tests.
Hands-on use of AI tools to write, maintain, and triage tests, with judgment to review their output.
Detail-oriented and collaborative, with clear communication skills, ability to influence without authority, and experience mentoring engineers.

Nice To Haves

Experience in healthcare or another regulated environment, such as HIPAA, PHI, or SOC 2, ideally at a startup.
Familiarity with radiology systems and standards, including PACS, RIS, dictation platforms such as PowerScribe or Fluency, HL7/FHIR, FHIRcast, or DICOM.
Experience automating Windows desktop or Electron applications, for example with WinAppDriver or UI Automation.
Experience testing ML, AI, or speech systems with non-deterministic outputs.
Experience owning performance or load testing, or integrating security checks into pipelines.
Experience with AWS or Google Cloud and provisioning test environments using tools such as Docker or Terraform.
Experience with a test case management system such as Testmo, or building quality dashboards in tools such as Grafana.
Experience with an object-oriented language such as C# or Java.

Responsibilities

Own test automation strategy for a product domain, determining what to test and at which level while balancing delivery and test-suite health.
Build scalable, maintainable automation frameworks, fixtures, and utilities for engineers to use; partner with engineers to improve testability and apply AI-assisted test generation where useful.
Conduct pre-release risk assessments, build test matrices across integrations, environments, and customer workflows, and lead test planning for high-risk releases and hotfixes.
Help define go/no-go release signals for the product domain.
Translate radiologist personas and real workflows into test cases; validate in customer-matched environments, including Windows and Electron workstations, dictation systems, and PACS/RIS integrations; use exploratory testing where automation is insufficient.
Improve test signal quality by reducing flaky and low-signal tests, triaging failures, reducing suite execution time, and improving test data and environment stability with platform teams.
Lead or co-lead performance and load testing, security checks in pipelines, and accessibility and compliance coverage, including HIPAA-related auditability.
Design evaluations for non-deterministic AI outputs, including speech recognition and generated impressions, without relying on a single expected result.
Use incident reviews and customer-reported defects to add regression coverage and update risk profiles to prevent repeat failures.
Track escaped defects, test stability, and time to detect; organize suites in the test case management system so results are visible and comparable.
Mentor test automation engineers and software engineers, advise Engineering Managers on risk and release readiness, and contribute to hiring, onboarding, and the Quality community.