Staff AI Test Automation Engineer

Gravie

11d•$155,000 - $206,500

About The Position

Gravie is seeking a Staff AI Test Automation Engineer to take hands-on ownership of their AI-driven test automation systems across product portals. The role involves continuing the migration from legacy tooling to the Playwright framework, improving, scaling, and optimizing AI-driven test automation. The engineer will build frameworks and infrastructure that enable AI tools to generate, execute, and maintain automated tests at scale, working closely with QA, Engineering, and Product teams to increase automated regression coverage, accelerate delivery, and reduce defects. A key aspect of this role is working with Agentic AI to generate, execute, and iterate on tests with minimal manual intervention.

Requirements

10+ years of experience in test automation engineering, SDET, or software engineering with strong testing practices.
Hands-on experience with Agentic AI tools (Claude Code, Cursor, GitHub Copilot, or similar) to generate and maintain code at scale.
Hands-on experience with Playwright or a comparable modern end-to-end testing framework.
Proficiency in TypeScript or JavaScript with the ability to write production-quality code.
Experience with CI/CD pipelines (GitLab preferred), including writing pipeline configuration, test stages, and merge request workflows.
Solid understanding of REST API testing and when to use API-level tests vs. UI-level tests.
Hands-on experience building test automation for claims systems, contract testing, integrations, and user flow journeys.
Experience migrating legacy test automation to a modern framework, including maintaining coverage during the migration.
Strong communication skills: able to explain technical trade-offs to non-technical stakeholders and write clear documentation.
A builder mindset. You think in terms of systems, feedback loops, and automation leverage, not individual test cases.
Comfort working in ambiguity. Our AI-driven approach is new, and the right person will help define what good looks like.

Nice To Haves

Experience with Groovy, Spock, or Geb (the legacy framework you will be migrating from).
Experience with contract testing (Pact or equivalent).
Background in health insurance, benefits administration, or regulated industries.
Experience building or contributing to internal developer tooling or productivity platforms.

Responsibilities

Continue building and hands-on improving AI-driven test automation workflows using tools like Claude Code, where AI agents generate, execute, and iterate on Playwright tests with minimal human intervention.
Continue evolving the existing Playwright test framework, refining patterns, page-object models, fixtures, and conventions to make them more efficient, scalable, and better optimized for AI-assisted test generation.
Build and improve tight feedback loops between local test execution, CI pipeline results, and merge request creation, so AI agents can run tests, interpret failures, fix issues, and resubmit with minimal manual intervention.
Drive the ongoing migration from Geb/Spock to Playwright to completion, using AI agents to accelerate conversion at scale while maintaining coverage continuity.
Integrate and continuously optimize Playwright test execution in GitLab CI/CD pipelines with automated quality gates on every merge request and deployment. Optimize for fast feedback through parallelization and caching strategies.
Instrument tests with reporting on pass/fail trends, flakiness rates, and coverage gaps. Use this data to prioritize AI-driven test generation toward the highest-value uncovered areas.
Identify scenarios where end-to-end Playwright tests are not the right tool and build API-layer automation or contract tests for microservice boundaries instead.
Build hands-on automation for claims systems, contract testing, system integrations, and end-to-end user flow journeys.
Continuously improve the AI automation workflow by refining prompts, context structures, and validation patterns to increase the percentage of tests that AI generates correctly on the first pass.