Senior QA Engineer

Absorb•Calgary, AB

1d•Remote

About The Position

Quality Assurance at Absorb is redefining what it means to build quality software. We develop top-down thinkers who apply their technical skills to solve not only today's defects, but tomorrow's challenges through AI-powered, Spec-Driven Development. We have moved beyond Test-First Development to defining requirements with the precision demanded by AI-assisted software engineering, identifying gaps before a single line of code is written. Our team partners daily with Engineering, Product Management, and Customer Experience to solve complex technical problems while advancing the use of AI, machine learning, and emerging technologies throughout the software development lifecycle. This role supports the Development team through manual and automated QA testing of enhanced software through AI-augmented testing practices, including agentic coding tools, evaluations, and spec-driven development.

Requirements

Proven experience in manual and automated QA testing of web applications.
Proficiency in TypeScript/JavaScript with hands-on experience in Playwright or a comparable automation framework (Cypress, Selenium, WebdriverIO).
Practical experience using AI coding assistants and agentic tools (e.g., Claude Code, GitHub Copilot) for writing, maintaining, or reviewing test automation.
Working understanding of spec-driven development methodologies (e.g., AI-DLC / AI-Driven Development Lifecycle) and how structured specifications drive AI-assisted planning, execution, and validation.
Ability to build and maintain custom AI skills, prompts, or agent workflows to streamline QA processes.
Understanding of non-deterministic testing: recognizing that AI/LLM outputs vary between runs and require different validation strategies than deterministic UI/API testing.
Experience designing evaluation ("eval") datasets and metrics (accuracy, relevance, hallucination, toxicity, latency) for AI-powered features.
Familiarity with AI observability and evaluation-monitoring platforms such as Arize (AX/Phoenix), LangSmith, or Braintrust for tracking model/agent quality in production.
Solid grounding in QA methodologies, the SDLC, and test planning/estimation.
Strong analytical, written, and verbal communication skills, with the ability to mentor junior QA Analysts.
Bachelor's Degree in a Technical field and 5+ years of experience, or equivalent job experience.

Responsibilities

Review requirements specifications and technical design documents to provide timely and meaningful feedback.
Create detailed, comprehensive, and well-structured test plans and test cases and analyze and update existing test cases.
Lead and assist more junior QA Analysts.
Estimate, prioritize, plan and coordinate testing activities.
Design, develop and execute automation scripts using open-source tools, TypeScript, Playwright, or similar automation frameworks (e.g., Cypress, Selenium).
Use AI coding assistants and agentic tools, such as Claude Code, to accelerate test authoring, maintain automation scripts, and triage defects.
Build, customize, and maintain reusable AI skills and agentic workflows that streamline repeatable QA tasks (test generation, regression checks, defect documentation).
Apply spec-driven development practices, for example the AI-Driven Development Lifecycle (AI-DLC), to translate requirements into structured specifications that guide AI-assisted test generation and validation.
Design and run evaluations ("evals") for AI-powered features to measure accuracy, relevance, hallucination rate, and other quality metrics against defined test datasets.
Apply non-deterministic testing approaches for AI/LLM-based features, using statistical thresholds, sampling, and repeated-run analysis rather than fixed pass/fail assertions.
Configure and monitor AI observability and evaluation tooling (e.g., Arize AX/Phoenix, LangSmith, or similar) to track model/agent performance, detect drift, and flag regressions in production.
Identify, record, document, track, and close bugs.
Perform thorough regression testing when bugs are resolved.
Actively transfer knowledge to other QA Team members via documentation and QA presentations, including AI-assisted testing practices and tooling.
Ad hoc duties as required.