Senior QA Engineer (AI Systems & Automation)

Essential Software-posted 6 days ago
$107,000 - $138,000/Yr
Full-time • Mid Level
Hybrid • Rockville, MD
Resume
Match Score
Upload and Match ResumeTrack Jobs with Teal

Essential Software Inc. is a trusted partner to federal agencies, including the National Cancer Institute (NCI), delivering secure, cloud-based platforms that support large-scale cancer data and biomedical research. As a Senior QA Engineer (AI Systems & Automation), you will lead quality strategy and test automation for critical data platforms and AI-powered experiences. You will ensure both traditional software and AI/agentic systems are reliable, explainable, and safe in a federal, mission-driven environment. You will: Own end-to-end quality for complex web, API, data, and AI/ML-powered features Design AI-aware test strategies and automation that leverage GenAI and agentic frameworks Mentor QA engineers and collaborate closely with cross-functional teams and government partners

  • Develop and maintain test plans, test cases, traceability, and test data for product and AI features
  • Execute manual and automated tests for web applications, RESTful APIs, data workflows, and AI/ML features
  • Own automated regression suites, release readiness criteria, and provide clear go / no-go quality signals
  • Participate in agile ceremonies, validate end-to-end functionality, and ensure user stories (including AI features) meet acceptance criteria
  • Manage the full defect lifecycle, including triage, prioritization, root cause analysis, and verification of fixes
  • Maintain QA documentation, runbooks, and quality dashboards
  • Design and execute test strategies for AI/LLM-powered capabilities, including virtual agents, chatbots, copilots, and RAG-based systems
  • Use LLM-powered tools (e.g., ChatGPT, Claude, Copilot) to accelerate test design, data generation, exploratory testing, and script authoring
  • Build and refine QA-focused AI agents that can: Scrape UI and verify DOM structures Validate data against backend or ground-truth sources Auto-generate and maintain test scripts Run self-correcting / autonomous test flows
  • Evaluate and integrate agentic frameworks (e.g., OpenAI Assistants API, AWS Bedrock Agents, LangGraph, MCP) into QA workflows
  • Define and monitor AI-specific quality metrics (accuracy vs. ground truth, hallucination and error rates, safety / policy adherence)
  • Ensure AI and virtual agent experiences are accurate, consistent, and high quality in a federal context
  • Plan and execute performance, load, and scalability testing (e.g., JMeter or equivalent)
  • Validate data integrity and transformation quality across complex biomedical data pipelines and AI-enhanced workflows
  • Partner with engineers and data scientists to ensure AI/ML models and integrations are testable, observable, and measurable post-deployment
  • Mentor QA team members in both traditional and AI-augmented QA practices
  • Collaborate with development, DevOps, product, UX, and data teams to improve testability, shift-left quality, and increase automated coverage
  • Integrate automation into CI/CD (e.g., GitHub Actions, Jenkins, Azure DevOps, GitLab CI), monitor test health and flakiness, and address coverage gaps
  • Communicate quality risks, trends, and mitigation plans to technical and non-technical stakeholders, including government partners
  • Bachelor’s degree in computer science, Information Technology, Engineering, or related field
  • 5+ years of software QA experience (manual and automation) in production environments
  • 2+ years providing technical or process leadership (e.g., lead QA, primary product QA owner, mentor, or manager)
  • Strong experience with UI automation tools (Selenium WebDriver, Playwright, or Cypress)
  • Experience testing RESTful APIs and microservices architectures
  • Hands-on experience integrating automated tests into CI/CD pipelines (GitHub Actions, Jenkins, Azure DevOps, or GitLab CI)
  • Professional proficiency in Python or JavaScript for test automation
  • Hands-on use of GenAI tools (e.g., ChatGPT, Claude, Copilot) for QA tasks such as test-case generation, data creation, and exploratory testing
  • Understanding of AI/agentic concepts: Tool-calling / function invocation Multi-step / chain-of-thought workflows Autonomous / self-healing test flows AI-driven data comparison and validation
  • Experience with performance / load testing (e.g., JMeter or equivalent)
  • Proficiency with Jira or similar issue tracking tools
  • Strong written and verbal communication skills, including the ability to explain AI-related quality risks to stakeholders
  • Ability to prioritize, multitask, and operate effectively in complex, mission-driven environments
  • AWS Cloud Practitioner certification
  • Experience with modern automation stacks (Playwright or Cypress) and API testing tools (Postman, REST-assured, pytest, or similar)
  • Experience testing AI/ML-powered features (LLM applications, RAG systems, agents, recommendation engines, or chatbots)
  • Experience with one or more: LangChain or LangGraph AWS Bedrock Agents or OpenAI Assistants API MCP (Multi-Context Protocol) or similar orchestration frameworks
  • Experience designing or testing internal QA copilots or automation bots for test authoring or execution
  • Familiarity with test management tools (e.g., TestRail, Zephyr)
  • Knowledge of accessibility standards (WCAG) and basic security testing practices
  • Prior QA experience in healthcare, life sciences, biomedical informatics, or other regulated data environments
  • ISTQB or similar certification
  • Competitive benefits
  • Professional development opportunities
  • Collaborative, supportive culture
Track Jobs with Teal

Job Search Resources

•
AI Resume Builder
•
AI QA Engineer Resume Examples
•
Cover Letter Examples

Tools

Career Hubs

Guides

Company

© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service