QA Automation Lead
Accuity
•Remote
About The Position
The Quality Assurance (QA) Automation Lead plays a critical role in ensuring the quality, reliability, and scalability of Accuity's AI-driven clinical documentation and revenue cycle solutions. This highly hands-on technical leadership role is responsible for designing and implementing comprehensive test automation frameworks across web applications, APIs, data pipelines, and AI/ML systems. Working closely with data scientists, AI engineers, product teams, and DevOps partners, the QA Automation Lead establishes modern quality engineering and MLOps practices to ensure systems and models are production-ready, compliant, and performant.
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related field required
- 7 to 10+ years of experience in software quality assurance and automation
- Proven experience designing and implementing automation frameworks across web, API, and data systems
- Strong programming skills in one or more languages, such as Python, C#, JavaScript, or TypeScript
- Experience with test automation tools such as Selenium, Playwright, Cypress, or PyTest
- Strong understanding of CI/CD pipelines, DevOps practices, and version control systems
- Experience with Agile methodologies and tools such as Jira
Nice To Haves
- Experience testing AI and ML systems, including model validation and data pipeline testing
- Experience working in Azure-based environments and integrating with Azure DevOps
- Familiarity with MLOps concepts and frameworks
- Experience in healthcare technology, including CDI, revenue cycle or coding workflows, and EHR systems and integrations
- Knowledge of regulatory and compliance requirements in healthcare systems
Responsibilities
- Define and implement enterprise-grade test automation strategies across AI/ML model evaluation, APIs and service layers, data pipelines, and web applications
- Design and build scalable, reusable automation frameworks that support continuous delivery and high test coverage
- Establish standardized approaches for functional, regression, performance, and data validation testing
- Drive adoption of modern testing methodologies, including shift-left testing and test-driven development practices
- Develop advanced testing frameworks for model validation, including accuracy, precision and recall, drift detection, and reliability monitoring
- Implement evaluation approaches for large language models and AI systems, including prompt testing, output validation, and hallucination detection
- Build data quality and feature validation pipelines using synthetic and production-like datasets to support robust model testing
- Ensure AI systems meet reliability, explainability, and safety standards appropriate for healthcare environments
- Partner with DevOps teams to integrate automated testing into CI/CD pipelines and release processes
- Contribute to the design and implementation of MLOps frameworks that support continuous integration and deployment of machine learning models
- Implement automated validation gates, monitoring, logging, and feedback loops for model promotion and production performance
- Ensure alignment between QA automation practices, DevOps workflows, and Azure-based infrastructure
- Establish quality metrics and KPIs, including test coverage, pass rates, defect leakage, and execution time, and build reporting visibility around performance
- Lead defect management processes, including triage, root cause analysis, and resolution tracking
- Collaborate with Product Owners and Scrum Masters to embed quality practices into Agile workflows
- Ensure test coverage aligns with business-critical workflows across clinical documentation and revenue cycle processes
- Serve as a technical leader and mentor within a small QA and engineering team
- Establish best practices for automation, code quality, and test design
- Conduct code reviews and provide guidance on automation frameworks, tools, and implementation approaches
- Influence engineering culture toward a quality-first mindset and continuous improvement
- Ensure testing frameworks support auditability, traceability, and regulatory compliance requirements
- Validate systems against healthcare-specific workflows, including CDI, coding accuracy, and EHR integrations
- Incorporate controls for data privacy, security, and model governance into testing and release practices
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Upload and Match Resume
What This Job Offers
Job Type
Full-time
Career Level
Senior
Number of Employees
1-10 employees
