QA Engineer (AI Forms Platform)

Filevine

5d

About The Position

We are looking for a forward-thinking QA Engineer to lead the testing strategy for our new AI-first Forms Platform. In this role, you will move beyond traditional functional testing to build "evaluation harnesses" that measure the accuracy of our auto-mapping AI models against golden datasets. You will be responsible for minimizing defect escape rates and ensuring that complex government forms (for Immigration, Family Law, etc.) are rendered perfectly. You will use AI tools daily to generate test cases, write automation scripts, and analyze regression data to ensure our migration from legacy systems is seamless.

Requirements

• Bachelor’s degree in Computer Science, Engineering, or related field.
• 3+ years of experience in QA Automation, with proficiency in tools like Cypress, Selenium, or Playwright.
• Coding Proficiency: Strong ability to write test scripts in JavaScript, TypeScript, or Python (Java/C++ proficiency also accepted).
• AI/Data Testing: Experience testing non-deterministic systems (AI/ML models) or complex data migration projects is highly preferred.
• Agile/Scrum: 1 to 3 years experience working in agile environments with frequent release cycles.
• Problem Solving: Demonstrated ability to use AI tools to solve coding roadblocks and optimize testing workflows.
• Communication: Excellent verbal and written communication skills to articulate quality risks to the product team.

Nice To Haves

• AI/Data Testing: Experience testing non-deterministic systems (AI/ML models) or complex data migration projects is highly preferred.

Responsibilities

• AI Model Validation: Design and maintain evaluation harnesses (evals) to measure the accuracy of AI auto-mapping models, ensuring they meet strict quality thresholds before release.
• Test Automation: Build robust, automated test suites (end-to-end and integration) that run within our CI/CD pipelines to catch regressions immediately.
• Golden Datasets: Maintain and expand "golden datasets" of perfectly mapped forms to serve as the ground truth for regression testing.
• Defect Prevention: Collaborate with developers to identify "model quality variance" and implement gates that prevent low-confidence predictions from reaching production.
• Migration Quality: Verify data integrity during the migration of pilot customers from the legacy platform to the new architecture.
• Efficiency: Leverage LLMs and AI coding assistants to rapidly write test scripts and generate edge-case scenarios for complex legal forms.

Benefits

• Medical, Dental, & Vision Insurance (for full-time employees)
• Competitive & Fair Pay
• Maternity & paternity leave (for full-time employees)
• Short & long-term disability
• Opportunity to learn from a dedicated leadership team
• Centrally located open office building in Sugar House (onsite employees)
• Top-of-the-line company swag

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Number of Employees: 501-1,000 employees