Senior Software Quality Engineer
About The Position
Zipliens is seeking a senior quality engineer to lead quality for applications supporting lien resolution in the legal industry. The role sets quality standards for a growing engineering team and builds testing and evaluation practices for complex, data-heavy products. Quality engineering is embedded in engineering teams and combines exploratory testing with automation. The senior engineer will shape team standards for coverage, merge and deploy gates, frameworks, risk analysis, and how new quality engineers learn to assess risk. The position includes establishing evaluation practices for product features that use LLMs to extract structured information from medical records and lien correspondence. It also involves using and reviewing agentic coding tools and internal automation harnesses, assessing new tools, mentoring quality engineers, and advising on application testability early in design. The role works closely with product and engineering partners and owns quality strategy and sign-off for medium to large features, including cross-team work.
Requirements
- At least 5 years on a QA or software development team, with experience across a range of products and test approaches.
- Expert-level skill in at least one of exploratory testing and test design, test automation and infrastructure, or AI/LLM evaluation, with solid proficiency in the other two.
- Experience setting test standards and choosing frameworks for a team and gaining team adoption.
- Significant experience building and owning automation in a modern framework such as Playwright, Cypress, or pytest, including designing its CI/CD gating strategy.
- Comfort working in code; experience reviewing test code, reading changes for risk, and contributing to shared test infrastructure. The backend uses Python/Django and the frontend uses React.
- Strong SQL and PostgreSQL skills, including verifying the data produced by workflows as well as their UI output.
- Hands-on experience evaluating LLM or ML features in production, including golden datasets, acceptance thresholds, calibration of LLM judges against human labels, and post-release quality monitoring.
- Experience using agentic coding tools for test work and understanding where their output requires human review.
- Experience mentoring quality engineers or developers and raising standards on a growing team.
- Excellent written and verbal communication skills for defining release criteria, explaining risk to non-technical stakeholders, and advocating for quality investment to engineering leadership.
- Authorization to work in the United States without employer-sponsored visa support now or in the future.
Nice To Haves
- Experience testing document processing or extraction from unstructured data, including defining acceptance thresholds.
- Experience building or owning test infrastructure shared across multiple teams.
- Experience testing third-party integrations, particularly when documentation is incomplete and contract tests, sandboxes, or recorded fixtures are needed.
- Familiarity with AWS and Kubernetes, including collaboration on test environments and failure investigation.
- Experience handling sensitive data such as PHI or working under HIPAA-style requirements.
Responsibilities
- Lead quality for medium to large features, including cross-team features, and own risk analysis, test strategy, release criteria, and final sign-off.
- Set, follow, and advocate for team test standards and frameworks, including coverage expectations and what a passing build guarantees.
- Own the automation and CI/CD gating strategy, including which suites run at each stage, merge and deploy gates, and controls for suite runtime and flakiness.
- Own evaluation methodology for LLM features, including sampling and significance, rubric design, regression gates for prompt and model changes, production monitoring, drift alerts, and rerunning golden sets after vendor model updates.
- Set standards for validating AI-generated output in the product, tests, and tooling generated with AI.
- Collaborate with the Product Owner to define acceptance and release criteria and serve as the primary QA contact for cross-functional planning.
- Evaluate and introduce new tools and AI capabilities for the team.
- Mentor quality engineers and coach engineers on exploratory techniques and risk-based testing.
- Advise on application testability early in design, while changes are less costly.
Benefits
- Private health care plan covering medical, dental, and vision.
- Company HSA contributions for HDHP participants.
- Flexible Spending Accounts for health and dependent care.
- Company-paid short-term disability coverage.
- Voluntary long-term disability, life, AD&D, and supplemental coverage options.
- 401(k) plan with company match.
- Paid time off, including vacation, sick time, and 10 paid holidays.
- Parental leave.
- Opportunity to earn a discretionary quarterly bonus.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
