QA Engineer

Hatz AI • New York, NY

21d • $140,000 - $165,000 • Hybrid

About The Position

We're looking for a QA Engineer who operates less like a gatekeeper and more like the most demanding customer on the engineering team. Based in New York City's vibrant Flatiron District, Hatz AI is building the AI-native platform for Managed Service Providers (MSPs) and the small businesses they serve. Because you sit closest to the customer and understand our critical workflows better than anyone on the team, you'll own product quality strategy outright—deciding where risk-based coverage matters most, and owning end-to-end testing with full autonomy over the frameworks and tools you use to get there.
This isn't a role for someone who just executes a test plan handed to them. Critical workflow coverage is your version of code coverage—it's the metric you're accountable for, and you decide how to hit it. You'll partner closely with engineering: reviewing and getting hands-on with the integration tests they own, helping testability get considered before code is written, and using AI to build the rules, skills, and frameworks that scale your judgment across the team rather than routing everything through you.

Requirements

• Experience owning quality strategy and risk-based test prioritization, not just executing a test plan someone else scoped for you.
• Proven ownership of end-to-end test suites, including the judgment to choose and evolve your own frameworks and tooling. (We care more about the coverage decisions you've made than years alone.)
• Deep enough product fluency to independently identify critical customer workflows and where risk actually lives.
• Comfort reviewing and getting hands-on with integration-level tests that engineers write.
• Fluency using AI tools in your daily workflow—including building reusable rules, skills, or frameworks that scale your judgment across the team.
• Experience with test automation frameworks (e.g., Playwright, Cypress, or Selenium).
• Experience testing RESTful APIs and asynchronous workflows; comfort writing and reading SQL to verify data integrity.
• Experience with version control systems, particularly Git.
• Excellent communication—this is a highly visible role that represents the customer inside engineering.
• Ability to work independently and as part of a team in a fast-paced, changing environment.

Nice To Haves

• Experience evaluating LLM or generative AI outputs—building eval datasets, scoring rubrics, or regression harnesses to catch hallucinations and accuracy drift.
• Familiarity with eval tooling (e.g., Braintrust, LangSmith) or experience building an in-house eval harness.
• Experience with performance and load testing.
• Exploratory testing experience.
• A/B testing experience.
• Experience with beta programs or MSP/channel customer programs.
• Support engineering experience.
• Ability to contribute bug fixes directly using AI-assisted coding (TypeScript/JavaScript or Python).
• Understanding of CI/CD pipelines and how automated tests fit into deployment gates.
• Knowledge of PostgreSQL/SQL and Row Level Security.
• Experience validating zero-downtime schema migrations on live systems.

Responsibilities

• Own product quality strategy: help decide, based on critical customer workflows, where risk-based test coverage matters most.
• Be responsible for end-to-end testing, with discretion to choose and evolve your own test frameworks and tooling. Critical workflow coverage is your version of code coverage.
• Build AI-assisted rules, skills, and frameworks that let testing judgment scale across the team without requiring manual testing input on every ticket.
• Review and get hands-on with the integration tests engineers write. Unit and component-level testing stays owned by engineering; you're encouraged to get involved wherever it strengthens coverage.
• File clear, reproducible bug reports, and grow into contributing fixes directly—AI-assisted coding makes this realistic once you're covering the core of the role.
• Define and track the quality metrics (critical workflow coverage, defect escape rate, regression trends) that tell us whether the strategy is working.
• Build and run evals for our AI features—golden datasets, regression suites, and scoring rubrics that catch hallucinations, accuracy drift, and prompt or model regressions before they reach customers.
• Communicate clearly with teammates and customers, translating quality tradeoffs and technical issues into plain language.

Benefits

• Competitive salary.
• Flexible work hours in a hybrid work environment.
• Comprehensive health, dental, and vision insurance; life and disability insurance, 401(k) with match, unlimited vacation and equity.
• Opportunities for professional development and career growth.
• A collaborative and inclusive work environment.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed