Staff Quality Assurance Engineer
Pearson•Hoboken, NJ
•$130,000 - $150,000•Hybrid
About The Position
We are seeking a Senior Expert, Quality Engineer (QE) with deep expertise in quality engineering for AI-powered systems. In this role, you will own and lead end-to-end quality for your project while playing a key role in testing and evaluating complex, non-deterministic AI systems. You will help shape the quality strategy for the project and raise the overall quality bar for the teams you work with.
Requirements
- 8+ years of experience in Quality Engineering or Quality Assurance
- Proven hands-on experience testing AI/LLM-based systems, including agents, multi-agent workflows, and RAG pipelines
- Experience with AI evaluation approaches (e.g., RAGAS, LLM-as-judge, regression baselines) and non-deterministic testing strategies
- Strong automation skills using Playwright or a comparable framework
- Proficiency in JavaScript/TypeScript and/or Java
- Solid experience with API testing and data validation
- Demonstrated track record of leading quality end-to-end for complex projects, from strategy through execution
- Ability to define and drive QE technical direction within a project or product area
- Experience working in Agile delivery environments and contributing to fast-moving release cycles
Nice To Haves
- Hands-on experience with prompting, embeddings, or vector databases
- Cloud experience (AWS preferred)
- Exposure to AI-assisted development or testing tools (e.g., Cursor, Copilot)
- Experience with AI observability or monitoring tools for production LLM systems
Responsibilities
- Define quality strategies and own test planning from requirements through release, partnering closely with cross-functional teams
- Design and execute scalable functional, regression, and integration testing strategies
- Apply hands-on automation framework expertise across UI, API, and backend layers to support major product initiatives
- Test AI-driven functionality, including LLM agents, multi-agent workflows, and RAG pipelines
- Validate AI outputs for accuracy, relevance, consistency, and edge cases across non-deterministic systems
- Own evaluation frameworks (e.g., RAGAS), test datasets, and regression baselines for AI features
- Stay current with the evolving landscape of LLM evaluation, agent testing, and AI observability, and translate emerging techniques into practical testing approaches
- Lead QE team for your project by setting technical direction, defining quality strategy, and ensuring delivery confidence
- Contribute to QE standards and frameworks within your project scope, establishing practices that other engineers can adopt
- Integrate automated tests into CI/CD pipelines and partner with engineering leads to embed quality into the delivery process
- Leverage data, risk signals, and production insights to continuously improve test coverage and effectiveness
Benefits
- annual incentive program
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Upload and Match Resume
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
