QA Engineering Lead (AI)

Meta•Menlo Park, CA

2d•$138,000 - $191,000

About The Position

The QA Engineering Lead (AI) will lead quality engineering for a GenAI product area at Meta. The role covers end-to-end quality strategy, from early development and model evaluation through launch and ongoing product improvements. The lead will turn ambiguous requirements into measurable quality standards and partner with engineering, product, and research teams to deliver products rapidly and confidently. The position combines hands-on testing of consumer-facing and business-facing applications, APIs, backend services, and AI models with the design of scalable evaluation processes and reusable automation. It also involves assessing quality risks, guiding cross-team work, using metrics and failure analysis to inform release readiness, and mentoring QA engineers.

Requirements

Bachelor's degree in Computer Science, Computer Engineering, a relevant technical field, or equivalent practical experience; the degree must be completed before joining Meta.
At least 7 years of experience in quality assurance and test engineering for software products or services.
At least 2 years of experience in AI model testing or evaluation, including developing evaluation methods and defining model quality standards.
Experience independently leading quality engineering for a complex product or system and delivering measurable quality improvements.
Experience developing test automation or evaluation tooling in Python, PHP, Java, C/C++, or an equivalent programming language.
Hands-on experience testing and debugging consumer-facing and business-facing web or mobile applications, APIs, and backend services, and evaluating AI models.
Hands-on experience with test planning, test design and execution, performance and stress testing, and maintaining client and server test automation.
Experience using metrics and failure analysis to prioritize quality improvements and inform release decisions.
Experience coordinating work across engineering and cross-functional partners, resolving dependencies, and adapting priorities in a fast-moving development environment.
Experience communicating technical risks and trade-offs to stakeholders and mentoring other engineers.

Nice To Haves

Demonstrated improvements in product coverage, test signal quality, automation reliability, or testing efficiency within medium to large engineering teams.
Experience using data and quality metrics to identify trends, prioritize coverage gaps, and drive product quality improvements.
Experience adhering to and implementing responsible, ethical AI practices, such as risk assessment, bias mitigation, and quality and accuracy reviews.
Experience building reusable testing or evaluation frameworks adopted by multiple engineering teams.
Demonstrated ability to integrate AI tools to optimize or redesign workflows and drive measurable impact, such as efficiency gains or quality improvements.
Experience combining human evaluation with automated model-based evaluation and validating the reliability of evaluation results.
Familiarity with frontier AI model evaluation practices, including weighted scoring frameworks for assessing model quality and performance.
Experience evaluating generative text, image, voice, or multimodal models, including nondeterministic behavior and model regressions.
Demonstrated ongoing AI skill development, such as prompt/context engineering or agent orchestration, and staying current with emerging AI technologies.
Experience partnering with AI research teams to translate model capabilities and limitations into product evaluation criteria.

Responsibilities

Own the end-to-end quality strategy for a GenAI product area, from early development and model evaluation through launch and ongoing product improvements.
Perform hands-on testing of consumer-facing and business-facing applications, APIs, and AI models.
Design, implement, and evolve AI model evaluation processes, including representative datasets, product-aligned weighted scoring frameworks, automated evaluations, human review, and regression coverage.
Establish product-specific model behavior standards for correctness, relevance, safety, and robustness in partnership with engineering, product, and research teams.
Assess quality risks, identify coverage gaps, and drive resolution of issues across model behavior, client applications, server systems, and infrastructure.
Plan and execute risk-based test strategies covering functional, integration, end-to-end, performance, stress, and cross-platform testing.
Build and maintain reusable test automation and evaluation tooling, integrate quality checks into development workflows, and improve the reliability and speed of test feedback.
Define quality metrics and use data to improve test effectiveness, testing efficiency, regression prevention, and the overall user experience.
Lead cross-team quality projects, align stakeholders on priorities and dependencies, and provide evidence-based recommendations on release readiness.
Investigate complex failures, identify root causes, and partner with engineering and infrastructure teams to prevent recurring defects.
Mentor QA engineers in test design, automation, and AI evaluation, and contribute to a quality-driven engineering culture.