QA Engineering Lead (AI)

Meta•Menlo Park, CA

2d•$163,000 - $224,000

About The Position

Meta is seeking QA Engineering Leads to set technical direction and lead major quality initiatives for its generative AI products. This hands-on role shapes quality strategy across related products and platforms, leads the design and implementation of shared testing and evaluation capabilities, and helps other engineers deliver high-quality work. The role involves solving complex, ambiguous quality problems and building evaluation systems that enable multiple teams to ship reliable AI experiences. The lead defines quality strategy and roadmaps for major AI initiatives, performs hands-on testing of applications, APIs, and AI models, and builds shared AI evaluation and test automation systems. The role establishes evaluation methods and quality standards, improves the reliability of evaluation signals, identifies systemic quality risks, and drives launch planning and quality metrics. It also includes cross-functional technical partnership, adoption of reusable quality practices, and mentoring QA engineers and technical leads.

Requirements

Bachelor's degree in Computer Science, Computer Engineering, a relevant technical field, or equivalent practical experience; a degree in progress must be completed before joining Meta.
10+ years of quality assurance, test engineering, or software engineering experience, including substantial experience building quality engineering systems.
3+ years of experience in AI model testing or evaluation, including designing evaluation frameworks and establishing model quality standards.
Experience leading a major technical quality initiative across multiple teams from problem definition and architectural design through implementation, rollout, and measurable outcomes.
Experience architecting and implementing scalable testing, automation, or evaluation systems in Python, PHP, Java, C/C++, or an equivalent programming language.
Experience solving complex quality and reliability problems across client applications, backend services, and distributed systems.
Hands-on experience testing consumer-facing and business-facing applications and APIs, and evaluating AI models.
Hands-on experience with risk-based test strategy, performance and stress testing, and designing and maintaining client and server test automation.
Experience evaluating the reliability and limitations of quality metrics and using data to guide technical decisions.
Experience leading quality engineering in fast-paced development teams and adapting testing strategies and priorities as requirements evolve.
Experience setting technical direction, resolving ambiguous requirements and cross-team dependencies, and influencing stakeholders without direct authority.
Experience communicating complex technical problems, quality risks, and investment trade-offs to technical and non-technical stakeholders.
Experience mentoring engineers or technical leads and driving adoption of quality engineering practices beyond a single project.

Nice To Haves

Experience leading evaluation strategy for generative text, image, voice, or multimodal products at scale.
Experience designing shared evaluation infrastructure or testing frameworks and leading their adoption across multiple engineering teams.
Experience designing evaluation systems that combine human review, model-based judgments, adversarial testing, and automated regression detection.
Experience implementing responsible and ethical AI practices, such as risk assessment, bias mitigation, and quality and accuracy reviews.
Demonstrated improvements in product reliability, evaluation coverage, regression prevention, or engineering efficiency across a major product initiative.
Demonstrated ability to integrate AI tools to optimize or redesign workflows and drive measurable impact, such as efficiency gains or quality improvements.
Experience leading cross-platform quality efforts within large engineering organizations and building consensus on technical standards and priorities.
Demonstrated ongoing AI skill development, such as prompt or context engineering and agent orchestration, and staying current with emerging AI technologies.
Experience applying frontier AI model evaluation practices, including weighted scoring frameworks for assessing model quality and performance.
Experience with statistical evaluation methods, benchmark validity, evaluator calibration, and connecting offline evaluation results to production quality.
Experience partnering with AI research teams and translating evolving research capabilities into scalable product quality practices.

Responsibilities

Define and drive technical quality strategy and roadmaps for major generative AI initiatives spanning multiple engineering teams.
Lead and perform hands-on testing of consumer-facing and business-facing applications, APIs, and AI models.
Architect and implement shared AI evaluation and test automation systems for multiple models, products, and platforms; directly solve challenging technical problems.
Establish evaluation methods, benchmark design, product-aligned weighted scoring frameworks, and quality standards for model behavior and end-to-end product experiences.
Develop reliable evaluation signals by calibrating human and automated judgments, analyzing measurement uncertainty, and investigating gaps between offline evaluations and real-world outcomes.
Identify systemic quality risks across models, data, applications, and infrastructure, and lead cross-team efforts to address root causes.
Drive quality planning for launches and phased rollouts, aligning teams on success criteria, dependencies, monitoring, and regression response.
Define metrics connecting model performance, product reliability, and user experience, and use them to guide quality priorities and investment decisions.
Lead adoption of reusable evaluation frameworks, automation, and development practices that improve quality and reduce testing costs and time to feedback.
Partner with engineering, infrastructure, research, product, and data teams to resolve technical trade-offs and influence product and system designs.
Mentor QA engineers and technical leads and raise engineering quality across the broader team.