QA Program and Analytics Specialist, Critical Harms
About The Position
Reporting to the Director of Vendor Operations and Enablement, your role as the QA Program and Analytics Specialist, Critical Harms, will be essential in designing, scaling, and maintaining the quality governance framework for our Critical Harms agent operations. You will not be focused on software testing; your mission is to ensure world-class quality for human policy adherence and decision-making across our global vendor network. You are the central, strategic authority for all agent quality insights, policy health analytics, and BPO performance strategy. You will be an integral part of driving the future of agent quality at Roblox. If you're excited to join our team and build industry-leading quality governance from scratch, this is the role for you!
Requirements
- A bachelor's degree in a quantitative or relevant field.
- A minimum of 5 years of experience in Program Management, Quality Operations, or Strategy within a Trust & Safety, Content Moderation, or BPO/Vendor management environment.
- Proven expertise in designing and governing quality assurance frameworks for human performance and policy adherence (not software testing).
- Advanced proficiency in SQL, Tableau, or other BI tools for complex data interpretation, visualization, and strategic reporting.
- Exceptional ability to synthesize data into a clear narrative and present recommendations to senior executive leadership.
- Demonstrated ability to manage high-impact, cross-functional programs and hold global vendor teams accountable to rigorous standards.
- Strong team leadership and cross-functional communication capabilities.
Responsibilities
- Design, implement, and govern the global QA framework used to evaluate Critical Harms agent performance and policy application across BPO partners.
- Serve as the primary point of contact and subject matter expert for BPO QA teams, ensuring calibration and consistency of quality standards across all sites and shifts.
- Continuously monitor, evaluate, and report on policy interpretation, process adherence, and data health in the operational environment.
- Lead complex root cause analysis (RCA) efforts to diagnose systemic failures in agent judgment, policy clarity, or workflow design, driving preventative solutions.
- Engage with key stakeholders (Policy, Product, Engineering) to communicate narrative-driven insights and strategic recommendations based on quality findings, facilitating actionable change.
- Manage the vendor accountability model by defining Key Performance Indicators (KPIs) for quality and holding BPO partners accountable for meeting strict quality targets.
- Transform quality data into actionable, strategic insights that directly inform policy roadmaps, process optimization, and training curriculum, shifting the focus from detection to prevention.
- Present regular, high-impact reports to leadership, highlighting trends, quality gaps, and strategic recommendations for investment.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Industry
Administrative and Support Services
Number of Employees
1,001-5,000 employees
