Director of Quality

LILT•Boston, MA

6d•$115,000 - $140,000

About The Position

The Director of Quality owns LILT's quality system end to end, including standards, audits, compliance, procedures, continuous improvement, and the roadmap and operation of Quality Agents. The role will help shift Quality from a primarily reactive, execution-led function to a proactive, data-driven partner for customers and internal teams, with quality designed into AI-native workflows. This hands-on quality subject-matter expert and operator will combine experience designing and building AI agents and improving LLM output with a solid grounding in traditional quality management. The Director serves as the single Quality partner for Customer Success, Production, Research, and Product, translating customer requirements into measurable acceptance criteria, effective controls, and improved models and workflows. The role advises customers on trade-offs among quality, cost, and speed, including appropriate use of automated, agent-verified, and human-verified workflows, with stronger assurance for high-risk and regulated content. The position is remote or hybrid in the EMEA or US Eastern time zone, within the Production, Quality team. It reports to the VP Production and directly manages a Quality Manager (AMER, customer staffing) and two Quality Coordinators.

Requirements

Demonstrable, hands-on experience designing, building, and running LLM-based agents in production.
Scripting or data skills, such as Python or SQL, and experience turning quality data into actionable metrics, preventive controls, and measurable improvements.
Proven experience improving LLM output quality.
Significant experience running quality systems, including leading internal and external audits, compliance programmes, and writing and maintaining processes and procedures.
Experience leading structured continuous improvement, including root cause analysis, corrective and preventive actions, and KPI reviews.
Experience managing and developing people.
Strong customer advisory and cross-functional partnership skills, including translating customer quality requirements into acceptance criteria and governance, advising on quality/cost/speed trade-offs, and working credibly with customers, executives, and technical teams.

Nice To Haves

Quality certifications such as Six Sigma.
Background in localisation, translation, or language services.
Experience in customer-facing quality roles for enterprise accounts.
Track record of collaborating with research scientists and product managers to turn quality insights into product requirements and shipped improvements.

Responsibilities

Evaluate, design, and build reliable Quality Agents with well-defined tools and clear, structured instructions.
Use orchestration patterns to evolve single agents into multi-agent systems.
Define evaluation criteria, test sets, and success metrics for agent performance; operate agents in production with monitoring and escalation paths.
Drive Quality Agent adoption across teams and demonstrate measurable reductions in manual quality effort.
Partner with Research and Product on ADQ, LLM model improvements, AI Review, AI QA, and emerging AI workflows.
Turn quality findings into product requirements, evaluation criteria, and continuous feedback loops that improve model and workflow output.
Represent quality in roadmap discussions and help determine when human review, AI review, or both are appropriate.
Own internal and external audit approaches, including ISO certification and customer audits.
Plan and conduct periodic internal audits, manage evidence collection, and track corrective and preventive actions to closure.
Standardize and maintain SOPs, workflows, and quality documentation as AI workflows change.
Ensure quality processes meet customer, contractual, and regulatory requirements.
Partner with Customer Success and Delivery to define proactive quality programmes, success metrics, and governance for strategic accounts, including AI-forward enterprise and life science/regulatory programmes.
Translate customer quality requirements into pre-delivery acceptance criteria, validation methods, and risk-appropriate controls.
Advise customers on quality, cost, and speed trade-offs and appropriate automated, agent-verified, human-verified, or combined workflows.
Represent LILT's quality function in customer business reviews, audits, and escalations, using quality data and metrics to demonstrate fitness for purpose.
Lead structured continuous improvement across the Commercial business, including root cause analysis, corrective and preventive actions, KPI reviews, and retrospectives.
Own a proactive quality metrics framework covering errors, PQs, rework, and trends; use it to prioritize improvements and demonstrate outcomes.
Strengthen quality controls across review, delivery, and follow-through, including mandatory account-specific checks, intervention on critical findings, and clear escalation ownership.
Partner with Product and Engineering to automate checks and escalation, and verify corrective actions prevent recurrence.
Partner with Expert Supply and Ops to embed processes, guidance, and quality feedback that maintain a high-quality pipeline of experts.
Use quality findings and recurring failure patterns to improve expert qualification, guidance, and ongoing quality management.
Lead and develop the Quality team, including the customer-facing Quality Manager and Quality Coordinators.
Establish the team's operating cadence and evolve team practices as Quality Agents reduce manual coordination work.
Oversee embedded Quality Manager staffing sold as a service to customers, ensuring that role owns the specific customer-funded quality programmes and initiatives.
Develop playbooks for AI-driven quality management and keep staffing-as-a-service delivery aligned with LILT's quality standards.