AI Operations Manager

Fanatics Betting & Gaming•Denver, CO

6d•$100,000 - $135,000

About The Position

Fanatics Betting & Gaming (FBG) is building some of the most advanced AI-powered customer experience systems in the betting and gaming industry. We are investing heavily in a portfolio of generative AI initiatives and are looking for an AI Operations Manager to help bring these capabilities to life and keep them running at scale. This role sits at the intersection of applied AI, data analytics, and customer operations. You will own the day-to-day performance, iteration, and quality of our AI systems, with a particular emphasis on our customer-facing AI Agent. Beyond the AI Agent, you will support an expanding portfolio of AI tools that augment our customer support agents and proactively resolve customer issues before they escalate. You bring hands-on technical depth in how AI systems work, a strong instinct for data, and the ability to translate insights into meaningful product improvements.

Requirements

3+ years of experience in AI operations, applied AI, technical product management, or a closely related data-heavy technical role.
Hands-on technical understanding of how LLMs work including prompt engineering, context management, retrieval-augmented generation (RAG), evaluation strategies, and the tradeoffs between models and inference approaches.
Strong data skills: proficiency in SQL, comfort with analytics tooling, and the ability to independently build dashboards, run analyses, and extract actionable insights from complex datasets.
Experience operating AI systems in production including monitoring, regression detection, and structured iteration based on performance data.
Proven ability to work cross-functionally across engineering, operations, and compliance in a fast-moving environment.
Excellent written and verbal communication skills, including the ability to present AI performance and insights clearly to non-technical stakeholders.
Prior people management experience, with a track record of building and developing small technical or operations teams.

Nice To Haves

Hands-on experience building or managing GenAI-powered chatbots, virtual agents, or co-pilot tools in production environments.
Experience in regulated industries such as online gaming, fintech, payments, or e-commerce.
Familiarity with agent frameworks, tool-calling patterns, and multi-step reasoning in LLM systems.
Experience with customer support platforms, CRM systems, or contact center tooling.
Comfort working with data platforms such as Snowflake, BigQuery, or similar; experience with Python is a plus.
Experience collaborating with ML or Data Science teams on model evaluation, fine-tuning, or experiment design.
Demonstrated ability to scale and mature AI systems with a focus on trust, reliability, and safety.
Experience managing teams responsible for knowledge management, content operations, or AI evaluation/QA.

Responsibilities

Own the end-to-end performance of FBG’s GenAI customer service AI Agent, from prompt engineering and quality to escalation logic and containment metrics.
Drive continuous improvement across containment rate, resolution rate, retention rate, and user satisfaction with a data-first approach to prioritization.
Design and refine conversational flows, intent coverage, fallback handling, and multi-turn experiences in close partnership with Engineering and Product.
Ensure chatbot responses are accurate, policy-compliant, brand-aligned, and safe, including guardrails for hallucination prevention and confident human handoff thresholds.
Own knowledge article quality and freshness, including ingestion workflows and automated content evaluation.
Support the buildout and optimization of an AI co-pilot for customer support agents, including response suggestions, knowledge surfacing, and real-time guidance tooling.
Contribute to the development of an in-app predictive troubleshooting system that proactively identifies and resolves customer issues before they require agent intervention.
Flex across additional AI initiatives as the portfolio grows; this role is built for breadth, and the roadmap will evolve alongside FBG’s AI ambitions.
Monitor CS AI system performance daily; identify regressions, anomalies, and edge cases before they impact customers.
Support and help maintain QA frameworks, evaluation datasets, and testing protocols across AI systems.
Own incident response for AI-related issues, including root-cause analysis, corrective actions, and post-mortems.
Establish and evolve governance processes for model updates, prompt changes, content refreshes, and release readiness.
Own the analytics layer for AI system performance, building dashboards, writing SQL, and generating insights that drive roadmap decisions.
Synthesize signals from conversation data, user behavior, agent feedback, and operational metrics to identify opportunities and surface risks.
Translate complex data findings into clear narratives for both technical and non-technical stakeholders, including executive-level reporting.
Partner with Data Engineering and Analytics teams to ensure reliable, well-modeled data pipelines supporting AI evaluation and reporting.
Partner with Engineering, Product, Customer Support, Compliance, and Legal to coordinate delivery across the AI portfolio.
Ensure AI systems meet regulatory requirements across gaming, payments, and data privacy, working closely with Compliance and Responsible Gaming teams.
Communicate status, risks, and outcomes clearly to senior leadership, with the ability to tailor messaging to both technical and executive audiences.
Drive internal enablement and documentation so that operational and support teams can effectively work alongside AI systems.
Manage a small, dedicated team responsible for the creation and ongoing management of groundings (knowledge articles, retrieval content, and prompt context) for all AI agents across FBG.
Lead the team's testing and quality assurance function, including evaluation dataset development, regression testing, golden-answer authoring, and pre/post-release validation across the AI portfolio.
Set goals, prioritize the team's roadmap, and develop team members through coaching, feedback, and clear performance expectations.
Build scalable processes for grounding intake, refresh cadence, and quality scoring so coverage keeps pace with new products, promotions, and regulatory changes.