Manager, AI Operations

Xsolis, Inc

4d•Onsite

About The Position

We are seeking an experienced AI Operations leader to own the reliability, performance, and observability of our AI portfolio in production — spanning traditional machine learning, Generative AI, and Agentic AI systems. This role is ideal for a technically strong manager who thinks in terms of uptime, drift, and risk, and who can translate models and agents built by our data science teams into scalable, monitored, and secure production systems. You will lead a team responsible for AI production support, observability, and the operational connective tissue between our AI build organization, security, and infrastructure functions to lead tactical and strategic roadmap for us to become a leader in “Ethical AI”. This role does not own model development or AI governance policy — it owns making sure everything we ship runs well, is visible, and is safe to operate at scale.

Requirements

7+ years of experience in AI/ML operations, MLOps or production data science, including 2+ years of people or team leadership.
Hands-on experience operating traditional ML and GenAI systems in production (deployment, monitoring, incident response).
Strong understanding of observability practices — drift detection, model performance monitoring, logging, alerting — at scale.
Experience partnering with Security and Infrastructure teams on production risk and access controls.
Excellent communication skills, with the ability to translate operational risk into terms both technical and business stakeholders understand.
Must pass a background check.
Must be authorized to work in the United States on a full-time, permanent basis without the need for current or future sponsorship.

Nice To Haves

Prior experience in healthcare, health tech, payer, or provider organizations.
Experience operating agentic AI systems specifically (tool-use monitoring, guardrails, action-level tracing).
Familiarity with MLOps/observability tooling (Datadog, AWS Cloud Watch, LangFuse) and native AWS cloud platform capabilities.
Experience working alongside a dedicated AI Governance function, with clear boundaries between operational monitoring and governance policy.

Responsibilities

Own deployment support, monitoring, and day-to-day reliability of AI systems in production across traditional ML, GenAI, and agentic AI.
Lead incident response and root-cause troubleshooting for AI/ML services, including agentic systems with autonomous or tool-using behavior.
Partner with MLOps, Cloud Engineering, and Infrastructure teams to build and maintain robust deployment pipelines.
Build and maintain observability practices (drift detection, latency, uptime, accuracy decay, agent action-level tracing) across all AI system types.
Define and track production health metrics; flag performance degradation and route it to the appropriate build team for remediation.
Establish dashboards and alerting that give the organization real-time visibility into AI system health.
Serve as the primary connection point between AI systems and enterprise Security, Infrastructure, and IT teams — ensuring production AI meets security and access-control standards.
Support the AI Governance function by instrumenting and operating the fairness, safety, and bias-monitoring metrics it defines (governance sets the requirement; this team builds and runs the instrumentation).
Coordinate handoffs from the AI Build team into production, including a defined production-readiness review before launch.
Lead and grow a small, high-leverage production support and observability team.
Set clear priorities across production support and monitoring workstreams and manage team capacity as the AI portfolio scales.
Mentor team members and build a strong operational and on-call culture.
Lead cross functional team to assess what is required in design, dev and run-time across our entire AI stack to establish “Ethical AI” best practices and SOPs for Xsolis
Ensure the Cost of AI Dev and Run-time is clear and targeted for specific annual reduction YoY
Work with Governance and Compliance team to set standards that are measurable, monitored and auditable through the observability platform across Traditional, Generative and Agentic AI.