Agent Tuner

McKesson•Fort Worth, TX

17h•$101,600 - $169,400•Hybrid

About The Position

McKesson is an impact-driven, Fortune 10 company that touches virtually every aspect of healthcare. We are known for delivering insights, products, and services that make quality care more accessible and affordable. Here, we focus on the health, happiness, and well-being of you and those we serve – we care. What you do at McKesson matters. We foster a culture where you can grow, make an impact, and are empowered to bring new ideas. Together, we thrive as we shape the future of health for patients, our communities, and our people. If you want to be part of tomorrow’s health today, we want to hear from you. Role Summary The Agent Tuner is accountable for what Sierra AI agents know and how they behave. This role tunes agent prompts, intents, conversation flows, and the knowledge they retrieve so that agents respond accurately, consistently, and in the McKesson voice across email, chat, and voice. This is a hands-on operator role inside the Agentic Service Experience pillar. The Tuner adjusts the agent every day; reading transcripts, testing changes, curating the agent-facing knowledge base, and shipping iterations in partnership with Sierra and McKesson Technology. They translate the AI Agent Product Manager's prioritized backlog into actual agent behavior and translate quality findings from the AI Performance & Quality Analyst into prompt, flow, and knowledge changes. This is a new role created as part of CxO's agentic service operating model and will help shape how agent tuning and knowledge curation scale through FY27 and beyond.

Requirements

Degree or equivalent and typically requires 7+ years of relevant experience.
Bachelor's degree
7+ years of experience in Technical Writing, Information Architecture, Conversation Design, Knowledge Management, or Product Operations.
1+ year of demonstrated hands-on work with AI knowledge for a deployed conversational or agentic AI system.
Demonstrated experience authoring structured content for technical or digital systems with the ability to author in a defined brand voice and translate ambiguous human guidance into precise, machine-actionable content.
Analytical mindset with attention to precision and clarity; able to read transcripts, latency, and accuracy logs to diagnose where the agent failed and why.
Excellent collaboration skills with both technical (product managers, engineers, data scientists) and non-technical stakeholders (frontline supervisors, SMEs, Legal, brand).
Experience working in environments with evolving requirements and limited established processes. This role is shaping how the work gets done, not following an established playbook.

Nice To Haves

Experience working with leading LLMs (e.g., Claude, GPT-4, Gemini) and Gen AI concepts including RAG, agentic AI, chain-of-thought prompting, decision flows, and AI reasoning.
Direct experience with the Sierra platform or a comparable agentic AI platform deployed at enterprise scale.
Systems thinking: ability to identify downstream implications of knowledge gaps and judgment around edge cases, ambiguity, and exception handling.
Understanding of semantic precision in the AI context.
Experience managing knowledge bases or content systems (e.g., KM platforms, CMS tools)
Contact center, customer service, or BPO domain experience in healthcare, life sciences, or regulated industries.
Prior experience as a contact center agent, trainer, QA analyst, or knowledge base owner before moving into AI.
Familiarity with responsible AI principles and risk considerations for regulated customer interactions.
Familiarity with SQL, Python, or JSON is a plus.

Responsibilities

Author and structure prompt-aware content for AI agents. Write knowledge so agents respond accurately and empathetically without hallucinating. Decide where the agent needs no additional content, where it needs context, where it can pull data from upstream systems, and where it needs detailed instructions. Tailor knowledge accordingly.
Incorporate knowledge from across NAPS. Pull domain expertise from SMEs across service delivery teams into the agent knowledge base so agents respond to real customer scenarios, not just documented ones.
Author in the McKesson voice. Ensure agent responses are brand-aligned, empathetic, and appropriate for healthcare and pharmaceutical customers. Partner with brand, communications, and Responsible AI partners on tone and standards.
Ensure content is relevant, accurate, current, non-conflicting, and succinct. Identify gaps in content that cause agents to provide wrong answers or fail to help and fix them quickly.
Optimize knowledge for retrieval. Clean up metadata, improve vector match probabilities, and decide how to chunk knowledge so retrieval is accurate and efficient on latency and cost. Treat the knowledge base as a system the agent reasons over, not a document library.
Maintain knowledge freshness. Establish review cadences, last-verified dates, and removal protocols. Stale knowledge in an agent KB becomes a confident wrong answer; manage it accordingly.
Analyze failed AI conversations to identify knowledge gaps. Read transcripts, listen to calls, and read latency and accuracy logs to see where the agent “got lost” in the knowledge base. Update documentation or metadata to improve future performance and validate the fix worked.
Optimize retrieval accuracy and response crafting. Use working knowledge of the Sierra tech stack to ensure responses are crafted according to segmentation requirements and decision logic across customer groups and channels.
Feed back to content owners. Provide feedback to the human-facing KB owners and content standards teams when their standards or content are hindering agent performance, and partner on resolution.
Provide feedback to product and engineering. Inform the AI Agent PM and McKesson Technology on prompt improvements and platform enhancements needed for authoring effective agent knowledge. Operate in an agile fashion to adapt to a rapidly changing AI ecosystem.
Experiment with knowledge approaches. Lead use case evolution by experimenting with options to decide how much deterministic knowledge is necessary versus where to depend on LLM reasoning. The right answer is different for different intents, customer groups, and risk levels.