Sr AI Data Science Manager

KeyBank • Brooklyn, OH

4d • $116,000 - $216,000 • Hybrid

About The Position

Within the Consumer Bank, the Senior Manager, Data Science leads and develops a team of Data Scientists who identify, build, and critically evaluate AI-powered solutions across use cases such as sales enablement, marketing content generation, enhanced customer onboarding, intelligent servicing, and novel risk assessment methodologies. The role leverages current and emerging Generative AI technologies, particularly Large Language Models (LLMs), to guide work in prompt and context engineering, data curation, model evaluation, responsible AI practices, and evidence-based solution design.

The position combines hands-on technical leadership with people management. It is accountable for delivering high-quality, well-documented, ethically sound AI solutions that satisfy model risk and regulatory requirements while advancing the bank’s broader data science strategy. The role works closely with business, technology, model validation, compliance, and risk partners and represents the data science team in cross-functional and executive discussions. The position is hybrid, with a stated expectation of 3+ days in-office, and involves occasional travel, including overnight stays.

Requirements

• Master’s or Ph.D. in a quantitative discipline.
• At least 7 years of relevant experience, including at least 2 years of direct people or team leadership.
• Demonstrated experience leading data science or AI teams, developing talent, managing delivery, setting technical standards, and guiding cross-functional partners through complex modeling and evaluation decisions.
• Deep expertise in traditional Machine Learning and Artificial Intelligence modeling practices, plus demonstrated experience with Generative AI techniques and a strong understanding of LLM architecture, lifecycle methods, evaluation approaches, capabilities, and limitations.
• Hands-on statistical coding experience in Python, including LLM libraries and frameworks such as Hugging Face Transformers, LangChain, or OpenAI API, and data manipulation libraries such as Pandas and NumPy.
• Knowledge of and ability to use traditional databases, cloud-based computing, and distributed computing.
• Knowledge of financial crime regulatory requirements, technology, and data analysis best practices.
• Strong understanding of AI ethics principles, responsible AI governance frameworks, and regulatory considerations for AI/LLM deployments.
• Excellent verbal, written, and visual communication skills, including the ability to explain complex AI concepts, evaluation findings, and risks to technical teams and non-technical executive audiences.

Nice To Haves

• Experience in Consumer Banking modeling or analytics, especially leveraging AI in these domains.
• Experience building and scaling data science teams and establishing LLM evaluation frameworks, including defining and applying relevant metrics.
• Familiarity with prompt injection vulnerabilities and mitigation strategies.
• Experience evaluating Retrieval-Augmented Generation architectures, knowledge graph integrations, context-engineering patterns, post-training or model-adaptation approaches, and LLM evaluation frameworks.
• Familiarity with vector databases, retrieval systems, knowledge graphs, orchestration patterns, MLOps principles, and monitoring approaches for supporting generative AI models at scale.

Responsibilities

• Lead, mentor, and develop a team of Data Scientists; set goals, provide technical guidance, and foster innovation, accountability, and continuous learning.
• Manage team workload, prioritization, and delivery across concurrent AI initiatives.
• Guide the development and implementation of generative AI solutions, including prompt and context engineering, retrieval-augmented generation, fine-tuning or other adaptation approaches, experimentation, evaluation methodology, data and ground-truth requirements, and model behavior analysis.
• Ensure the team’s work adheres to model risk and regulatory requirements.
• Establish and govern internal frameworks for evaluating LLM performance, including metrics for accuracy, fluency, coherence, factual correctness, toxicity, bias, and safety.
• Set consistent standards for data curation, ground-truth dataset creation, and adversarial testing.
• Oversee documentation of model configurations, prompt and context strategies, data and ground-truth assumptions, evaluation methodologies, observed behaviors, limitations, and failure modes.
• Partner with model validation, compliance, technology, and risk teams to support safe and ethical deployment of AI applications.
• Champion improvement approaches such as reinforcement learning from human feedback, supervised fine-tuning or other post-training adaptation, prompt and context optimization, and bias and safety mitigation.
• Translate generative AI concepts, evaluation findings, and potential risks into clear insights for technical and non-technical stakeholders.
• Represent the data science team in cross-functional planning and executive discussions.
• Develop team skills and business knowledge, particularly in Generative AI ethics, responsible AI principles, and LLM-specific risk management, while keeping current with emerging technologies and industry practices.
• Travel occasionally, including trips requiring overnight stays.

Benefits

• Eligible for incentive compensation, which may include production, commission, and/or discretionary incentives.
• Eligible for company benefits; specific benefits are not listed in the posting.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior