Model Validation 2nd LOD Lead Analyst

Citi • Wilmington, DE

1d • $173,000 - $173,400 • Hybrid

About The Position

Citibank, N.A. seeks a Model Validation 2nd LOD Lead Analyst for its Wilmington, Delaware location. This role involves providing independent review and effective challenge on the soundness and fit-for-purpose of Generative AI, AI (Artificial Intelligence) / ML (Machine Learning) objects. The analyst will act as a senior model validator, independently managing Generative AI, AI (Artificial Intelligence)/ML(Machine Learning) risk across all life-cycle activities, including initial review, ongoing monitoring, and periodic reviews. Responsibilities include conducting analysis, preparing detailed technical documentation reports to meet regulatory guidelines and industry standards, identifying weaknesses and limitations of Generative AI, AI/ML objects, and informing stakeholders of their risk profile with recommendations for compensating controls. The role requires communicating results to diverse audiences, managing stakeholder interactions with developers and owners, and interacting with senior management to facilitate understanding of risk models and inform critical decisions. Additionally, the analyst will develop NLP and GenAI applications to automate and streamline manual validation processes, develop state-of-the-art risk management practices, validation guidelines, and testing playbooks for AI/ML (including GenAI) models. This includes leading collaborative efforts with cross-functional teams, presenting complex validation findings and strategic recommendations to senior stakeholders, and successfully negotiating outcomes. The role also involves providing structured mentorship to junior validators, driving team initiatives for process streamlining and knowledge sharing, and exploring/adopting innovative methodologies or tools to enhance GenAI model validation efficiency or effectiveness. Developing internal guidance, best practices documents, or training sessions based on these initiatives is also a key part of the role. A telecommuting/hybrid work schedule may be permitted.

Requirements

• Master’s degree or foreign equivalent in Statistics, Mathematics, Finance or related field and 3 years of experience as a Model Analysis Validation Officer, Senior Model Developer, Data Scientist or related position developing AI/ML models to analyze and interpret data for a financial services institution.
• Alternatively, a Bachelor’s degree in the listed fields and 5 years of progressively responsible, post-baccalaureate experience in the listed positions.
• Model Risk Management
• Artificial Intelligence
• Machine Learning
• Natural Language Process
• Unsupervised Learning
• Statistical Data Analysis
• Python
• R
• SQL
• PyTorch
• TensorFlow
• Azure ML
• Spark
• Statistical Modeling
• Hypothesis Testing
• Perplexity
• Benchmarking (MMLU, TruthfulQA)
• Sensitivity Analysis
• Stress Testing
• Back-Testing
• Technical Documentation
• Stakeholder Engagement
• Cross-functional Collaboration
• AI Alignment
• Bias Detection & Mitigation
• Guardrail Implementation
• Interpretability
• 1 year of experience in Generative Artificial Intelligence
• Big Data
• Object Oriented Programming
• LLMs
• Transformer Architectures
• RAG Pipelines
• Prompt Engineering
• Fine-Tuning (LoRA, RLHF, PEFT)
• Embedding Models
• Vector Databases
• Multi-Modal Models
• Agentic AI Frameworks
• Databricks
• Uncertainty Quantification
• BERT Score
• HuggingFace
• MLflow
• Vertex AI
• LangChain
• BLEU
• ROUGE
• Validation Report Authoring
• Regulatory Communication

Responsibilities

• Provide independent review and effective challenge on the soundness and fit-for-purpose of Generative AI, AI (Artificial Intelligence) / ML (Machine Learning) objects.
• Act as a senior model validator to independently manage Generative AI, AI (Artificial Intelligence)/ML(Machine Learning) risk across all life-cycle activities including initial review, ongoing monitoring, and periodic reviews.
• Conduct analysis and prepare detailed technical documentation reports sufficient to meet regulatory guidelines and exceed industry standards.
• Identify weaknesses and limitations of Generative AI, AI/ML objects and inform stakeholders of their risk profile and recommend compensating controls.
• Communicate results to diverse audiences such as object owners and developers and senior management.
• Manage stakeholder interactions with Generative AI, AI(Artificial Intelligence)/ML(Machine Learning) object developers and owners across the review lifecycle.
• Interacts with senior levels of management to facilitate understanding of usage of risk models and inform critical decisions.
• Develop NLP and GenAI applications to automate and streamline existing manual validation processes, aiming to increase operational efficiency for the organization.
• Developed state-of-art risk management practice, validation guidelines, and testing playbooks for AI/ML(including GenAI) models.
• Lead collaborative efforts with cross-functional teams (e.g., model development, risk governance, business units) on key AI/ML/GenAI projects and present complex validation findings and strategic recommendations to senior stakeholders and successfully negotiating outcomes.
• Provide structured mentorship for junior validators, focusing on their technical skills, critical thinking, and communication.
• Drive team initiative to streamline a common validation process or improve knowledge sharing.
• Drive the exploration and adoption of at least one innovative methodology or tool to enhance GenAI model validation efficiency or effectiveness.
• Develop internal guidance, best practices document, or training session based on these initiatives.

Benefits

• medical
• dental & vision coverage
• 401(k)
• life, accident, and disability insurance
• wellness programs
• paid time off packages, including planned time off (vacation), unplanned time off (sick leave), and paid holidays

What This Job Offers

• Job Type: Full-time
• Career Level: Senior