Director, Control Design & Evidence

RBC•Toronto, ON

1d•Hybrid

About The Position

The Control Evidence & Assurance Lead is a senior technical leadership role responsible for ensuring AI controls are working through rigorous evaluation and documentation. You will design and assure the effectiveness of the evaluation framework that verifies the controls operate effectively, partner with control owners across the enterprise to maintain control efficacy and generate audit-ready evidence that proves controls reduce risk. Operating at the intersection of evaluation and governance requirements, you’ll translate control requirements into testable assertions, establish evidence traceability, and ensure documentation meets regulatory and audit standards. You’ll partner closely with and inform the Governance Design Lead and Runtime Governance to ensure evaluations serve governance needs and feed decision-making effectively.

Requirements

Built and implemented evaluation frameworks that verify AI controls operate as intended and produce defensible, audit-ready evidence.
Demonstrated ability to review system architecture, evaluate technical implementations, and understand code well enough to validate that controls are effective.
Hands-on coding experience (python) is a strong asset.
Partnering with engineering, governance and risk teams to convert policy and control requirements into testable assertions, measurable outcomes, and repeatable evaluation processes.
Designed scalable approaches for control validation, evidence traceability, telemetry, and continuous monitoring to improve AI control effectiveness over time.
Influential, adept at communicating complex technical findings in a way that enables engineers, risk partners, business stakeholders to make informed, evidence-based decisions.
Bachelor’s degree in Computer Science, Mathematics, Engineering, Risk Management, Systems Engineering, or related field with proficiency in evaluation frameworks and tools

Nice To Haves

Good understanding of LLM evaluation methodology including grounding, factuality, policy compliance, and adversarial testing
Professional certifications in quality assurance, risk management, or auditing (e.g., FRM, CISM, CIA, or equivalent)
Advanced degree in Computer Science, Mathematics, Engineering, Risk Management, or related discipline
Direct experience with output assurance, content moderation, policy compliance evaluation at scale, or model risk management control design and verification
Experience with adversarial attack methods, robustness testing, model monitoring systems, drift detection, or regulatory testing requirements (e.g., bias testing, fairness assessment)

Responsibilities

Defining Eval Strategy & Design Framework
Control-Specific Eval Design & Execution
Test Case Generation & Coverage Management
Eval Execution & Evidence Capture
Eval Quality & Continuous Improvement
Evidentiary Artifact Management & Assurance
Partnership Evidence Partnership & Governance Integration
Control Documentation & Owner Partnership

Benefits

Bonuses
Flexible benefits
Competitive compensation
Stock where applicable
Leaders who support your development through coaching and managing opportunities to build deep expertise in AI evaluation and control verification
Ability to make a lasting impact by designing the evaluation frameworks that ensure RBC’s AI systems are safe, compliant, and effective
Work in a dynamic, collaborative, high-performing team at the forefront of AI risk management and quality assurance in financial services
Flexible work/life balance options that support your professional and personal priorities
Opportunities to take on challenging, technical work that shapes how AI controls are verified and how evidence is captured for governance
Access to cutting-edge evaluation tools and methodologies with opportunities for continuous learning and professional growth