GenAI QE Strategy Lead

RBC•Vancouver, BC

2d•Onsite

About The Position

Lead QE to design, develop and implement testing strategy and automation tools for a GenAI based development project. Will need to design the testing framework to be implemented in a CI/CD pipeline. Will be working with the BA and Dev leads in Vancouver, plus other team members based out of Toronto / Halifax / Offshore. This role will work on LLM integration and testing along with QA automation.

Requirements

Undergraduate degree coupled with change management experience in complex programs with high frequency of requests and multiple priorities.
Demonstrated ability in written and oral communication skills along with strong presentation skills.
Ability to determine the information and communication needs of the stakeholders and project.
Strong organizational and time management capabilities.
Deadline-driven and results-oriented; able to meet consistently high quality standards while handling a variety of tasks and deadlines simultaneously.
End to end test architecture / test integration for GenAI development project
GenAI Experience
Strong experience in testing and evaluating GenAI models, prompt engineering frameworks, and RAG systems.
Technical Proficiency: Proficiency in Python and AI evaluation tools like LangChain.
Architecture Knowledge Ability to design scalable testing frameworks within cloud environments (AWS, Azure, GCP).
Software Testing Background: Deep knowledge of Software Testing Life Cycle (STLC), CI/CD pipelines, and API testing.
Hands-on Automation Experience in python and LLMs
Experience with Integration of multiple tools with Automation (not just selenium)
Hands-on DB experience - SQL/Snowflake
Gen AI exposure
CI/CD with test automation

Nice To Haves

Ability to facilitate between and influence key decision makers and work with team members in different time zones
Experience with regulatory reporting projects and/or financial services industry.
QTest or any other defect management tool
Experience working in Unix environment

Responsibilities

Work with the Dev and BA leads to understanding QE scope in an agile sprint approach
Create a testing framework to be used in CI/CD pipeline
Designing and coding application programs; performing testing for developed applications.
Reviewing, analyzing, and modifying programming systems, including encoding, testing, debugging and installing for a large-scale system.
Ensuring the operating efficiency of existing application systems.
Maintaining records for the documentation of systems development and revisions.
Conducting analyses of organizational needs and goals for the development and implementation of application systems.

Benefits

A comprehensive Total Rewards Program including bonuses and flexible benefits
competitive compensation
commissions
stock where applicable
Leaders who support your development through coaching and managing opportunities
Ability to make a difference and lasting impact
Work in a dynamic, collaborative, progressive, and high-performing team
A world-class training program in financial services
Flexible work/life balance options
Opportunities to do challenging work