AI Engineer

Scotiabank•Dallas, TX

About The Position

Scotiabank is seeking a Senior AI Engineer to design, build, and operate AI and agentic solutions supporting Canadian Banking, International Banking, Global Banking and Markets, Global Wealth Management, Fraud, and Anti-Money Laundering. This hands-on engineering role spans AI applications, data pipelines, software services, and MLOps. The engineer will work with business stakeholders, product owners, data scientists, and platform teams to turn business workflows into secure, reliable production solutions using Databricks, MLflow, Kubernetes, and Azure-native services. The role owns the design and delivery of assigned components and workflows, contributes to technical decisions with Principal Engineers and architects, and supports solutions throughout their production lifecycle. It requires sound judgment when choosing between deterministic automation, machine learning, generative AI, and agentic approaches. The engineer will participate in business discovery and solution design; build LLM-powered applications, agentic workflows, data pipelines, APIs, and services; and establish evaluation, MLOps, CI/CD, deployment, monitoring, and operational practices. The role also includes implementing security and responsible-AI safeguards, collaborating with enterprise partners on compliance and governance, and contributing to engineering quality through reviews, mentoring, reusable components, and clear communication. The position is based in Dallas, Texas. Scotiabank describes its benefits as flexible programs supporting employees’ family, financial, physical, mental, and social health needs. The bank emphasizes an inclusive, accessible, and equal-opportunity workplace.

Requirements

Strong professional experience in software, data, or machine learning engineering, including hands-on delivery of production AI or machine learning solutions.
Proficiency in Python and SQL, API development, automated testing, Git, and modern software engineering practices.
Hands-on experience building generative AI applications or agentic workflows, including retrieval, tool integration, orchestration, and evaluation.
Experience with Databricks, Apache Spark, Delta Lake, and scalable data processing pipelines.
Experience with MLflow and MLOps practices such as experiment tracking, model registration, versioning, deployment, and monitoring.
Practical experience with Microsoft Azure services and cloud application security, identity, networking, and observability.
Experience with Docker, Kubernetes, CI/CD pipelines, and infrastructure as code, including deploying and troubleshooting services.
Ability to independently deliver well-defined engineering work, investigate ambiguous problems, and collaborate across business and technical teams.
Understanding of AI evaluation, data privacy, secure tool use, human oversight, and limitations of probabilistic model outputs.

Nice To Haves

Experience in financial services, particularly banking, capital markets, wealth management, fraud detection, or AML.
Experience implementing retrieval and vector search, document processing, semantic models, or knowledge graph integrations.
Familiarity with Model Context Protocol (MCP), event-driven architectures, and agent orchestration frameworks.
Experience working in regulated environments with model risk, audit, and data governance requirements.
Bachelor’s or master’s degree in computer science, engineering, mathematics, or a related discipline, or equivalent practical experience.

Responsibilities

Participate in discovery sessions with business stakeholders to understand workflows, data sources, pain points, and opportunities for AI-enabled improvement.
Translate requirements into technical designs, implementation tasks, estimates, and testable acceptance criteria with product owners and technical leads.
Assess data readiness, integration dependencies, and technical feasibility; build prototypes to validate assumptions.
Define appropriate boundaries for agent autonomy, human review, approvals, and deterministic business rules.
Build LLM-powered applications and agentic workflows using tool calling, enterprise APIs, retrieval-augmented generation, structured outputs, and state management.
Implement document ingestion, parsing, chunking, embeddings, indexing, retrieval, and reranking to provide relevant, permission-aware context.
Integrate predictive models and business rules into AI workflows with data scientists and domain experts.
Implement validation, timeouts, retries, idempotency, and fallback paths for dependable workflows.
Compare model and orchestration options based on task quality, latency, cost, and operational requirements.
Develop and maintain batch and streaming data pipelines using Azure Databricks, Apache Spark, Delta Lake, Python, and SQL.
Implement data quality checks, schema validation, lineage, and access controls using enterprise standards and Unity Catalog where applicable.
Build tested APIs and services connecting AI applications to enterprise data, business systems, and approved tools.
Write maintainable, documented code with clear interfaces and unit and integration tests; participate in peer review and troubleshoot application and data issues.
Use MLflow and supporting tools to track experiments, register models, version prompts and configurations, and retain evaluation results and deployment evidence.
Build representative evaluation datasets and automated checks for model performance, retrieval relevance, grounded responses, tool execution, and end-to-end task completion.
Implement CI/CD pipelines, automated regression testing, environment configuration, and controlled releases with rollback support.
Containerize services with Docker and deploy to Kubernetes, including AKS, using approved infrastructure-as-code patterns.
Instrument applications with tracing, logs, and metrics; monitor AI quality, data and model drift where applicable, latency, availability, and consumption costs.
Investigate production incidents, perform root-cause analysis, maintain runbooks, and support operations under the team’s agreed support model.
Implement least-privilege access, secure secrets management, data protection, and audit logging using enterprise security and Azure identity standards.
Apply safeguards against prompt injection, sensitive data disclosure, unsafe tool execution, and unauthorized agent actions.
Implement human approval and escalation for sensitive decisions or actions, preserving traceability of inputs, outputs, and tool activity.
Work with cybersecurity, model risk, compliance, and data governance partners to address controls and remediation items.
Align component designs with enterprise architecture and reusable platform capabilities in collaboration with Principal Engineers and architects.
Collaborate with data engineers, data scientists, platform engineers, and business teams to resolve dependencies and deliver integrated solutions.
Mentor engineers through code reviews, pairing, and knowledge sharing; contribute reusable components and practical engineering guidance.
Communicate progress, blockers, technical trade-offs, and delivery risks clearly and early.