Data Engineer III (US)

TD Bank•Mount Laurel, NJ

3d•Onsite

About The Position

The Data Engineer III provides a broad range of data engineering functions, including data modeling, data quality and profiling, data acquisition and ingestion, ETL, metadata enrichment and management, data provenance and lineage, and other specialized data management functions. The role performs highly complex technical work independently and applies expert knowledge of data engineering frameworks, technologies, tools, processes, patterns, and procedures, including their integration with areas such as architecture and infrastructure. It designs, develops, tests, debugs, and executes solutions to business and technical requirements, using and evolving established patterns and developing new ones where needed. The role is a subject matter expert across multiple related areas and consults with peers and partners across teams on research, analysis, design, and support. The Data Engineer III analyzes complex data-management requirements for platforms or journeys; develops data models, ETL jobs, and frameworks; supports data acquisition, ingestion, development, testing, and production issue resolution; and communicates complex technical details to non-technical business partners. It ensures data, metadata, lineage, governance, and technical practices align with enterprise standards and policies. The role contributes to design and architecture reviews, source control, CI/CD and deployment, release management, QA investigations, post-implementation support, risk management, and the development of data engineering capabilities and best practices. It also collaborates across technology and business teams, shares knowledge, mentors colleagues, and supports productive stakeholder relationships.

Requirements

Degree, postgraduate degree, technical certificate in Data Management or a related discipline (such as Computer Science or Engineering), or equivalent practical experience.
At least 5 years of relevant experience.
Expert knowledge of data-engineering frameworks, technologies, tools, processes, patterns, and procedures.
Ability to independently perform highly complex technical tasks and design, develop, analyze, integrate, code, test, debug, and execute solutions.
Experience applying data modeling, data quality, profiling, acquisition and ingestion, ETL, metadata management, provenance, and lineage practices.
Ability to work across teams and communicate complex technical information to non-technical partners.
Ability to support data development, testing, QA investigations, production issues, source control, deployment, and release activities.
Ability to identify and manage risks and follow enterprise standards, security practices, internal policies, and applicable regulatory guidelines.

Nice To Haves

Graduate degree preferred.
5+ years of experience with Azure Databricks, including designing, developing, optimizing, and supporting notebooks, transformations, and scalable data-processing workloads in enterprise environments.
5+ years of experience with PySpark, including distributed data transformations, large-scale data processing, performance optimization, and troubleshooting.
5+ years of experience with SQL, including complex query writing, optimization, troubleshooting, data-validation logic, and performance-intensive workloads.
5+ years of experience with Azure Data Factory, including designing, developing, scheduling, monitoring, and troubleshooting enterprise ingestion and orchestration pipelines.
5+ years of experience with big data, ETL, and data engineering, including data-integration patterns, batch and streaming processing, pipeline design, performance, scalability, reliability, and production support.
5+ years of experience with GitHub or Bitbucket, including source control, branching, pull requests, code reviews, versioning, and collaborative engineering practices.
5+ years of demonstrated analytical and problem-solving skills, including complex issue investigation, root-cause analysis, evaluating solution options, and implementing sustainable solutions.
3+ years of experience with Delta Lake, including lakehouse solutions, table design, schema management, data versioning, performance optimization, and scalable data operations.
3+ years of experience with Unity Catalog, including centralized governance, access controls, metadata management, lineage, and data discovery across Databricks environments.
3+ years of experience with CI/CD and DevOps practices, including automated build, test, deployment, and promotion processes for data pipelines, notebooks, and related components.

Responsibilities

Analyze data and assess data-management requirements for a platform or journey, including complex work involving multiple pods or products.
Maintain expert knowledge of upstream data through profiling, data-quality reporting, and metadata production.
Support data acquisition and ingestion.
Explain complex, large-scale, high-impact technical design and development details to non-technical business partners.
Elicit and analyze business and data requirements and develop complete solutions, including data models, ETL and business rules, lifecycle management, governance, lineage, and metadata.
Ensure data complies with enterprise standards, policies, and guidelines.
Develop and maintain complex data models using industry-standard modeling tools.
Develop and maintain complex ETL jobs and frameworks using the bank’s standard tools.
Support development and testing teams in resolving data issues, including escalation support for complex issues.
Support partners and stakeholders in interpreting and analyzing data.
Coordinate with technology teams, including ITS, ARE, Architecture, and Enterprise Protect, to support delivery success.
Support QA investigations and data analysis for complex issues and test cases during SIT, UAT, and PAT testing.
Provide oversight of post-implementation activities during the warranty period.
Execute and approve source-code check-in and check-out as part of source-code management.
Work with ITS and ARE teams on code packaging and deployment through CI/CD into higher environments.
Lead participation in application design and architecture reviews.
Raise ServiceNow requests and work with change management to support release activities.
Lead data engineering initiatives and capabilities, including application of data-governance principles across the organization.
Ensure metadata and data lineage are captured and compatible with enterprise metadata and data-management tools and processes.
Follow and contribute to secure coding practices to reduce common application vulnerabilities.
Identify, document, and address technical decisions, risks, lessons learned, and enhancements.
Identify and manage organizational risks and escalate non-standard or high-risk activities as needed.
Follow internal policies, procedures, and applicable regulatory guidelines; keep current with emerging trends and relevant tools and techniques.
Participate as a team member, promote a positive and collaborative work environment, and communicate project progress and relevant information in a timely manner.
Provide thought leadership on data-engineering best practices, transfer knowledge, and mentor and guide team members.
Participate in performance management, professional development, and cross-training.
Build productive relationships with internal and external business partners and stakeholders.
Contribute to a fair, positive, equitable, and diverse workplace and act as a brand ambassador for the business area and bank.

Benefits

Health and well-being benefits
Savings and retirement programs
Paid time off, including Vacation PTO, Flex PTO, and Holiday PTO
Banking benefits and discounts
Career development
Reward and recognition
Training programs and onboarding
Online learning platform
Mentoring programs
Variable compensation or incentive awards, potentially including cash and/or equity awards