Principal Data Engineer (US)

TD Bank•Mount Laurel, NJ

11h

About The Position

The Principal Data Engineer provides senior technical expertise to develop bank- and industry-wide solutions that close capability gaps and create value for customers. The role plans, evaluates, recommends, designs, operationalizes, and supports efficient, reliable, scalable, and secure solutions. It provides technical leadership and direction for solutions that address emerging business needs, strategies, and efficiency opportunities, and advances data engineering architecture and technology capabilities in line with industry developments. This enterprise-level specialist works autonomously on high-profile, complex, high-risk initiatives with significant organizational impact. The role leads and oversees other data engineering professionals on strategic projects, establishes working groups across Journeys and Platform, and provides consultative guidance to executive leadership. It combines expertise in data engineering, data management, architecture, application security, and coding, and typically reports to the data engineering practice owner. The position advances data engineering practices, standards, tools, automation, and delivery processes. It develops end-to-end solution options and detailed designs, oversees application and systems development, and ensures alignment with architecture, requirements, performance expectations, and development standards. The Principal Data Engineer also leads data governance, metadata management, security, risk management, testing, deployment, and release-related activities while mentoring team members and collaborating with internal and external stakeholders. The role requires a university or graduate/postgraduate degree in Data Management, Computer Science, Engineering, or a related discipline, or equivalent practical experience, and 15 or more years of relevant experience. Preferred qualifications include extensive experience with Azure data platforms, Python, SQL, Spark/PySpark, modern data architectures, DevOps and infrastructure automation, data governance and security, and at least three years of hands-on AI and generative AI experience.

Requirements

University or graduate/postgraduate degree in Data Management, Computer Science, Engineering, or a related discipline, or equivalent practical experience.
15 or more years of relevant experience in the field of specialization.
Ability to provide senior technical leadership and work autonomously on complex, high-profile, high-risk initiatives.
Expertise spanning data engineering, data management, architecture, solutions architecture, application security, and coding.
Ability to lead and oversee data engineering professionals and provide strategic and consultative guidance to executive leadership.
Knowledge of data engineering standards, security coding practices, risk management, and applicable internal policies and regulatory guidelines.

Nice To Haves

15 or more years of experience with Microsoft Azure data platforms, including Azure Data Factory, Azure Databricks, ADLS Gen2, Synapse Analytics, and cloud-native data architecture patterns.
15 or more years of data engineering and programming experience, with expert-level Python, SQL, and PySpark/Spark proficiency and experience building high-volume batch and real-time pipelines using technologies such as Kafka.
15 or more years of experience with modern data architecture, including data lakehouse, data mesh, Data as a Service (DaaS), ETL/ELT, distributed processing, API-based integration, and scalable data platforms.
15 or more years of experience with DevOps, infrastructure, and automation, including Git, Azure DevOps/GitHub, CI/CD, Terraform, automated testing, observability, and Infrastructure as Code.
15 or more years of experience with data governance, security, and quality, including Microsoft Purview and metadata management, cataloging, lineage, data quality, access controls, encryption, and governance in regulated environments.
At least 3 years of hands-on experience with generative AI, LLMs, RAG, vector databases, embeddings, and AI/ML data pipelines.
Experience with Azure AI Foundry, Azure OpenAI, and MLflow to build and operationalize enterprise AI solutions.

Responsibilities

Advance data engineering practices through processes and tools, and set development and design standards at the business-unit or enterprise level.
Execute data engineering strategy through new tools, automation, improved delivery processes, robust design practices, and adoption of improved paradigms across applications, platforms, and infrastructure.
Lead the creation of solution options and comprehensive end-to-end solutions, coordinating stakeholders to support development initiatives.
Develop and document detailed solution designs and act as an application subject-matter expert throughout the design and development lifecycle, ensuring alignment with architecture.
Oversee complex application and systems development, programming, and coding; develop functional, technical, and user-interface designs that meet business requirements and enterprise architecture, performance, and development standards.
Consult on requirements and specifications, apply functional knowledge of business needs, and review technical deliverables for quality, gaps, and redesign needs.
Identify, recommend, source, negotiate, and implement buy, build, or reuse improvements that enhance customer experience and availability or reduce cost.
Work with Technology partners to define application metrics and performance goals; identify and resolve application, data, cross-capability, and cross-release issues affecting integrity.
Analyze business requirements and system environments to recommend solutions within business technology architecture and gather infrastructure implementation requirements.
Work with Architects to ensure environment changes, data-conversion impacts, and other areas of impact are addressed and meet project performance requirements.
Lead coordination and expectation management with technology teams, including ITS, ARE, Architecture, and Enterprise Protect, to support delivery.
Lead or conduct data analysis and investigations of complex issues and test cases during UAT/PAT; serve as an escalation point for post-implementation warranty activities.
Approve source-code repository check-in/check-out and coordinate code packaging and CI/CD deployment with ITS/ARE teams.
Lead application design and architecture reviews and act as an escalation point for ServiceNow requests and release-management activities.
Lead data engineering initiatives and capabilities, data governance principles, and enterprise or segment metadata-management initiatives.
Lead standard security coding practices, oversee documentation of technical decisions, risks, and lessons learned, and identify, manage, and escalate risks and non-standard high-risk activities.
Adhere to internal policies, procedures, and applicable regulatory guidelines; support the Technology asset owner and enterprise protection team on risk, information security, and audit matters.
Lead initiatives on emerging trends and developments, maintain business and technical knowledge, and contribute research, analysis, collaboration, and problem-solving to the engineering practice.
Mentor and support team members by sharing knowledge and engineering best practices, providing guidance, and proactively identifying and resolving issues.
Keep stakeholders informed of project progress and relevant day-to-day information, contribute to a positive and equitable team environment, and build productive relationships with internal and external partners.