Lead Data Engineer

M&T Bank•Wilmington, DE
•$116,400 - $194,000

About The Position

Designs, develops, implements, and operates highly complex data pipelines, data products, and data management solutions across modern cloud, warehouse, and lakehouse platforms. Applies software-engineering, data-modeling, and distributed-processing practices to deliver scalable, secure, resilient, observable, and cost-efficient solutions. Independently resolves ambiguous engineering problems, contributes to technical direction and reusable standards, and mentors other engineers while partnering across product, architecture, governance, cybersecurity, and platform teams. The role provides technical leadership and mentorship without direct people-management responsibility. It also includes adhering to the Company’s risk and regulatory standards and controls, maintaining and enhancing internal controls, escalating risk-related issues, addressing audit and regulatory findings as applicable, and promoting an environment that supports belonging and reflects the Company brand. Other related duties may be assigned.

Requirements

  • Associate’s degree and at least 7 years of systems analysis and/or application development work experience; or a bachelor’s degree and at least 5 years of such experience; or, in lieu of a degree, a combined minimum of 9 years of education and/or relevant work experience, including at least 5 years of systems analysis and/or application development work experience.
  • Expert proficiency in at least one relevant programming language and professional proficiency in at least one additional relevant programming language.
  • Advanced SQL skills, including complex transformations, analytical queries, and workload optimization.
  • Experience with Python or another general-purpose programming language used for production data engineering.
  • Hands-on engineering experience with Snowflake and/or Databricks.
  • Experience with modern ELT/ETL and transformation practices, including dbt or a comparable framework.
  • Experience with data modeling for warehouse, lakehouse, and data-product use cases.
  • Experience with distributed data processing using Apache Spark/PySpark or a comparable framework.
  • Experience with Git-based development, automated testing, code review, and CI/CD.
  • Experience with batch and streaming architectures, orchestration, and production operations.
  • Knowledge of cloud computing and object-storage concepts; Microsoft Azure is preferred.
  • Experience working with structured and semi-structured data using formats such as Parquet, JSON, Avro, CSV, and XML.
  • Ability to independently design and deliver complex data solutions spanning ingestion, processing, persistence, quality, and consumption.
  • Ability to troubleshoot complex distributed-processing, data-quality, and production-reliability issues.
  • Ability to balance performance, cost, scalability, resiliency, security, and maintainability in technical decisions.
  • Ability to review and improve technical designs and code developed by other engineers.
  • Ability to communicate complex designs and tradeoffs to engineering, architecture, product, and business audiences.

Nice To Haves

  • Advanced production experience with Snowflake, Databricks, Apache Spark/PySpark, and dbt.
  • Experience implementing Apache Iceberg or another open table format, including schema evolution, partitioning, snapshots, and table maintenance.
  • Experience with Apache Kafka, CDC, and event-driven architectures.
  • Experience with orchestration technologies such as Apache Airflow, Azure Data Factory, Databricks Workflows, or dbt orchestration.
  • Experience with Infrastructure as Code and configuration automation, including Terraform or comparable tooling.
  • Experience with data catalogs, metadata management, lineage, data observability, and automated data-quality frameworks.
  • Experience building governed enterprise data products or domain-oriented data solutions.
  • Experience working in a highly regulated environment such as financial services.
  • Master’s degree in Computer Science, Engineering, Data Engineering, Information Systems, or a related technical field.

Responsibilities

  • Lead detailed engineering design, development, testing, deployment, and production support of complex data pipelines and reusable data products.
  • Develop maintainable data-processing and transformation solutions using advanced SQL and programming languages such as Python.
  • Engineer solutions using Snowflake, Databricks, dbt, and comparable modern data platforms; select warehouse, lakehouse, or hybrid patterns to fit workload needs.
  • Design analytical storage solutions using relational databases, cloud object storage, columnar formats, and open table formats such as Apache Iceberg.
  • Implement batch, micro-batch, change data capture (CDC), event-driven, and streaming patterns using Apache Kafka or comparable platforms.
  • Integrate data from relational and mainframe systems, APIs, event streams, SaaS platforms, files, and object storage with appropriate security and operational controls.
  • Apply normalized, dimensional, denormalized, and data-product-oriented modeling approaches according to access, performance, and lifecycle requirements.
  • Optimize Snowflake, Databricks/Spark, and SQL workloads for performance, scalability, reliability, and cost, including partitioning, parallelism, data movement, and compute utilization.
  • Build data quality, reconciliation, metadata, lineage, observability, alerting, and service-level controls into production solutions.
  • Design for restartability, idempotency, retry, replay, recovery, and failure isolation; lead diagnosis and root-cause analysis for complex production incidents.
  • Apply source control, peer review, automated testing, CI/CD, environment promotion, and infrastructure/configuration automation to data engineering delivery.
  • Create reusable libraries, templates, implementation patterns, and reference solutions that improve consistency and engineering velocity.
  • Evaluate design alternatives and document tradeoffs involving security, privacy, resiliency, maintainability, performance, cost, and operational complexity.
  • Contribute to technical standards, identify technical debt, and recommend pragmatic modernization or remediation approaches.
  • Mentor engineers through design reviews, code reviews, pairing, and technical problem solving; influence engineering practices across the team and adjacent teams.
  • Partner with product owners, architects, governance, cybersecurity, platform, and consuming teams to translate business and data requirements into scalable solutions.
  • Develop technical epics and stories, estimate complex work, surface dependencies and risks, and support iterative delivery through Agile practices.
  • Understand and adhere to Company risk and regulatory standards, policies, and controls in accordance with the Company’s Risk Appetite.
  • Design, implement, maintain, and enhance internal controls to mitigate risk; identify risk-related issues requiring escalation to management.
  • Maintain internal control standards, including timely implementation of internal and external audit points and issues raised by external regulators, as applicable.
  • Promote an environment that supports belonging and reflects the Company brand.
  • Complete other related duties as assigned.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service