Lead Data Engineer (Banking)
About The Position
ITTConnect is seeking a Lead Data Engineer for a direct-hire, full-time position with a large financial institution. The role is hybrid in Miami. The Lead Data Engineer turns business and domain needs into secure, observable, reusable, and documented pipelines. It combines hands-on technical leadership, portfolio delivery, consultant coordination, engineering standards, and stakeholder engagement in a regulated bank. The position owns delivery across multiple concurrent projects, coordinates a consultant team, guides on-premises data-platform execution, and leads cloud transition and pipeline re-engineering. It also partners with business and data owners and technical control functions to align delivery with domain needs, architecture, security, governance, and compliance standards.
Requirements
- 15+ years of experience in IT.
- 8+ years in data engineering, platform delivery, analytics engineering, or a related field.
- 3+ years in technical or delivery leadership.
- Experience coordinating parallel projects and distributed or consulting teams in complex enterprises.
- Strong SQL expertise and practical experience with NiFi, Airflow, dbt, or comparable tools.
- Experience building production pipelines with automated tests, quality controls, monitoring, lineage, documentation, and support.
- Working knowledge of AWS and Databricks architecture, lakehouse patterns, security, deployment, and migration.
- Experience in regulated financial services or another control-intensive environment.
- Strong English communication, planning, problem-solving, and stakeholder management skills.
Nice To Haves
- Fluency in Portuguese or Spanish.
- Experience with Unity Catalog, Delta Lake, Databricks Workflows, CI/CD, infrastructure as code, and cloud observability.
- Knowledge of AWS data-platform services, including S3, IAM, KMS, CloudWatch, and private networking.
- Experience with catalogs, metadata, lineage, data quality, and DataOps or DataSecOps.
- Familiarity with dimensional modeling, data vault, medallion/lakehouse, streaming, and data products.
- Relevant AWS, Databricks, data engineering, or agile delivery certifications.
Responsibilities
- Own integrated delivery for 4–6 concurrent projects, balancing priorities, dependencies, capacity, risks, and production commitments within a scaled agile framework.
- Translate transformation needs into technical scope, milestones, acceptance criteria, and measurable outcomes.
- Provide portfolio visibility using dashboards, risk logs, dependency tracking, and escalation.
- Ensure releases meet change, security, testing, documentation, and production-readiness requirements.
- Direct an 8–12 person consultant team through work allocation, onboarding, delivery follow-up, and quality review.
- Establish a delivery rhythm that includes backlog refinement, design and code reviews, stand-ups, demos, and retrospectives.
- Build team capability through reusable patterns, playbooks, pairing, coaching, and knowledge transfer.
- Partner with vendor and bank leaders to manage capacity, capability, performance, continuity, and key-person risks.
- Guide solutions using NiFi for ingestion, Airflow for orchestration, dbt for transformation and testing, and SQL for processing and validation.
- Improve pipeline stability through modular design, automated tests, version control, disciplined deployment, observability, and incident follow-up.
- Prioritize technical debt based on business criticality, operational risk, migration value, and delivery impact.
- Follow the AWS and Databricks operating model in line with architecture, security, governance, networking, and compliance standards.
- Lead cloud-native pipeline delivery and phased re-engineering of on-premises pipelines.
- Redesign and remodel rather than lift and shift when doing so improves scale, reliability, cost transparency, reuse, lineage, or supportability.
- Define migration waves, readiness, parallel runs, reconciliation, cutover, rollback, and decommissioning.
- Align delivery with business and data owners, stewards, architects, security, infrastructure, risk, compliance, and operations.
- Lead cross-domain requirements and design discussions, resolving ambiguity and clarifying trade-offs.
- Explain decisions, delivery health, risks, and business impact to technical and executive audiences.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
