Lead Data Engineer (Banking)
About The Position
ITTConnect is seeking a Lead Data Engineer for a direct-hire, full-time position with a large financial institution. The role is hybrid in Miami. The Lead Data Engineer turns business and domain needs into secure, observable, reusable, and documented pipelines. This position combines hands-on technical leadership, portfolio delivery, consultant coordination, engineering standards, and stakeholder engagement in a regulated bank. The role owns integrated delivery across 4–6 concurrent projects and coordinates an 8–12 person consultant team. It guides on-premises platform execution with NiFi, Airflow, dbt, and SQL, while leading cloud-native pipeline delivery and phased re-engineering using AWS and Databricks. The position partners with business, data, architecture, security, infrastructure, risk, compliance, and operations stakeholders.
Requirements
- 15+ years of experience in IT.
- 8+ years in data engineering, platform delivery, analytics engineering, or a related field.
- 3+ years in technical or delivery leadership.
- Experience coordinating parallel projects and distributed or consulting teams in complex enterprises.
- Strong SQL expertise and practical experience with NiFi, Airflow, dbt, or comparable tools.
- Experience building production pipelines with automated tests, quality controls, monitoring, lineage, documentation, and support.
- Working knowledge of AWS and Databricks architecture, lakehouse patterns, security, deployment, and migration.
- Experience in regulated financial services or another control-intensive environment.
- Strong English communication, planning, problem-solving, and stakeholder-management skills.
Nice To Haves
- Fluency in Portuguese or Spanish.
- Experience with Unity Catalog, Delta Lake, Databricks Workflows, CI/CD, infrastructure as code, and cloud observability.
- Knowledge of AWS data-platform services, including S3, IAM, KMS, CloudWatch, and private networking.
- Experience with catalogs, metadata, lineage, data quality, and DataOps or DataSecOps.
- Familiarity with dimensional modeling, data vault, medallion/lakehouse, streaming, and data products.
- Relevant AWS, Databricks, data engineering, or agile delivery certifications.
Responsibilities
- Own integrated delivery for 4–6 concurrent projects, balancing priorities, dependencies, capacity, risks, and production commitments within a scaled agile framework.
- Translate transformation needs into technical scope, milestones, acceptance criteria, and measurable outcomes.
- Provide portfolio visibility using dashboards, risk logs, dependency tracking, and escalation.
- Ensure releases meet change, security, testing, documentation, and production-readiness requirements.
- Direct an 8–12 person consultant team through work allocation, onboarding, delivery follow-up, and quality review.
- Establish a delivery rhythm with backlog refinement, design and code reviews, stand-ups, demos, and retrospectives.
- Build team capability through reusable patterns, playbooks, pairing, coaching, and knowledge transfer.
- Partner with vendor and bank leaders to manage capacity, capability, performance, continuity, and key-person risks.
- Guide solutions using NiFi for ingestion, Airflow for orchestration, dbt for transformation and testing, and SQL for processing and validation.
- Improve pipeline stability through modular design, automated tests, version control, disciplined deployment, observability, and incident follow-up.
- Prioritize technical debt based on business criticality, operational risk, migration value, and delivery impact.
- Follow the AWS and Databricks operating model in accordance with architecture, security, governance, networking, and compliance standards.
- Lead cloud-native pipeline delivery and phased re-engineering of on-premises pipelines.
- Redesign and remodel pipelines rather than simply lift and shift when doing so improves scale, reliability, cost transparency, reuse, lineage, or supportability.
- Define migration waves, readiness, parallel runs, reconciliation, cutover, rollback, and decommissioning.
- Align delivery with business and data owners, stewards, architects, security, infrastructure, risk, compliance, and operations.
- Lead cross-domain requirements and design discussions, resolving ambiguity and clarifying trade-offs.
- Explain decisions, delivery health, risks, and business impact to technical and executive audiences.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
