Data Engineer
About The Position
Design, develop, and maintain scalable data pipelines, ingestion frameworks, transformation processes, and reusable data products using Azure Databricks, PySpark, SQL, and Delta Lake. Implement Bronze, Silver, and Gold architecture patterns supporting enterprise reporting, analytics, AI, and self-service data consumption. Build reusable frameworks, utilities, and platform components that improve engineering productivity, quality, consistency, and deployment speed. Develop and support batch, near-real-time, and streaming integration solutions, including modernization of legacy warehouse and ETL workloads. Serve as technical lead for complex initiatives; develop solution designs, lead technical reviews, recommend tools and approaches, and guide work through production implementation. Partner with data architects and platform leaders to ensure solutions are scalable, secure, governed, cost-conscious, and operationally supportable. Mentor engineers and promote standards for coding, testing, documentation, performance, and production readiness. Implement data quality, validation, reconciliation, monitoring, metadata, and lineage capabilities; support RBAC and enterprise security controls. Build and maintain CI/CD, automated testing, deployment, and release processes using Azure DevOps and Git-based practices. Contribute to platform observability, alerting, operational dashboards, health metrics, performance tuning, and cost optimization. Participate in production support, incident response, pager, and on-call rotations; troubleshoot issues, lead root cause analysis, and implement durable remediation.
Requirements
- Azure Databricks
- PySpark
- SQL
- Delta Lake
- Azure DevOps
- Git-based practices
Responsibilities
- Design, develop, and maintain scalable data pipelines, ingestion frameworks, transformation processes, and reusable data products using Azure Databricks, PySpark, SQL, and Delta Lake.
- Implement Bronze, Silver, and Gold architecture patterns supporting enterprise reporting, analytics, AI, and self-service data consumption.
- Build reusable frameworks, utilities, and platform components that improve engineering productivity, quality, consistency, and deployment speed.
- Develop and support batch, near-real-time, and streaming integration solutions, including modernization of legacy warehouse and ETL workloads.
- Serve as technical lead for complex initiatives; develop solution designs, lead technical reviews, recommend tools and approaches, and guide work through production implementation.
- Partner with data architects and platform leaders to ensure solutions are scalable, secure, governed, cost-conscious, and operationally supportable.
- Mentor engineers and promote standards for coding, testing, documentation, performance, and production readiness.
- Implement data quality, validation, reconciliation, monitoring, metadata, and lineage capabilities; support RBAC and enterprise security controls.
- Build and maintain CI/CD, automated testing, deployment, and release processes using Azure DevOps and Git-based practices.
- Contribute to platform observability, alerting, operational dashboards, health metrics, performance tuning, and cost optimization.
- Participate in production support, incident response, pager, and on-call rotations; troubleshoot issues, lead root cause analysis, and implement durable remediation.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
