Lead, Arch Solutions

Northern Trust • Chicago, IL

1d • Remote

About The Position

We are seeking an experienced Azure Data Engineer to design, build, and operate secure, scalable, and reliable cloud data solutions across Corporate Services. As a key member of the Corporate Architecture team, this role will translate business requirements into production-ready data products that support analytics, reporting, and advanced data use cases. The successful candidate will combine strong software engineering and data architecture skills with hands-on expertise in Azure Data Factory, Azure Databricks, Azure Data Lake Storage Gen2, Spark, Python, and SQL. The role will contribute to the modernization of legacy data platforms through reusable engineering patterns, automated delivery, strong data governance, and operational excellence.

Requirements

• Proven expertise in designing, developing, and supporting enterprise-grade data pipelines and analytics platforms in Azure.
• Strong hands-on experience with Azure Data Factory, Azure Databricks, Azure Data Lake Storage Gen2, and Delta Lake.
• Advanced proficiency in Python, PySpark , and SQL, including performance tuning and development of reusable data-processing components.
• Strong understanding of ETL and ELT patterns, distributed data processing, orchestration, incremental loading, change data capture, and schema evolution.
• Experience building relational, dimensional, and lakehouse data models, including star schemas and conformed dimensions.
• Hands-on experience with relational, analytical, columnar, and NoSQL technologies such as Oracle, PostgreSQL, Azure SQL, Snowflake, or equivalent platforms.
• Experience with Git-based development, automated testing, and CI/CD using Azure DevOps, GitHub Actions, Jenkins, or equivalent tools.
• Familiarity with infrastructure-as-code and deployment automation using Terraform, Bicep, ARM templates, or similar technologies.
• Experience working in Agile delivery environments and participating in estimation, design reviews, code reviews, release planning, and production support.

Responsibilities

• Partner with business, architecture, analytics, security, and engineering stakeholders to translate business needs into clear functional and non-functional data requirements.
• Design and implement scalable batch and near-real-time data pipelines that ingest, transform, validate, and publish structured and unstructured data.
• Build cloud-native Azure data solutions using Azure Data Factory, Azure Databricks, Azure Data Lake Storage Gen2, Spark, Python, SQL, and related services.
• Develop lakehouse and data warehouse solutions using layered data patterns such as Bronze, Silver, and Gold, with appropriate dimensional, normalized, or Data Vault modeling techniques.
• Evangelize modern data modeling and data quality practices
• Develop reusable, metadata-driven frameworks and parameterized pipelines that reduce duplication and improve delivery consistency.
• Build automated unit, integration, regression, and data-validation tests in partnership with quality-engineering teams.
• Implement CI/CD and infrastructure-as-code practices for repeatable deployment across development, test, and production environments.

Benefits

• retirement benefits (401k and pension)
• health and welfare benefits (medical, dental, vision, spending accounts and disability)
• paid time off
• parental and caregiver leave
• life & accident insurance
• other voluntary and well-being benefits
• discretionary bonus program that may include an equity component

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed