Cloud Data Platform Engineer

Northern Trust • Chicago, IL

1d • Hybrid

About The Position

The Cloud Data Platform Engineer is responsible for designing, implementing, operating, and evolving a scalable, secure, reliable and high-performance cloud-based data platform that enables data warehousing, analytics, artificial intelligence capabilities, and business-critical data platform services built on a modern Lakehouse architecture. This role partners with architects, product owners, and application teams to deliver modern data platform capabilities that accelerate innovation while ensuring operational excellence, governance, security, and cost efficiency. The engineer drives automation, platform standardization, and continuous improvement to provide reliable and self-service data services across the organization.

Requirements

• Bachelor’s degree in computer science, Data Engineering, or related field (Master’s preferred).
• 7+ years of experience in data engineering, cloud engineering, or platform engineering.
• Hands-on experience with at least one major cloud provider (Azure, AWS).
• Strong experience with Cloud Data platform like Snowflake and Databricks.
• Proven expertise in Devops engineering including cloud infrastructure resources creation, Data object change management and other IAAC’s.
• Proven expertise in building data pipelines with tools like Azure data factory, Synapse, Apache Airflow, dbt, Kafka, or similar.
• Proficiency in infrastructure-as-code (Github, Copilot, Terraform, yaml, Azure Devops, devSecops, CloudFormation, Pulumi).
• Experience in cloud data platform like Databricks or Snowflake.
• Experience in programming languages (Sql, Python, yaml, Java, or Scala).
• Deep understanding of cloud-native data services (e.g., Azure Synapse, Data Factory, Azure App services, AWS Redshift, Glue, S3, Athena).
• Familiarity with containerization and orchestration (Docker, Kubernetes).
• Proficiency in infrastructure-as-code (Github, Copilot, Terraform, yaml, Azure Devops, devSecops, CloudFormation, Pulumi).
• Knowledge of FinOps principles for cost-efficient data platform design.
• Strong problem-solving, communication, and cross-functional collaboration skills.
• Experience operating cloud platforms within regulated environments with strong governance, audit, security, and compliance requirements preferred.

Responsibilities

• Platform Engineering & Development: Design, implement, operate, and continuously improve cloud-based data platform services and infrastructure supporting enterprise data, analytics, and AI workloads on technologies including Databricks, Snowflake, Azure and AWS.
• Support self-service data platform capabilities by developing standardized onboarding, provisioning, and operational processes for platform consumers.
• Engineer reusable platform services, templates, and automation patterns that accelerate developer productivity and platform adoption.
• Integrate cloud data services with enterprise applications, APIs, and third-party technologies.
• Cloud Infrastructure and Automation: Build and maintain scalable Lakehouse architecture capabilities that support structured, semi-structured, and unstructured data at enterprise scale.
• Implement CI/CD, Infrastructure as Code (IaC), and configuration management practices using enterprise-standard automation and deployment tools including Terraform and automated provisioning.
• Develop and maintain reusable Infrastructure as Code modules, deployment templates, and platform blueprints to ensure standardization and operational consistency.
• Develop platform services, automation, and documentation that enable self-service onboarding, provisioning, governance, and operational management.
• Design and maintain GitHub-based CI/CD pipelines to automate platform provisioning, application deployment, testing, and operational workflows.
• Build reusable frameworks and automation for data workflows, monitoring, and scaling.
• Develop and optimize frameworks that enable data ingestion, transformation, orchestration, processing, and AI-powered workloads across the platform ensuring a reliable and efficient data lifecycle.
• Data Platform Governance, Security, Reliability, Optimization: Continuously optimize Cloud Data platform performance, cost, and scalability, including providing tools to application teams to support them in this process.
• Implement Resiliency solutions including disaster recovery, backup, and failover strategies that meet enterprise standards and take full advantage of automation.
• Establish observability, monitoring, alerting, and operational tooling to ensure platform reliability, availability, and performance.
• Enforce governance, security, and compliance standards (e.g., GDPR, HIPAA, SOC2).
• Implement and maintain secure-by-design platform patterns, access controls, encryption standards, and audit capabilities.
• Work with architects and security teams to implement role-based access, encryption, and audit controls.
• Support metadata management, data discoverability, and enterprise-wide data democratization.
• Collaboration, Innovation, and Continuous Improvement: Evaluate emerging cloud data services and recommend adoption based on enterprise needs.
• Provide technical expertise in designing cloud-based data architectures and solutions.
• Collaborate with architects and engineering teams to define platform standards, design patterns, and technical roadmaps.
• Partner with data scientists, BI teams, and application developers to enable them with advanced analytics and AI/ML workloads.
• Mentor junior engineers and contribute to best practices in cloud engineering and data development.

Benefits

• retirement benefits (401k and pension)
• health and welfare benefits (medical, dental, vision, spending accounts and disability)
• paid time off
• parental and caregiver leave
• life & accident insurance
• other voluntary and well-being benefits
• discretionary bonus program that may include an equity component

What This Job Offers

• Job Type: Full-time
• Career Level: Senior