Head of Engineering - Data Distribution

S&P Global•Washington DC, NY

7d•Remote

About The Position

This team is essential to the division's data strategy, playing a key role in data transformation and leading efforts to replace legacy technical debt. The team embodies a strong commitment to delivering on business needs and demonstrates exceptional collaboration with both product and business partners. They are known for their collaborative approach, strong technical expertise, and the high energy and dedication they bring to their work. This role will lead and scale a large, multi-disciplinary engineering organization of 50+ data engineers, ML engineers, and platform developers, including managing managers and staff-level engineers across cross-functional teams. The position will drive the strategic architecture and implementation of enterprise data management and governance, integrating Databricks Unity Catalog with enterprise data catalog systems and AWS-native governance services to ensure automated data lineage, security, and accuracy at petabyte scale. The role will oversee the development of production-grade autonomous AI systems using Databricks Mosaic AI, ensuring AI agents are grounded in highly governed, accurate data through robust semantic modeling and RAG architectures. Additionally, the Head of Engineering will guide the technical strategy for zero-copy, multi-platform lakehouse architecture leveraging Delta Lake and Apache Iceberg, with deep integration across AWS cloud data and analytics services including S3, Glue, and Lake Formation. The role involves partnering with Product, Security, and executive leadership to translate business and compliance requirements into actionable engineering roadmaps while fostering a high-performance culture of innovation and engineering excellence. This position will also champion organizational transformation initiatives that position the company at the forefront of enterprise AI and data platform capabilities, directly impacting business growth and operational excellence.

Requirements

15+ years of engineering experience with 5+ years directly managing and scaling large engineering teams at Director or Head of Engineering level, including experience managing managers and Staff/Principal-level individual contributors
Deep technical expertise in data platform architecture including Delta Lake and Apache Iceberg internals, with proven ability to guide senior engineers on complex architectural trade-offs and design decisions
Demonstrated experience leading enterprise data management initiatives including data lineage automation, schema management, data governance, and security implementations at petabyte scale across distributed platforms
Hands-on experience with cloud data platforms such as AWS services (S3, Glue, Lake Formation, EMR), Databricks Data Intelligence Platform, or similar enterprise-scale data infrastructure technologies
Proven track record integrating operational data catalogs with enterprise metadata platforms, implementing complex access controls (RBAC/CBAC/ABAC), and managing data security across multi-cloud environments
Strong leadership and communication skills with experience driving organizational transformation, building high-performance engineering cultures, and partnering effectively with cross-functional teams including Product, Security, and executive leadership

Nice To Haves

Experience with AI/ML platforms such as Databricks Mosaic AI, Amazon SageMaker, or Amazon Bedrock, with understanding of autonomous AI agent deployment and semantic modeling for LLM grounding
AWS certifications such as Solutions Architect, Data Analytics, or similar cloud credentials, along with experience in multi-account governance, multi-region deployment strategies, and cost optimization for data-intensive workloads
Background in enterprise data governance frameworks, compliance requirements, and experience with tools like data.world for enterprise knowledge graphs and metadata management
Experience partnering with FinOps and cloud platform teams to optimize cloud consumption and platform ROI, with familiarity in multi-cloud data platform strategies spanning various public cloud ecosystems

Responsibilities

Lead and scale a large, multi-disciplinary engineering organization of 50+ data engineers, ML engineers, and platform developers, including managing managers and staff-level engineers across cross-functional teams
Drive the strategic architecture and implementation of enterprise data management and governance, integrating Databricks Unity Catalog with enterprise data catalog systems and AWS-native governance services to ensure automated data lineage, security, and accuracy at petabyte scale
Oversee the development of production-grade autonomous AI systems using Databricks Mosaic AI, ensuring AI agents are grounded in highly governed, accurate data through robust semantic modeling and RAG architectures
Guide the technical strategy for zero-copy, multi-platform lakehouse architecture leveraging Delta Lake and Apache Iceberg, with deep integration across AWS cloud data and analytics services including S3, Glue, and Lake Formation
Partner with Product, Security, and executive leadership to translate business and compliance requirements into actionable engineering roadmaps while fostering a high-performance culture of innovation and engineering excellence
Champion organizational transformation initiatives that position the company at the forefront of enterprise AI and data platform capabilities, directly impacting business growth and operational excellence

Benefits

Annual incentive plan
Health care coverage designed for the mind and body
Generous time off
Access to a wealth of resources to grow your career and learn valuable new skills
Competitive pay
Retirement planning
Continuing education program with a company-matched student loan contribution
Financial wellness programs
Perks for partners and little ones
Retail discounts
Referral incentive awards