Head of Engineering - Inbound Data

S&P Global•Washington, DC

1d•$185,000 - $265,000

About The Position

The Head of Engineering - Inbound Data leads and scales a world-class organization of more than 100 data engineers, ML engineers, and platform developers. The role sets strategic technical direction for enterprise-scale lakehouse architecture using the Databricks Data Intelligence Platform and AWS, while advancing the organization’s transition from AI proofs of concept to production-grade autonomous AI systems that deliver measurable business value. The position establishes engineering standards and operational excellence practices, oversees enterprise semantic modeling and retrieval-augmented generation architecture, and ensures AI systems use governed data with appropriate access controls, lineage, and auditability. It also builds a high-performance culture based on psychological safety, continuous learning, and experimentation, partnering with Product, FinOps, and executive leadership to turn business goals into technical roadmaps.

Requirements

15+ years of engineering experience.
5+ years directly managing and scaling large engineering teams at Director or Head of Engineering level, including managing managers and guiding Staff/Principal-level individual contributors through organizational transformation.
Deep expertise in the Databricks Data Intelligence Platform, including Unity Catalog, Delta Lake internals, Delta Live Tables, Databricks Workflows, MLflow, and Mosaic AI.
Extensive experience architecting enterprise-scale data platforms on AWS, including Amazon S3, AWS Glue, Amazon EMR, and AWS Lake Formation.
Strong understanding of multi-region deployment strategies and security patterns, including IAM, VPC, and KMS.
Technical depth in Apache Iceberg internals, schema evolution, and cross-platform table strategies.
Hands-on experience deploying autonomous AI agents and architecting RAG pipelines for production environments.
Demonstrated ability to build and scale engineering organizations from 20+ to 100+ engineers while establishing engineering standards, agile delivery practices, and operational SLAs for complex data and AI products.
Strong leadership and communication skills, including translating technical trade-offs for non-technical stakeholders and influencing executive decisions.

Nice To Haves

AWS and/or Databricks certifications.
Experience partnering with FinOps and cloud platform teams to optimize cloud consumption, performance, and platform ROI across multi-cloud data platform strategies.
Experience with semantic modeling, knowledge graphs, and enterprise data governance frameworks, including metadata management and secure data sharing at enterprise scale.
Background in LLMOps, model evaluation, and prompt orchestration.
Familiarity integrating Amazon SageMaker and Amazon Bedrock into broader enterprise AI platform strategies.
Experience with Terraform, CI/CD pipelines, deployment automation, and Databricks Asset Bundles for repeatable, auditable delivery across distributed cloud environments.

Responsibilities

Lead and scale an organization of 100+ data engineers, ML engineers, and platform developers, including managers and Staff/Principal-level engineers.
Drive hiring, retention, and talent development strategies.
Set strategic technical direction for enterprise-scale lakehouse architecture using Databricks and AWS, ensuring performant, cost-effective, interoperable data foundations.
Lead the transition to production-grade autonomous AI systems using Databricks AI capabilities and deliver measurable business value.
Establish engineering standards and operational practices, including SLOs, error budgets, incident management, and infrastructure-as-code.
Oversee enterprise semantic modeling and RAG architecture, ensuring AI systems use Unity Catalog-governed data with appropriate access controls, lineage, and auditability.
Foster a high-performance engineering culture rooted in psychological safety, continuous learning, and rapid experimentation.
Partner with Product, FinOps, and executive leadership to translate business goals into technical roadmaps.
Translate complex technical trade-offs for non-technical stakeholders and influence executive decisions through data-driven recommendations and ROI analysis.