Director, Data Integration & Workflows

S&P Global•New York, NY

3d•Hybrid

About The Position

The Director of Data Integration & Workflows leads a team responsible for building, templating, and standardizing data pipelines and workflow patterns that power S&P Dow Jones Indices (SPDJI) index and analytical solutions. This role is crucial for the Data Platform enablement strategy, aiming to accelerate delivery, reduce production risk, and ensure scalable, observable, and maintainable pipelines and workflows. The position involves strategic leadership, technical vision, delivery through enablement, quality assurance, operational excellence, and team development.

Requirements

Bachelor's degree in Computer Science, Engineering, Information Systems, or related field; Master's degree preferred.
10+ years of experience in data engineering, data integration, or related technical roles.
5+ years of leadership experience managing technical teams and delivering complex data solutions.
Proven track record of building and scaling production data platforms in complex enterprise environments.
Deep expertise in ETL/ELT design patterns, data pipeline architecture, and workflow orchestration frameworks.
Strong knowledge of both batch and streaming data processing technologies.
Experience with modern data platforms, cloud infrastructure (AWS, Azure, or GCP), and data engineering tools.
Proficiency in programming languages commonly used in data engineering (Python, SQL, Scala, etc.).
Understanding of data quality frameworks, observability, monitoring, and alerting systems.
Knowledge of DevOps practices, CI/CD pipelines, and infrastructure-as-code.
Familiarity with data governance principles and compliance frameworks.
Strong leadership and team management skills with ability to inspire, mentor, and develop technical talent.
Excellent collaboration skills with ability to partner effectively across technical and business teams.
Outstanding communication abilities, able to articulate complex technical concepts to non-technical audiences and translate business requirements into technical solutions.
Strategic thinking with ability to align team activities with business objectives and drive innovation.
Problem-solving mindset with focus on root-cause analysis and sustainable solutions.
Experience managing stakeholder expectations and delivering on commitments in a matrixed organization.
Right to work in the United States.
Must attend at least one in-person interview.

Nice To Haves

Experience in financial services or index management preferred.
Experience across the breadth of index concepts and asset classes.
Familiarity with SPDJI products, methodologies, or index calculation processes.
Expertise in Python or SQL Development.
Experience with Agile/Scrum methodologies and product-oriented delivery models.
Knowledge of data lineage, metadata management, and data cataloging.

Responsibilities

Define and drive the data automation and transformation strategy for the Data Integration & Workflows group, setting standards for production-ready pipelines and workflow automation.
Establish reference architectures and engineering standards for ETL/ELT, orchestration, error handling, observability, and performance/cost optimization.
Foster a culture of technical excellence, craftsmanship, reusable component development, and continuous improvement in automation maturity.
Lead delivery through enablement by assessing SME technical capability and selecting the right engagement model.
Partner with value stream SMEs to co-develop and review pipelines, adapting support based on SME technical capability and fostering their growth.
Oversee the design and implementation of robust, reusable data integration and workflow patterns for both batch and streaming use cases.
Run the code review and quality gate process for SME-built pipelines, ensuring consistency in maintainability, testing, logging, data validation, and documentation prior to IT handover.
Ensure all solutions are production-ready with comprehensive documentation, testing, error handling, and operational monitoring.
Coordinate seamless IT handover and production gateway readiness, ensuring complete deployment packages.
Provide L3 support for production business-logic issues, driving root-cause analysis and permanent fixes.
Ensure implementation of strong data reliability controls including validation rules, reconciliation checks, anomaly detection, and completeness/timeliness controls.
Drive performance and cost optimization through appropriate partitioning, caching, incremental processing patterns, and compute usage tuning.
Establish and monitor operational metrics to track solution delivery timeliness, pipeline reliability, and platform performance.
Lead, mentor, and develop a high-performing team of Data Integration Leads and Experts.
Build data engineering capability across the organization through structured mentorship, knowledge sharing, and hands-on coaching.
Collaborate effectively with other pillars within Data AI & Enablement (AI Solutions and Data Governance).
Foster strong partnerships with PPD teams, IT, Data Value Streams, and Data Services & Strategy.
Own the technical strategy, standards, and execution approach for all data pipeline and workflow automation initiatives.
Responsible for defining and enforcing the "definition of done" for production-ready data solutions.
Own the capability development strategy for value stream SMEs in data integration and automation.
Maintain the quality assurance framework for all data pipelines prior to production deployment.

Benefits

Health care coverage designed for the mind and body.
Generous time off.
Access to a wealth of resources to grow your career and learn valuable new skills.
Competitive pay.
Retirement planning.
Continuing education program with a company-matched student loan contribution.
Financial wellness programs.
Perks for partners and children.
Retail discounts.
Referral incentive awards.