Associate Director, Data Engineering

S&P Global • Princeton, NJ

128d • $127,000 - $185,000

About The Position

S&P Dow Jones Indices is seeking a Sr. Lead Data Engineer with expertise in Databricks/AWS EMR and distributed big data processing technologies to be a key player in the implementation and support of data platforms for S&P Dow Jones Indices. This role requires a seasoned technologist who contributes to application development and maintenance. The candidate should actively evaluate new products and technologies to build solutions that streamline business operations. The candidate must be delivery-focused with solid financial applications experience. The candidate will assist in day-to-day support and operations functions, design, development, and unit testing.

Requirements

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or equivalent work experience.
• 10+ years of IT experience in application support or development.
• Design & build ingestion pipelines using ETL/ELT and schema-on-read in data lake technologies.
• Experience in designing transactional systems, data warehouses, data lakes, and data integrations within a big data ecosystem, particularly leveraging cloud technologies.
• Provide technical expertise in the design and implementation of data ingestion pipelines with modern AWS cloud and other technologies such as S3, Hive, Databricks, Scala, Python, and large-scale data analytics tools.
• Strong experience in managing distributed data processing workloads, including configurations.
• Experience in working with multi-threaded, high-performance, low-latency messaging systems.
• Experience in AWS cloud-based technologies.
• Experience using system tools, source control systems, utilities, and third-party products.
• Excellent communication skills, with strong verbal and writing proficiencies.

Nice To Haves

• Proficiency in building data analytics solutions on AWS Cloud.
• Thorough understanding of replication strategies and implementation of high-availability protocols for 99.99% uptime.
• Experience with implementation of data governance and data lineage.
• Proficiency in implementing data models in SQL and NoSQL databases.
• Experience with financial applications such as Index/Benchmarks, Asset Management, Portfolio Investment modeling, or trading systems.
• Experience with microservice and serverless architecture implementation.

Responsibilities

• Lead the design and implementation of distributed data processing workloads using Python, including data access from relational databases and cloud storage technologies.
• Ensure performance, security, and reliability; coordinate with application teams to define database design.
• Collaborate with cross-functional teams to support data-driven initiatives.
• Mentor junior team members and promote best practices.
• Oversee maintenance and troubleshooting of distributed data processing systems.
• Drive innovation by evaluating and integrating new technologies.
• Produce system design documents and participate in technical walkthroughs.
• Perform application and system performance tuning and troubleshoot performance issues.
• Effectively interact with global customers, business users, and IT staff.

Benefits

• Health care coverage designed for the mind and body.
• Generous time off helps keep you energized for your time on.
• Access a wealth of resources to grow your career and learn valuable new skills.
• Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs.
• S&P Global has perks for your partners and little ones, too, with some best-in-class benefits for families.
• From retail discounts to referral incentive awards—small perks can make a big difference.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Bachelor's degree
• Number of Employees: 5,001-10,000 employees