Associate - Data Engineer

New York Life•Lebanon, NJ

4d•$81,000 - $115,500•Hybrid

About The Position

Join New York Life’s Data Engineering team and help deliver clean, well-structured, and reliable data that powers business, analytics, and AI solutions across the enterprise. In this role, you will design and maintain scalable data pipelines, contribute to data and engineering standards, and partner with data scientists, technology teams, and business stakeholders to make enterprise data more accessible and valuable. You will apply modern data engineering practices and technologies, including dbt, PySpark, AWS Glue, Databricks, and Kafka, while helping ensure data solutions meet expectations for quality, consistency, governance, and scalability.

Requirements

2–5+ years of relevant data engineering or related experience, including hands-on experience building and maintaining dbt data models and transformation pipelines.
Proficiency with AWS Glue, including ETL development, job orchestration, and catalog integration, and experience using PySpark for data processing.
Strong working knowledge of Databricks, including SQL, notebooks, and distributed data processing.
Experience with Kafka or comparable streaming technologies for real-time data ingestion and processing.
Strong analytical and problem-solving skills, with the ability to troubleshoot complex data and engineering issues and develop practical solutions.
Effective communication and collaboration skills, with the ability to work across technical and business teams and explain technical concepts to varied audiences.

Nice To Haves

Experience with Google Cloud Platform (GCP) or related cloud data technologies.
Experience working in the insurance, financial services, or another regulated industry.
Experience partnering with data science teams to support analytics, machine learning, or AI use cases.

Responsibilities

Build and maintain scalable data models and pipelines using dbt, PySpark, AWS Glue, Databricks, and streaming technologies to deliver clean, reliable, and well-structured data for analytics, data science, and AI use cases.
Apply software engineering best practices across data development, including version control, testing, integration, code reviews, and pull request approvals, to promote quality and consistency.
Partner with data scientists, business stakeholders, and data stewards to source strategic enterprise data, address data quality and availability needs, and align solutions with data governance standards.
Translate business and technical requirements into actionable development plans and Jira stories, establish milestones, and manage work through delivery.
Collaborate with Technology teams on data strategy, tooling, system integrations, and deployment processes while clearly communicating technical concepts and recommendations to diverse stakeholders.