Senior Associate, AWS Cloud Platform Engineer

New York Life Insurance Co•Lebanon, NJ

9d•Hybrid

About The Position

This role is responsible for designing, developing, and maintaining cloud-based data platform infrastructure using Amazon Web Services (AWS), Databricks, DBT, Spark, and Airflow. The engineer will implement and manage AWS cloud infrastructure to support data integration, analytics, reporting, and governance workflows. Key aspects of the role include building platform features with engineering best practices, utilizing Infrastructure as Code (IaC) and CI/CD for reliable deployments, ensuring platform security and compliance in a regulated AWS environment, and monitoring/tuning performance with robust logging and alerting. The position also involves using Databricks for data processing and ML workflows, integrating with AWS for self-service analytics, and employing Kubernetes for container orchestration. Cloud cost optimization techniques are expected, along with close collaboration with various technical teams and stakeholders. The role includes participation in quarterly planning and rotating Level 3 on-call support.

Requirements

Master's degree in Computer Engineering, Computer Science, Information Technology Management or related field (or foreign education equivalent) plus three (3) years of experience as an AWS Cloud Platform Engineer or related role building and automating secure, scalable cloud data platforms in the finance or insurance industry.
Alternatively, a bachelor's degree in Computer Engineering, Computer Science, Information Technology Management or related field (or foreign education equivalent) plus five (5) years of experience as an AWS Cloud Platform Engineer or related role building and automating secure, scalable cloud data platforms in the finance or insurance industry.
Engineering data lake, lakehouse, data warehouse, or data mesh environments on AWS using secure and scalable design patterns.
Implementing platform engineering with reusable infrastructure, automation, IaC, and CI/CD for data platforms.
Developing and managing Databricks solutions for data engineering, Machine Learning (ML) workflows, and AWS integrations (S3, Glue, and Redshift).
Utilizing AWS native tools and serverless technologies, including IAM, KMS, Lake Formation, Lambda, CloudFormation, and Step Functions to develop data platforms.
Applying cloud security best practices within multi-account environments, including access control, encryption, and auditability.
Containerizing and orchestrating applications using Kubernetes in cloud-native deployments.

Responsibilities

Design, develop, and maintain cloud-based data platform infrastructure using Amazon Web Services (AWS), Databricks, DBT, Spark, and Airflow.
Implement and manage AWS cloud infrastructure to support data integration, analytics, reporting, and governance workflows.
Build platform features in collaboration with cross-functional teams, applying engineering best practices such as reusable components, automation pipelines, and standardized environments.
Use Infrastructure as Code (IaC) and Continuous Integration and Continuous Deployment (CI/CD) to ensure reliable and maintainable deployments.
Ensure platform security and compliance in a multi-account, regulated AWS environment.
Monitor and tune performance with robust logging and alerting.
Use Databricks for data processing, ML workflows, and lakehouse architecture integration with AWS for self-service analytics.
Use Kubernetes for container orchestration and environment consistency.
Apply cloud cost optimization techniques.
Work closely with Data Architects, Engineers, DBAs, and stakeholders to implement scalable solutions.
Participate in quarterly planning sessions.
Provide rotating Level 3 on-call support to ensure platform reliability and resilience.