Data Engineer

EXL•United States,

2h•$85,000 - $140,000•Remote

About The Position

We are seeking an experienced and visionary Senior Full-Stack Data Engineer to lead the architecture, development, and optimization of a next-generation data platform. This is a critical role for an individual with over 10 years of deep data engineering expertise, capable of driving technical direction, mentoring team members, and delivering high-impact solutions in a fast-paced project environment.

Requirements

10+ Years of hands-on, progressive experience in Data Engineering, Data Architecture, or a closely related Full-Stack Data role
Deep conceptual understanding of core data engineering principles, ETL/ELT patterns, and metadata management
Proven track record of building and managing petabyte-scale data infrastructure in a cloud-native environment
Experience with AWS (S3, IAM, VPC, etc.)
Experience with Talend, dbt Core, Iceberg, AWS Glue Catalog, Snowflake, Redshift, Athena, Splunk, AWS streaming services, Git
Strong SQL, Pyspark and Python

Nice To Haves

Insurance industry experience preferred but not mandatory

Responsibilities

Define and champion the architectural roadmap and best practices for our end-to-end data pipelines, ensuring scalability, reliability, and security across the platform.
Act as a primary technical mentor, guiding a team of engineers, conducting code reviews, and aggressively driving the project timeline to ensure rapid delivery of data products.
Partner with Data Scientists, Analysts, and business stakeholders to translate complex requirements into robust, production-ready data solutions.
Collaborate with Data Scientists and ML Engineers for Data Accessibility, Support for Model Development, and Data Quality Assurance.
Design, build, and optimize high-volume data ingestion and transformation jobs using tools like dbt Core, AWS Glue, ensuring data quality and integrity.
Develop and maintain sophisticated data pipelines using orchestrators such as Dagster, focusing on modularity and reusability.
Implement and manage real-time data flows utilizing Confluent platforms or native AWS streaming services (e.g., Kinesis) for immediate data availability.
Ensure Data Security and Privacy through Data Anonymization and Compliance with Regulations.
Be well versed with DataOps and DevOps fundamentals.
Assist and drive the Data Ecosystem Management & Monitoring.
Implement and maintain the Iceberg open table format, utilizing tools for efficient schema evolution and data management.
Optimize query performance and cost efficiency across our primary compute engines: Snowflake, Amazon Redshift, and AWS Athena.
Integrate comprehensive monitoring and observability into all pipelines using Splunk to ensure high availability, rapidly identify bottlenecks, and troubleshoot production issues.