Senior Data Engineer (AWS, Databricks)

Travelers•Hartford, CT

21h•$139,400 - $230,000

About The Position

Travelers Data Engineering team constructs pipelines that contextualize and provide easy access to data by the entire enterprise. As a Senior Data Engineer you will accelerate growth and transformation of our analytics landscape. You will bring a strong desire to guide team members' growth and develop data solutions that translate complex data into user-friendly terminology. You will leverage your ability to design, build and deploy data solutions that capture, explore, transform, and utilize data to support Artificial Intelligence, Machine Learning and business intelligence/insights.

Requirements

Bachelor’s Degree in STEM related field or equivalent
Ten years of related experience building, designing and operating production data pipelines at scale.
Demonstrable experience architecting, designing and building scalable, secure data solutions using AWS, Databricks, Snowflake and Ab>Initio or similar platforms.
A track record of leveraging AI assistants, create skills/tools to augment data engineering practices throughout the development lifecycle.
The ability to lead technical direction for data engineering initiatives across cloud and on-premises infrastructure.
Willingness to run mentoring sessions and offer technical guidance to the 20-person admin team.
Ability to manage infrastructure deployment and optimize cloud resources.
Drive to learn, identify and set technical standards and influence engineering practices and data governance policies.
Ability to lead and take action even when there is no clear owner, inspire and motivate others, and be effective at influencing team members.
Cloud: Proficiency with commonly used AWS services and architectures for data and analytics solutions
Databricks: Workspace management, cluster configuration, open table formats (Iceberg, Delta Lake), Unity Catalog, building and tuning Spark/pySpark/SQL workloads.
Data Warehousing: Strong understanding of data modeling, dimensional modeling (star/snowflake schemas) and medallion (bronze/silver/gold) architecture.
AI Coding Assistants: Familiarity using Claude Code, Codex, Copilot, Cursor, etc. for day-to-day engineering tasks.
AbInitio: Proficiency with GDE, Co>Operating System, and EME, including maintaining and optimizing existing graphs.
CI/CD for data: Git-based workflows, branching strategy, and pipeline automation (e.g., GitHub Actions, GitLab CI, or Jenkins).
Infrastructure as Code (e.g., Terraform): Familiarity with a configuration-driven, repeatable approach to environments
Bachelor’s degree in computer science, related STEM field, or its equivalent in education and/or work experience.
6 additional years of data engineering experience.

Responsibilities

Design and build production data pipelines across AWS, Snowflake, Databricks supporting both batch and near real-time analytics workloads.
Establish reusable engineering patterns and frameworks - Parameterized, modular, idempotent pipeline templates that other engineers adopt, reducing duplicated effort and inconsistent implementations.
Drive down lead time from commit to production by removing manual steps, leveraging AI, building self-service tooling, and standardizing the path to deployment; treat cycle time as a metric you actively own and improve.
Champion SDLC discipline covering version control, peer code review, automated testing, environment promotion, change management, and documentation.
Integrate AI coding tools into daily workflow to accelerate scaffolding, refactoring, test generation, code optimization, and documentation, with measurable and demonstrable impact on throughput and quality.
Measure and demonstrate impact , tying AI-tool adoption to concrete outcomes such as reduced lead time, faster test coverage, and improved consistency, and sharing those results to drive broader adoption.
Evaluate emerging tooling and make pragmatic recommendations on what engineers should adopt, standardize on, or avoid.
DataOps - Blur the lines between data and software engineering practices. Employ CI/CD, automated testing, and apply trunk-based or short-lived branch development to data the same way it is to software.
Modernize legacy workloads - Help manage and optimize Ab>Initio pipelines and when applicable, help migrate or re-platform Ab>Initio pipelines toward cloud-native, declarative, ELT-based patterns on Snowflake and Databricks where it delivers value.
Embed data quality, observability, and lineage into pipelines as a default, not an afterthought, automated data tests, freshness/quality SLAs, and traceable lineage.
Optimize for cost and performance across Snowflake compute, Databricks clusters, and storage, applying FinOps-aware engineering practices.
Mentor and upskill engineers through code review, pairing, design guidance, and documented standards, acting as a technical multiplier for the team.