Lead AWS Data Engineer

Capgemini•Atlanta, GA
•$89,000 - $120,000

About The Position

As the Technical Lead - Data Engineering, you will architect, lead, and drive the technical execution of a high-throughput, near real-time data caching layer supporting Operational Data Store (ODS) views. You will lead the engineering team responsible for ingesting high-frequency account update events via Kafka/CDC, orchestrating cache refreshes, and designing high-performance PostgreSQL persistence architectures on AWS to guarantee sub-second lookups and consistency for downstream consumers. The role requires 8+ years in Data Engineering, including at least 2+ years in a lead or principal role. The position involves defining the end-to-end data architecture and leading engineers in the design, development, performance tuning, and operation of event-driven data pipelines and cache systems. It also includes establishing engineering practices, improving data integrity and reconciliation, adopting AI-assisted engineering tools, and collaborating with modernization, API, architecture, and platform teams.

Requirements

  • 8+ years of experience in Data Engineering, including at least 2+ years in a lead or principal role.
  • Experience with the AWS data ecosystem, including Amazon Aurora PostgreSQL, RDS, and DynamoDB.
  • Experience in data modeling, including ODS and transactional cache patterns, partitioning, indexing strategies, and data retention/archival policies.
  • Hands-on experience with Amazon ElastiCache (Redis), including cache design, key and TTL strategies, invalidation patterns, and consistency with source data.
  • Proficiency in Java or Python.
  • Experience consuming events from Apache Kafka or AWS MSK and managing offsets.
  • Hands-on experience integrating AI-assisted development and testing practices into engineering workflows.
  • Expert-level PostgreSQL and advanced SQL skills, including query optimization, execution plan analysis, vacuuming, and connection pooling.
  • Database DevOps experience with schema version control using Liquibase or Flyway.
  • Proven experience with large-scale ETL/ELT pipelines and Change Data Capture (CDC) or stream-based data synchronization.

Nice To Haves

  • Experience in financial services, core banking, or payment platforms.
  • Familiarity with ODS.

Responsibilities

  • Define end-to-end data architecture, data modeling, partitioning, indexing, and retention strategies for high-volume PostgreSQL/Aurora workloads.
  • Lead and mentor Senior and Mid-level Data Engineers and establish best practices for SQL performance tuning, database migrations, and CI/CD pipelines.
  • Build cache read/write logic in Amazon ElastiCache (Redis), including key design, TTLs, and invalidation handling.
  • Monitor and improve high-throughput read/write queries, lock contention, and index overhead.
  • Architect event-driven ingestion pipelines that consume events from event streams and drive near real-time cache invalidation and refresh mechanisms.
  • Design automated reconciliation frameworks and audit jobs to detect cache drift against source ODS views and maintain consistency.
  • Drive adoption of AI-assisted engineering tools for code generation, SQL optimization, and test automation.
  • Partner with Service Modernization leads, API engineers, enterprise architects, and platform/cloud operations.

Benefits

  • Paid time off based on employee grade, including 12–25 vacation days depending on grade, company-paid holidays, personal days, and sick leave.
  • Medical, dental, and vision coverage, or provincial healthcare coordination in Canada.
  • Retirement savings plans, such as 401(k) in the U.S. or RRSP in Canada.
  • Life and disability insurance.
  • Employee assistance programs.
  • Other benefits as provided by local policy and eligibility.
  • The role may be eligible for variable incentives, bonuses, or commissions, depending on the position and applicable laws.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service