Senior Data Engineer

Gravie

10d•$133,875 - $180,000

About The Position

Gravie is hiring a Senior Data Engineer to build, extend, and operate a near-real-time streaming data platform using Kafka and Flink. The platform converts operational data from across the business into canonical, contract-validated models for operational applications and agentic workloads. This hands-on role has meaningful ownership of a live production platform and its latency and throughput SLOs, in a regulated environment. The engineer will share full-lifecycle ownership of the platform, from architecture and implementation through production operations. The role includes developing CDC-based streaming pipelines, transforming and enriching data in Flink, enabling secure and governed access to PHI, and making pipelines reliable and observable. The engineer will provision AWS streaming, processing, storage, and catalog services as code using CDK, establish CI/CD for data pipelines, and collaborate with producers, consumers, Security, Compliance, and other stakeholders. Gravie describes its culture as non-hierarchical, merit-driven, opinionated but kind, and focused on high performance and positive impact. It seeks someone authentic, curious, creative, empathetic, and outcome oriented who is self-driven and calm under production pressure.

Requirements

At least 6 years building and operating production data systems, including demonstrated ownership of streaming or event-driven pipelines involving on-call, incident response, SLOs, runbooks, and recovery.
Deep production experience with Apache Kafka; MSK is preferred. Experience with partitioning, consumer groups, idempotent semantics, Kafka Connect, and replay/backfill under load.
Strong hands-on production experience with Apache Flink or a comparable stateful stream processor in Java or another JVM language, including state management, checkpointing, watermarks, and exactly-once sinks.
AWS-native data engineering across streaming, processing, storage, and catalog services, such as MSK, EMR, Glue, S3, and Athena; infrastructure as code using AWS CDK or Terraform; CI/CD for data pipelines; and cost awareness.
Ability to debug distributed data pipelines, including consumer lag, data skew, backpressure, checkpoint failures, and late or out-of-order events, using observability tools such as Datadog or CloudWatch.
SQL and Python skills, and experience building and consuming REST APIs.
Hands-on use of AI-assisted and agentic development tools; an informed perspective on where AI adds value; and understanding of agentic data-consumption patterns for trusted operational data, including context management, lineage, provenance, permissions, and freshness.
Experience with change data capture, such as Debezium, and open table formats such as Iceberg or Delta Lake, including schema evolution, partitioning, and table maintenance.
Experience with data contracts and schema governance, including schema registries, technical metastores such as Glue Data Catalog or Unity Catalog, compatibility rules, Avro or Protobuf, and dead-letter handling.
Degree in Computer Science, Information Systems, or another quantitative field.
Comfort using the command line and a Unix-based operating system; Gravie uses Mac and Linux.
Excellent communication skills and demonstrated success driving results through influence and collaboration.

Nice To Haves

Health insurance domain knowledge, including HIPAA Protected Health Information (PHI) and the compliance frameworks governing it.
Familiarity with serving data to AI and agentic consumers, including exposing canonical data as low-latency context or inputs for automated or agentic workloads.
Experience with a governance and data discovery catalog such as Atlan, Alation, or Collibra.
Previous experience at a venture-backed startup.
Experience with AWS MSK and AWS CDK are specifically preferred.

Responsibilities

Share full-lifecycle ownership of the streaming platform, including architecture, implementation, and production operations.
Operate the production platform and own latency and throughput SLOs, monitoring, and alerting, including replays, per-source backfills, connector-failure recovery, Flink checkpoint/savepoint recovery, dead-letter triage, and tuning for spiky upstream batch loads.
Build and extend pipelines that ingest CDC events from OLTP databases and SaaS sources into canonical, contract-validated models, and define standards for managing changes to those pipelines.
Transform and enrich data in Apache Flink using stateful processing and cross-stream joins to correlate events into unified lifecycle entities, handling late and out-of-order data with event time.
Enable secure, governed access to PHI through classification and row- and column-level controls applied as data is served, in partnership with Security and Compliance.
Design idempotent and replayable pipelines, implement data-quality validation, create runbooks, and provide observability the broader data team can rely on.
Provision streaming, processing, storage, and catalog services on AWS as code in CDK; establish CI/CD for data pipelines and right-size the platform for cost against latency SLOs.
Partner with upstream producers on source changes and contracts, downstream consumers on access and data needs, and stakeholders to translate requirements into platform capabilities.
Demonstrate the company’s core competencies of being authentic, curious, creative, empathetic, and outcome oriented.