Senior Data Infrastructure Engineer

Aircall.io•San Francisco, CA

2d•$150,000 - $200,000

About The Position

Aircall's Data team is undergoing a significant migration, moving from a single Redshift cluster to an Apache Iceberg lakehouse on S3. This new platform utilizes Flink CDC for ingestion into Kafka and dbt-on-Spark via Apache Kyuubi on EKS for transformation. This is a greenfield platform build, already scoped and underway, layered on top of a decade of startup growth. This role is dedicated to platform work, aiming to provide high-leverage frameworks that will simplify the jobs of engineers managing both infrastructure and business datasets. Key areas of focus include data quality automation, schema registry, and staging/gated promotion processes. The successful candidate will build foundational elements from the ground up, serving analytics engineers, data scientists, and AI agents as their customers, with their product being the leverage that enables these users.

Requirements

4+ years (Senior: 6+) in data engineering, data platform or infrastructure engineering.
Strong Python and SQL, with demonstrated experience building frameworks and tooling others depend on, not only pipelines.
Production experience with an orchestration framework (Airflow, Dagster, Prefect) at meaningful scale — including the operational side, not just DAG authoring.
Hands-on Apache Spark and distributed-systems fundamentals.
Deep AWS experience (S3, EKS/ECS, IAM, Glue/Athena or equivalent).
Comfortable building and debugging CI/CD, infrastructure as code (Terraform) and GitOps workflows; familiar with Kubernetes and Docker.
Track record owning reliability: SLAs, monitoring, alerting, on-call, and post-incident hardening.
Daily, hands-on use of AI coding tools (Claude Code, Cursor, or equivalent) as a core part of how you build and operate infrastructure.
Great cross-functional communication — you'll shape data contracts with backend engineering and align expectations with analytics consumers.

Nice To Haves

Production experience with an open table format (Apache Iceberg, Delta Lake, Hudi) and lakehouse migration off a classic warehouse.
Streaming experience: Kafka/MSK, Flink, CDC pipelines, Kinesis.
Experience with data governance tooling — Lake Formation, Unity Catalog, or equivalent RBAC/masking implementations.
Familiarity with dbt (as a platform provider — dbt-spark, adapters, CI for dbt) and with data observability tooling such as Monte Carlo.
Experience designing platforms consumed by AI/LLM workloads and low-latency analytics engines.
Open-source contributions to data infrastructure projects.

Responsibilities

Build and operate the lakehouse: Apache Iceberg on S3, table design and maintenance, partitioning and compaction, and the migration of remaining Redshift workloads onto it.
Own ingestion end to end — Flink CDC → Kafka (MSK) → Iceberg, plus Rudderstack, Fivetran and DMS sources — and hold the freshness and reliability SLAs on it.
Run and evolve the compute and orchestration layer: Apache Kyuubi on EKS for dbt-spark, Airflow (completing its ECS → EKS migration), autoscaling, spot strategy and cost efficiency.
Build the tooling, libraries and templates that let analytics engineers and data scientists own their own pipelines without filing a ticket — self-service is the deliverable, not a side effect.
Close our environment gaps: a real staging environment, CI that tests against staging rather than production, automated schema-change detection, gated promotion and canary deploys for critical models.
Own governance and access at the platform level: Lake Formation row/column RBAC, StrongDM zero-trust access, SSO, audit logging, and PII handling.
Own observability: Monte Carlo, lineage, alerting and the SLAs we publish — and drive incidents to root cause and to a durable fix.
Champion infrastructure as code and automation (Terraform, GitLab CI, GitOps) across everything the team runs.