Technical Lead/Architect (Temporal & Apache Kafka required)

Infinitive Inc•Mclean, VA

1d•$90,000 - $154,000

About The Position

We are seeking a Staff / Principal Data Engineer & Platform Architect to define the technical vision, architectural strategy, and technical execution for our enterprise-wide, next-generation event-driven data platforms. In this role, you will act as the principal authority bridging ultra-high-throughput streaming topologies, stateful distributed workflow orchestration, and federated data governance. You will spearhead our platform architecture centered around Apache Kafka for real-time event distribution and Temporal for durable execution and distributed state orchestration. A core mandate of this position is establishing enterprise data contracts, driving schema evolution frameworks, and designing resilient Lakehouse/warehouse architectures capable of serving high-concurrency analytical and operational workloads. As a technical leader, you will set the engineering standards, mentor senior talent, and align multi-team distributed systems initiatives with executive business priorities.

Requirements

8+ years of professional experience in data engineering, distributed systems, or backend platform architecture, with 2+ years acting in a Lead, Staff, or Principal/Architect capacity.
Deep architectural experience with Temporal (or Cadence), including deterministic execution modeling, activity orchestration, workflow versioning strategies, signal/query handling, dynamic task routing, and cross-boundary error recovery.
Deep foundational expertise with Apache Kafka internals (broker sizing, partition strategies, consensus mechanisms, custom connector development, consumer group rebalancing, and stateful stream processing via Kafka Streams or Flink).
Authority in schema specification frameworks (Apache Avro, Protocol Buffers/gRPC, JSON Schema).
Hands-on leadership implementing schema registries (Confluent, AWS Glue) with automated CI/CD schema linting, breaking-change prevention, and backward/forward compatibility strategies.
Experience standardizing automated data contract verification tooling (e.g., Great Expectations, Pandera, Pydantic, dbt tests, contract-testing frameworks).
Advanced engineering capabilities in Python, Go, or Java/Scala, emphasizing distributed design patterns, concurrency models, and resilient microservice design.
Extensive scale experience with Apache Spark / PySpark internals and query plan optimization.
Expert-level knowledge of dimensional data modeling, Data Vault, or semantic layer design, with practical experience scaling multi-terabyte/petabyte systems on Snowflake, Databricks, or BigQuery.
Demonstrated track record of aligning distributed engineering teams around unified architectural patterns, authoring architectural decision records (ADRs), and influencing executive stakeholders.

Responsibilities

Define the multi-year technical roadmap and reference architecture for our real-time streaming infrastructure, Lakehouse ingestion engines, and distributed orchestration layers.
Architect and govern multi-tenant, enterprise Apache Kafka ecosystems (producers, consumers, Kafka Connect, Schema Registry), ensuring sub-second latencies, exactly-once processing guarantees, and disaster recovery.
Formulate and enforce company-wide data governance, schema design patterns, and automated contract testing (Avro, Protobuf, JSON Schema) across polyglot microservices, streaming pipelines, and storage boundaries.
Architect long-running, fault-tolerant workflow topologies using Temporal, standardizing complex distributed transaction patterns (e.g., Saga patterns, compensation logic, distributed state machines, human-in-the-loop flows).
Direct the design and operational scaling of large-scale distributed computing workloads (Apache Spark / PySpark) and modern cloud Lakehouse platforms (Snowflake, Databricks, BigQuery) using medallion architectures and dimensional/semantic models.
Implement comprehensive observability platforms (tracing, telemetry, automated drift detection, data quality gates) while establishing cloud infrastructure cost optimization and capacity planning frameworks.
Act as the technical authority across platform and product teams; establish engineering excellence standards, drive RFC and Architecture Review Board (ARB) processes, and elevate senior engineering talent across the organization.