Principal Data Architect

Babel Street • Reston, VA

1d • $220,000 - $250,000

About The Position

Babel Street is hiring a hands-on Principal Data Architect to redesign and rebuild its data infrastructure and pipelines. The role will set the target architecture, write critical production code, and lead engineers through a multi-year migration of a petabyte-scale platform to Google Cloud. Babel Street collects, enriches, and resolves open-source and commercial data from hundreds of sources and dozens of languages. The current environment includes separate warehouse, search, and object-storage systems, along with legacy delivery systems that are difficult to change and costly to operate. The goal is a coherent, lower-cost, more reliable platform built for AI and agentic workloads.

This is a principal engineering role rather than a people-management position. Success is measured by the systems designed and delivered, the quality of technical decisions, and the ability to raise engineering standards. The role reports to the SVP of Engineering and works with the Head of Architecture and the AI, Graph, and Product teams.

During the first 90 days, the architect will map the existing data estate—including sources, pipelines, stores, consumers, delivery paths, volumes, costs, owners, and failure history—and publish a target architecture and sequenced migration plan for engineering and product leadership approval. The role will design and build a standardized ingestion and processing backbone for streaming and batch workloads, lead the GCP data-platform migration, rationalize storage and search, and establish a data foundation for AI through agent-callable services, hybrid retrieval, provenance, and lineage. It will also establish governance and compliance capabilities and operational practices such as data contracts, quality checks, observability, SLOs, and cost attribution.

The architect owns architecture and reference implementations for data collection, ingestion, parsing, enrichment, and delivery; the physical platform across warehouse, lakehouse, search, and object storage; and technical designs for moving workloads from AWS to GCP with minimal customer disruption. Designs must also support FedRAMP, disconnected, and air-gapped deployments where managed cloud services may not be available. The role develops retrieval APIs and query layers for analysts and agents, supports RAG and agentic workflows with AI and Graph teams, and establishes data-quality, lineage, provenance, observability, SLO, and unit-economics practices. Technical leadership includes architecture decision records, design reviews, coding standards, mentoring senior engineers, partnering with offshore engineering teams, and serving as the escalation point for major data incidents.

Requirements

• 12+ years building data systems, including at least 5 years as lead architect or principal engineer on a large-scale data platform.
• Strong production coding skills in Python and at least one of Java, Scala, or Go, and fluency in SQL.
• Personally led a major data-platform re-architecture or cloud migration and able to explain its design, sequencing, failures, and lessons learned.
• Deep hands-on experience with distributed processing and streaming technologies such as Spark, Beam/Dataflow, Flink, Kafka, Pub/Sub, or Pulsar.
• Production experience with GCP data services such as BigQuery, Dataflow, Pub/Sub, GCS, and Spanner; AWS experience is also expected.
• Expert-level operation of Elasticsearch or OpenSearch at scale, including cluster sizing, mappings, aggregation performance, and cost tuning.
• Experience with lakehouse table formats such as Iceberg, Delta, or Hudi and orchestration tools such as Airflow/Composer, Dagster, or similar.
• Track record of materially reducing platform costs without harming performance, supported by measurable results.
• Clear written communication, including the ability to write design documents understandable to both executives and senior engineers.
• Ability to obtain a US security clearance.
• Bachelor's degree in Computer Science, Engineering, or a related technical field, or equivalent hands-on experience.

Nice To Haves

• Experience with vector search, embeddings pipelines, and retrieval architectures for LLM and agentic applications.
• Experience with entity resolution, knowledge graphs, or graph databases, ideally with multilingual or messy real-world data.
• Background in OSINT, national security or law enforcement, risk intelligence, or financial data.
• Experience building for FedRAMP, IL5, or air-gapped environments, including operating data platforms without managed cloud services.
• Experience with data observability, lineage, and governance tooling such as OpenLineage, Great Expectations, Monte Carlo, or Dataplex.
• Experience combining large volumes of scraped multimedia and structured third-party data.
• A Master's degree or PhD.

Responsibilities

• Map the existing data estate end to end, including sources, pipelines, stores, consumers, delivery paths, volumes, costs, owners, and failure history.
• Publish a target-state data architecture and sequenced migration plan for engineering and product leadership approval.
• Design and build a standardized ingestion and processing backbone for streaming and batch ingestion, enrichment, and delivery at millions of documents per day with clear SLOs.
• Lead the data-platform workstream of the GCP migration, deciding what to move, rebuild, or retire and owning the technical design of each migration wave.
• Rationalize warehouse, search and index, object, vector, and graph storage patterns; consolidate systems and reduce the cost of expensive workloads.
• Build agent-callable data services and hybrid retrieval across keyword, analytical, vector, and graph data, with provenance and lineage on every record.
• Build a governance and compliance platform to centralize and serve governed data, including privacy-related parameters, exclusions, and retention.
• Establish data contracts, quality checks, observability, cost attribution, pipeline ownership, SLOs, and known unit costs.
• Own architecture and reference implementations for collection, ingestion, parsing, enrichment, and delivery, including translation, NER, and entity resolution.
• Design event-driven and queue-based processing, idempotency, backfills and replay, schema evolution, and exactly-once or effectively-once guarantees where needed.
• Own the physical data platform across warehouse, lakehouse, search, and object storage, including partitioning, indexing, tiering, retention, query and aggregation performance, and hardware and cost trade-offs.
• Design AWS-to-GCP workload migration with minimal customer disruption and patterns that also work in FedRAMP, disconnected, and air-gapped deployments.
• Build APIs and query layers for analysts and agents and develop hybrid retrieval with AI and Graph teams to support RAG and agentic workflows.
• Establish platform unit economics, including cost per source, pipeline, query, and customer.
• Write architecture decision records, conduct design reviews, and establish data-engineering coding standards.
• Mentor senior engineers, partner with offshore engineering teams, and serve as the escalation point for the hardest data incidents.
• Write production code and lead a major data-platform architecture and migration effort.

Benefits

• Babel Street covers 85–100% of monthly premium costs for medical, dental, vision, life, and disability insurance for employees and their families.
• Traditional and Roth 401(k) plans with a competitive match.
• Unlimited flexible leave.
• 12 paid federal holidays.
• Tuition reimbursement for continuing education.

What This Job Offers

• Job Type: Full-time
• Career Level: Principal