Principal Data Architect

Hatch IT • Reston, VA

3d

About The Position

Babel Street is hiring a hands-on Principal Data Architect to redesign and rebuild its data infrastructure and pipelines. The architect will set the target architecture, write critical production code, and lead engineers through a multi-year migration of a petabyte-scale platform to Google Cloud. Babel Street collects, enriches, and resolves open-source and commercial data across hundreds of sources and dozens of languages. The role will replace separately evolved warehouses, search and object stores, and legacy delivery systems with a coherent, lower-cost, more reliable platform built for AI and agentic workloads. This is a principal engineering role, not a people-management position; success is measured by the systems delivered, technical decisions, and contribution to the engineering team.

The role reports to the SVP of Engineering and works alongside the Head of Architecture and the AI, Graph, and Product teams. In the first year, the architect will map the existing data estate; publish a target architecture and migration plan; build a standardized ingestion and processing backbone; lead the data-platform workstream of the AWS-to-GCP migration; rationalize storage and search; and deliver AI-ready data services, hybrid retrieval, governance, compliance, lineage, observability, and cost attribution.

The position owns architecture and reference implementations for ingestion, processing, enrichment, storage, compute, search, cloud migration, retrieval, data quality, lineage, and platform economics. It also provides technical leadership through decision records, design reviews, coding standards, mentoring, collaboration with offshore engineering teams, and escalation support for major data incidents.

Babel Street provides AI and data analytics solutions for identity intelligence and risk operations, serving government and commercial organizations. The company is headquartered in Reston, Virginia, with regional offices in Boston and Cleveland and international offices in Australia, Canada, Israel, Japan, and the U.K.

Requirements

• At least 12 years building data systems, including at least 5 years as lead architect or principal engineer on a large-scale data platform.
• Continued ability to write production code.
• Strong Python skills, plus proficiency in at least one of Java, Scala, or Go, and fluency in SQL.
• Personal experience leading a major data-platform re-architecture or cloud migration, with the ability to explain its design, sequencing, failures, and lessons learned.
• Deep hands-on experience with distributed processing and streaming technologies, such as Spark, Beam/Dataflow, Flink, Kafka, Pub/Sub, or Pulsar.
• Production experience with GCP data services such as BigQuery, Dataflow, Pub/Sub, GCS, and Spanner.
• AWS experience to support migration from AWS to GCP.
• Expert-level operation of Elasticsearch or OpenSearch at scale, including cluster sizing, mappings, aggregation performance, and cost tuning.
• Experience with lakehouse table formats such as Iceberg, Delta, or Hudi, and orchestration tools such as Airflow/Composer, Dagster, or similar.
• Demonstrated record of materially reducing platform costs without harming performance, with measurable results.
• Clear written communication and ability to write design documents accessible to both executives and senior engineers.
• Ability to obtain a U.S. security clearance.

Nice To Haves

• Experience with vector search, embeddings pipelines, and retrieval architectures for LLM and agentic applications.
• Experience with entity resolution, knowledge graphs, or graph databases, ideally involving multilingual or messy real-world data.
• Background in OSINT, national security, law enforcement, risk intelligence, or financial data.
• Experience building for FedRAMP, IL5, or air-gapped environments, including operating platforms without managed cloud services.
• Experience with data observability, lineage, and governance tools such as OpenLineage, Great Expectations, Monte Carlo, or Dataplex.
• Experience combining large volumes of scraped multimedia and structured third-party data.

Responsibilities

• Map the existing data estate end to end, documenting sources, pipelines, stores, consumers, delivery paths, volumes, costs, owners, and failure history.
• Publish a target-state data architecture and sequenced migration plan for engineering and product leadership approval.
• Design and build a standardized ingestion and processing backbone for streaming and batch workloads, including enrichment and delivery at millions of documents per day with clear SLOs.
• Lead the data-platform workstream of the GCP migration, determining what to move, rebuild, or retire and owning the technical design of each migration wave.
• Rationalize warehouse, search and index, object, vector, and graph storage patterns, and consolidate workloads to reduce cost while maintaining performance.
• Build AI data foundations, including agent-callable data services, hybrid retrieval, and provenance and lineage on every record.
• Build governance and compliance capabilities to serve governed data and manage privacy parameters, exclusions, and retention for AI readiness.
• Establish data contracts, quality checks, observability, SLOs, ownership, and cost attribution across pipelines.
• Own architecture and reference implementations for collection, ingestion, parsing, enrichment, and delivery, including translation, NER, and entity resolution.
• Design event-driven and queue-based processing, idempotency, backfills and replay, schema evolution, and exactly-once or effectively-once guarantees where needed.
• Design the physical data platform across warehouse, lakehouse, search, and object storage, including petabyte-scale partitioning, indexing, tiering, retention, query performance, and cost trade-offs.
• Design migration of data workloads from AWS to GCP with minimal customer disruption, including patterns suitable for FedRAMP, disconnected, and air-gapped environments.
• Build APIs and query layers for analysts and agents, and collaborate with AI and Graph teams on hybrid retrieval for RAG and agentic workflows.
• Establish data contracts, automated quality checks, lineage, provenance, observability, SLOs, and unit economics across sources, pipelines, queries, and customers.
• Write architecture decision records, lead design reviews, establish data-engineering coding standards, mentor senior engineers, partner with offshore teams, and serve as escalation point for difficult data incidents.
• Continue writing production code and lead technical implementation of the platform architecture.

Benefits

• Medical, dental, vision, life, and disability insurance; Babel Street covers 85–100% of monthly premiums for employees and their families.
• Traditional and Roth 401(k) retirement plans with a competitive company match.
• Unlimited flexible leave.
• 12 paid federal holidays.
• Tuition reimbursement for continuing education.

What This Job Offers

• Job Type: Full-time
• Career Level: Principal
• Education Level: No Education Listed