Data Architect Jobs

About The Position

Requirements

• 12+ years building data systems, including at least 5 years as lead architect or principal engineer on a large-scale data platform.
• Continues to write production code.
• Strong Python skills, plus at least one of Java, Scala, or Go, and fluency in SQL.
• Personally led a major data platform re-architecture or cloud migration and can explain the design, sequencing, failures, and lessons learned.
• Deep hands-on experience with distributed processing and streaming technologies such as Spark, Beam/Dataflow, Flink, Kafka, Pub/Sub, or Pulsar.
• Production experience with GCP data services such as BigQuery, Dataflow, Pub/Sub, GCS, and Spanner.
• AWS experience, for migration from AWS to GCP.
• Expert-level operation of Elasticsearch or OpenSearch at scale, including cluster sizing, mappings, aggregation performance, and cost tuning.
• Experience with lakehouse table formats such as Iceberg, Delta, or Hudi, and orchestration such as Airflow/Composer, Dagster, or similar.
• Track record of materially reducing platform costs without harming performance, with supporting metrics.
• Clear written communication and ability to write design documents understandable to both executives and senior engineers.
• Ability to obtain a U.S. security clearance.
• Bachelor's degree in Computer Science, Engineering, or a related technical field, or equivalent hands-on experience.

Nice To Haves

• Master's degree or PhD.
• Experience with vector search, embeddings pipelines, and retrieval architectures for LLM and agentic applications.
• Experience with entity resolution, knowledge graphs, or graph databases, ideally involving multilingual or messy real-world data.
• Background in OSINT, national security or law enforcement, risk intelligence, or financial data.
• Experience building for FedRAMP, IL5, or air-gapped environments, including operating data platforms without managed cloud services.
• Experience with data observability, lineage, and governance tools such as OpenLineage, Great Expectations, Monte Carlo, or Dataplex.
• Experience combining large volumes of scraped multimedia and structured third-party data.

Responsibilities

• Map the existing data estate end to end, documenting sources, pipelines, stores, consumers, delivery paths, volumes, costs, owners, and failure history.
• Publish a target-state data architecture and sequenced migration plan for engineering and product leadership approval.
• Design and build a standardized ingestion and processing backbone for streaming and batch workloads, including enrichment and delivery at millions of documents per day with clear SLOs.
• Lead the data platform workstream of the GCP migration, determining which workloads to move, rebuild, or retire and owning the technical design of each migration wave.
• Rationalize warehouse, search and index, object, vector, and graph storage patterns and consolidate workloads to reduce costs.
• Build AI-ready data foundations, including agent-callable data services, hybrid retrieval, and provenance and lineage for records.
• Build a governance and compliance platform to serve governed data and manage parameters, exclusions, and retention for AI readiness.
• Establish data contracts, quality checks, observability, cost attribution, pipeline ownership, SLOs, and known unit costs.
• Own architecture and reference implementations for collection, ingestion, parsing, enrichment, and delivery, including translation, named entity recognition, and entity resolution.
• Design event-driven and queue-based processing, idempotency, backfills and replay, schema evolution, and exactly-once or effectively-once guarantees where appropriate.
• Own the physical data platform across warehouse, lakehouse, search, and object storage, including partitioning, indexing, tiering, retention, query and aggregation performance, and cost trade-offs at petabyte scale.
• Design migration of data workloads from AWS to GCP with minimal customer disruption and establish patterns for FedRAMP, disconnected, and air-gapped deployments.
• Build APIs and query layers for analysts and agents and hybrid retrieval across structured, unstructured, vector, and graph data in partnership with AI and Graph teams.
• Track platform unit economics by source, pipeline, query, and customer.
• Create architecture decision records, lead design reviews, establish coding standards, mentor senior engineers, partner with offshore engineering teams, and serve as escalation point for difficult data incidents.
• Write production code and communicate technical designs clearly to both executive and engineering audiences.

Benefits

• Medical, dental, vision, life, and disability insurance, with Babel Street covering 85–100% of monthly premiums for employees and their families.
• Traditional and Roth 401(k) retirement plans with a competitive match.
• Unlimited flexible leave.
• 12 paid federal holidays.
• Tuition reimbursement for continuing education.