Data Engineer (AI-ML Data Systems)

Pantheon Data • Reston, VA

3d • $110,000 - $190,000 • Remote

About The Position

Pantheon Data is seeking a hands-on Data Engineer to design, build, and operate the data foundations supporting analytics, AI/ML, and intelligent document processing. The role covers data ingestion, orchestration, transformation, quality checks, storage, query patterns, operational monitoring, and delivery to downstream applications. The engineer will work across structured, semi-structured, and unstructured data in relational, graph, and vector stores, building reliable, explainable, maintainable pipelines that serve engineering teams and products.

The role owns data pipelines and data stores and partners closely with Machine Learning Engineers, who own models, retrieval, extraction, and evaluation. Those engineers define document chunking, embedding, enrichment, and retrieval approaches; the Data Engineer implements and operates the pipelines and stores that apply those designs reliably at scale. Strong Python and SQL skills and sound data engineering judgment are central. Experience with AWS, graph databases and query languages such as AQL, vector search, document data, or AI/ML data pipelines is valuable.

Pantheon Data is a private small business founded in 2011 and based in the Washington, DC, area. The position is listed as remote, with a work location of Reston, Virginia. Employees may periodically work from a Pantheon Data office or client site when needed for client requirements. Local candidates should be prepared for an in-person interview; candidates outside the area may be considered for a virtual interview. Applicants must be able to obtain and maintain a Secret clearance; an active Secret clearance is preferred. The salary range is $110,000–$190,000, with actual compensation based on experience, geographic location, and possibly contractual requirements.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related technical field from an accredited university.
• At least 5 years of professional hands-on data engineering, software engineering, analytics engineering, or closely related experience.
• Strong Python programming skills, including production-oriented, maintainable code experience.
• Strong SQL skills and practical knowledge of data modeling, query performance, joins, indexing, schemas, normalization and denormalization, and data quality.
• Understanding of batch processing, event-driven workflows, ETL/ELT, orchestration, idempotency, retries, backfills, lineage, and failure handling.
• Working knowledge of OLTP versus OLAP systems and tradeoffs among transactional databases, analytical stores, object storage, search systems, and graph and vector databases.
• Experience building or supporting pipelines that move data between systems such as APIs, databases, files, object storage, queues, warehouses, or downstream applications.
• Ability to reason about data correctness, schema changes, validation, reconciliation, duplicate handling, missing data, and operational recovery.
• Comfort with Git, pull requests, code review, issue tracking, documentation, and collaborative software development practices.
• Strong communication skills and ability to explain data flows, design choices, limitations, and tradeoffs to technical and non-technical stakeholders.
• Ability to work effectively in distributed, cross-functional engineering teams, produce high-quality work with limited hand-holding, and work remotely.
• Ability to meet deadlines and produce quality work.
• Proficiency with Microsoft Suite software, including Outlook, Word, Excel, SharePoint, and PowerPoint.
• Ability to obtain and maintain a Secret security clearance.
• Candidates local to the area should be prepared to participate in an in-person interview; candidates outside the area may be considered for a virtual interview.

Nice To Haves

• AWS data services experience, including S3, Lambda, Glue, Athena, Step Functions, SQS/SNS, Kinesis, EMR, RDS, DynamoDB, Redshift, OpenSearch, or CloudWatch.
• Experience with workflow orchestration tools such as Airflow, Dagster, Prefect, AWS Step Functions, or Glue.
• Experience with PySpark, Spark, Databricks, EMR, Snowflake, Redshift, or other distributed or analytical data platforms.
• Experience supporting AI/ML or RAG-style data workflows, such as metadata enrichment, retrieval datasets, vector search, embeddings, evaluation datasets, or human validation workflows.
• Experience with document-oriented or unstructured data pipelines involving PDFs, OCR output, tables, forms, images, extracted text, metadata, or search indexes.
• Hands-on experience with graph databases and query languages such as ArangoDB/AQL, Neo4j/Cypher, or Gremlin, including graph-shaped data modeling.
• Experience with vector databases or vector search engines such as pgvector or OpenSearch, including index tuning and hybrid vector-plus-lexical lookups.
• Experience with Docker, CI/CD, infrastructure as code, automated testing, logging, monitoring, and production support.
• Familiarity with data governance, access control, PII handling, auditability, lineage, and compliance-sensitive environments.
• An active Secret security clearance.

Responsibilities

• Design, build, and maintain reliable data pipelines for structured, semi-structured, and unstructured data sources.
• Develop Python-based workflows for data ingestion, normalization, validation, enrichment, and transformation.
• Use SQL and relational data stores for transactional, analytical, and application-facing use cases.
• Model relationships between entities, documents, and metadata in graph databases such as ArangoDB, using AQL or similar query languages to support traversal and relationship-driven queries.
• Build and operate vector data pipelines and stores, including embedding generation at scale, vector index builds and refreshes, backfills, re-embedding when models change, and lookups for retrieval and RAG workloads.
• Implement the chunking, embedding, and metadata specifications defined by Machine Learning Engineers.
• Design data models and storage patterns appropriate to workloads, including OLTP, OLAP, object storage, document-oriented, search, vector, or graph-oriented patterns as applicable.
• Implement orchestration and scheduling for repeatable data workflows using tools such as Airflow, AWS Step Functions, Dagster, Prefect, or Glue workflows.
• Build automated quality checks, reconciliation logic, validation reports, and operational alerts to detect and diagnose data issues.
• Support data pipelines serving AI/ML, retrieval, document intelligence, analytics, and application workflows.
• Collaborate with machine learning engineers, software engineers, cloud engineers, and product stakeholders to turn data problems into working software.
• Write maintainable code, participate in code reviews, document data flows, and contribute to standards for testing, deployment, observability, and version control.
• Own well-scoped data engineering work and help improve the velocity of a growing engineering team while developing broader system ownership.

Benefits

• Competitive salaries and benefits.
• SmartBenefits through the Washington Metro Area Transportation Authority, allowing employees to direct a specified amount of pre-tax wages to a SmarTrip account.
• In some cases, tuition assistance may be available for continuing education expenses and position-related certifications.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level