RAG Data Engineer
About The Position
As a RAG Data Engineer, you will build and operate data ingestion and search infrastructure supporting mission-critical Retrieval-Augmented Generation capabilities. You will design, implement, and maintain pipelines that transform structured and unstructured information into retrieval-ready assets so AI-powered applications can deliver accurate, grounded, and relevant responses. You will own ingestion, embedding, and indexing workflows connecting enterprise data sources with Amazon Bedrock and OpenSearch. This includes document normalization, chunking strategies, embedding generation, metadata enrichment, lineage tracking, and reliable upsert or delete operations. You will also design and optimize OpenSearch indexes, mappings, shard configurations, lifecycle policies, and query performance. Working with cloud architects, security engineers, application developers, and AI/ML specialists, you will help establish a scalable, secure, and resilient data retrieval foundation for enterprise RAG applications. The position is onsite in Annapolis Junction, Maryland, with Fort Meade listed as an additional location. A TS/SCI clearance is required.
Requirements
- Experience developing and maintaining ETL or data ingestion pipelines for structured and unstructured data.
- Experience implementing document normalization, parsing, chunking, and content preparation workflows, generating embeddings, and integrating embedding workflows into data processing pipelines.
- Experience designing and maintaining OpenSearch or Elasticsearch indexes, schemas, mappings, and aliases, and optimizing search performance, including query tuning, indexing efficiency, and shard management.
- Experience implementing idempotent ingestion processes, including upsert, delete, replay, and failure recovery.
- Experience with metadata modeling, enrichment, lineage, and version tracking.
- Experience with Python or other programming languages for data engineering and automation.
- Experience with containerized applications or deployment environments such as Docker, ECS, EKS, or Kubernetes.
- Ability to collaborate with engineering, security, cloud, and application development teams.
- TS/SCI clearance.
- High school diploma or GED.
Nice To Haves
- Experience integrating Amazon Bedrock embeddings into RAG ingestion pipelines.
- Experience tuning hybrid retrieval systems combining BM25 keyword search and vector similarity search.
- Experience implementing OpenSearch lifecycle policies, snapshots, restoration, and index migration strategies.
- Experience with distributed or asynchronous data processing using Spark, Ray, Dask, Kafka, or Amazon SQS.
- Experience optimizing document chunking, embeddings, and indexing strategies using retrieval evaluation results.
- Experience handling multi-source data ingestion and cross-source normalization.
- Experience designing scalable, secure data infrastructure using AWS services.
- Experience with infrastructure-as-code tools such as Terraform, CloudFormation, or AWS CDK.
Responsibilities
- Build and operate data ingestion and search infrastructure for RAG capabilities.
- Design, implement, and maintain pipelines that transform structured and unstructured information into retrieval-ready assets.
- Own ingestion, embedding, and indexing workflows connecting enterprise data sources with Amazon Bedrock and OpenSearch.
- Perform document normalization, chunking, embedding generation, metadata enrichment, and lineage tracking.
- Implement reliable upsert and delete operations.
- Design and optimize OpenSearch indexes, mappings, shard configurations, lifecycle policies, and query performance.
- Collaborate with cloud architects, security engineers, application developers, and AI/ML specialists to establish a scalable, secure, and resilient data retrieval foundation.
Benefits
- Health, life, disability, financial, and retirement benefits.
- Paid leave.
- Professional development.
- Tuition assistance.
- Work-life programs.
- Dependent care.
- Recognition awards for exceptional performance and demonstration of company values.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Education Level
High school or GED
