Senior Data Engineer (AI/ML and AWS Cloud)

Pantheon Data • Charlotte, NC

1d • $140,000 - $175,000 • Remote

About The Position

We are seeking a Senior Data Engineer to design, build, and optimize the data foundations for our next-generation Generative AI applications. This role is focused on architecting the Data Enrichment and Vectorization pipelines that power Large Language Models (LLMs). You will be responsible for the end-to-end lifecycle of data, from ingestion in AWS to serving high-context, enriched datasets to AWS Bedrock.

Requirements

• Python Mastery: Expert-level Python programming with experience in libraries such as Pandas and LLM orchestration frameworks like LangChain or LlamaIndex.
• AWS AI/ML Ecosystem: Hands-on experience with AWS Bedrock and Amazon SageMaker.
• Data Engineering Foundations: Proven track record with AWS Glue (ETL), Athena, and Redshift.
• Certifications: Must hold a recognized Data Science Certification (e.g., AWS Certified Data Engineer, Databricks Certified Data Scientist).
• Database Expertise: Proficiency in both SQL and NoSQL, with specific experience in Vector Databases.
• Ability to work effectively remotely in cross-functional teams.
• Ability to meet deadlines and produce quality work.
• Proficient in Microsoft Suite software including Outlook, Word, Excel, SharePoint, and PowerPoint.
• U.S. Citizenship with the ability to obtain and maintain a DoD Secret clearance.

Responsibilities

• LLM Data Pipelines: Design and implement scalable data ingestion and transformation pipelines specifically for RAG (Retrieval-Augmented Generation) architectures.
• AWS Bedrock Integration: Operationalize LLM workflows using AWS Bedrock, managing model invocations, and embedding generation.
• Data Enrichment & Quality: Develop advanced Python-based processing jobs to clean and enrich unstructured data with metadata to improve LLM retrieval accuracy.
• Vector Database Management: Architect and maintain vector stores (e.g., OpenSearch Serverless or Postgressql pgvector) to support efficient semantic search.
• Cloud Architecture: Leverage core AWS services (S3, Glue, Lambda, Step Functions) to build resilient, automated data workflows.
• DevSecOps Collaboration: Work with the security team to ensure all data handling meets stringent compliance standards (e.g., FedRAMP/DISA STIGs) through Infrastructure as Code.

Benefits

• Pantheon Data is committed to providing its employees with competitive salaries and benefits in order to increase employee satisfaction and productivity.
• In addition to our benefits, we also offer SmartBenefits through the Washington Metro Area Transportation Authority, where you specify an amount of your pre-tax wages be paid directly to your SmarTrip account.
• In some cases, tuition assistance may be available for continuing education expenses and certifications related to their position.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior