Azure OpenAI & RAG Engineer

SkyePoint Decisions•Laurel, MD

2d•Onsite

About The Position

SkyePoint Decisions is seeking an experienced AI Engineer with Azure OpenAI experience to design and implement an AI inference and Retrieval-Augmented Generation (RAG) pipeline for the Department of State. This role is responsible for provisioning and configuring Azure OpenAI Service on Azure Government, building the RAG pipeline that grounds AI responses in agency playbooks and live vulnerability feeds, and implementing the confidence scoring mechanism that drives the human validation queue. The engineer works in close coordination with a Power Platform Developer on prompt context assembly and with the agency SME on response quality. This position is based on-site at the Beltsville, MD office, with occasional travel to the Washington, DC office.

Requirements

3+ years of experience with Azure OpenAI Service including API integration, prompt engineering, and deployment in enterprise environments.
Demonstrated experience building RAG pipelines using Azure AI Search or equivalent vector search technology.
Experience with Python or JavaScript/TypeScript for AI pipeline development.
Familiarity with LLM prompt engineering techniques including few-shot prompting, chain-of-thought, and system prompt design.
Experience with FedRAMP-authorized cloud environments, specifically Azure Government.
Experience with REST API integrations for data feed ingestion (JSON APIs, rate limiting, error handling).
Active Top Secret security clearance OR ability to obtain TS within program timeline (interim clearance accepted to start).
US Citizenship is required.

Nice To Haves

Active Top Secret clearance.
Experience with LangChain, Semantic Kernel, or Azure AI Foundry for agentic AI pipeline orchestration.
Experience integrating AI systems with ServiceNow or similar ITSM platforms for context enrichment.
Familiarity with CISA KEV, NVD CVE data structures, and cybersecurity vulnerability terminology.
Experience with Microsoft Copilot Studio or Power Platform AI Builder integrations.
Azure AI certifications (AI-102, DP-100, or equivalent).
Experience with GCC High M365 environments and their AI service constraints.
Familiarity with NIST AI RMF, OMB M-25-21 AI governance requirements, or federal AI policy.

Responsibilities

Provision and configure Azure OpenAI Service on Azure Government in compliance with DOS GovCloud FedRAMP High requirements.
Design and implement the RAG pipeline using Azure AI Search: vector index configuration, document chunking strategy for agency playbooks and runbooks, embedding model selection, and semantic search tuning.
Build and maintain automated data ingestion pipelines for CISA KEV (daily), NVD CVE database (daily), Microsoft Security Update Guide (weekly), and agency SharePoint document library.
Implement ServiceNow ticket context injection: when a matching SNOW ticket is retrieved, structure the vulnerability description, CVE identifier, asset details, and current ticket status into the AI prompt context for response generation.
Design and implement the AI confidence scoring mechanism — per-response confidence scores surfaced to agency analysts in the validation queue.
Implement CUI-safe prompt construction: ensure sanitized email content and structured SNOW ticket data are assembled into prompts without including raw CUI elements that have been vaulted.
Configure Azure AI Content Safety filters appropriate to the advisory use case.
Monitor model performance, token consumption, and response latency in the Azure Government environment.
Support the monthly review of AI confidence score distribution, override rate, and coverage metrics.
Produce AI system documentation including prompt templates, RAG configuration, and model monitoring procedures.

Benefits

certification incentive program
PTO
floating federal holiday options
several insurance options including HMO and High Deductible plans with Health Savings Accounts [HSAs]
Flex Spending Accounts [FSAs]
Full Dental Plans
Vision
ST/LT Disability
Life Insurance
401k matched