AI Engineer

CACI International

3d•$114,600 - $252,100

About The Position

CACI is seeking an AI Engineer to build, test, and support AI workflows and agents on an enterprise AI workflow management platform supporting the Department of the Air Force. The role works directly with the AI Platform Architect to turn platform designs and government use cases into tested, production-ready solutions in Azure IL-5 cloud environments integrated with ServiceNow. The engineer builds within established frameworks, patterns, and tools, including Microsoft AI Foundry and third-party agentic platforms, and helps translate customer mission needs into prototypes and deployed workflows. The role includes developing AI workflows and retrieval-augmented generation pipelines; integrating AI solutions with Azure OpenAI, ServiceNow, and internal data sources; creating reusable components; testing and evaluating AI outputs; supporting platform design and integrations; monitoring production workflows; and meeting government security and compliance requirements, including DISA STIGs, IL-5 requirements, Department of the Air Force policies, CUI controls, ATO evidence, and continuous monitoring. The position requires U.S. citizenship and the ability to obtain and maintain a Secret clearance. The stated experience requirement is 15 or more years of relevant experience, with a relevant bachelor's degree substitutable for five years of experience. The role is full-time and requires up to 10% local travel. The proposed national salary range is $114,600–$252,100.

Requirements

U.S. citizenship.
Ability to obtain and maintain a Secret clearance.
15 or more years of relevant experience; a bachelor’s degree in a relevant field may substitute for five years of relevant experience.
Hands-on experience building LLM applications, including RAG, prompt engineering, and agentic workflows.
Proficiency in Python and experience with LangChain, LlamaIndex, or similar AI frameworks.
Experience with Azure cloud services, including Azure OpenAI, AI Foundry, DevOps, and API Management.
Experience with containers such as Docker and REST API development and integration.
Working knowledge of vector databases and embedding strategies.
Experience with test automation and quality assurance for software or AI systems.
Experience with ServiceNow integration or scripting, such as Flow Designer, REST APIs, or development.
Ability to translate requirements into working solutions and communicate technical results to non-technical stakeholders.
Familiarity with workflow orchestration tools and business process automation.

Nice To Haves

Active Secret clearance; Top Secret eligibility preferred.
Experience in government cloud environments such as Azure Government, AWS GovCloud, or similar.
Experience with Kubernetes or Azure Kubernetes Service (AKS).
Familiarity with DoD compliance frameworks, including IL-4/IL-5, FedRAMP, NIST 800-53, and STIGs.
Experience with MLOps tools and practices, including MLflow, CI/CD for ML, and model monitoring.
Experience with PostgreSQL and data modeling for AI workloads.
Experience with ATO processes and security accreditation.
Experience with ServiceNow Flow Designer or a ServiceNow development certification.
Security+ certification.

Responsibilities

Build AI workflows and agents using the platform’s frameworks, tools, and reference architectures.
Develop RAG pipelines, including document ingestion, chunking, embedding, vector search, and retrieval tuning.
Build prompt chains, tool calls, and agent logic for document generation, summarization, and task automation.
Integrate AI workflows with Azure OpenAI services, ServiceNow APIs, and internal data sources.
Build reusable components, templates, and connectors for technical and non-technical users.
Implement workflow orchestration and business process automation.
Develop and execute unit, integration, and user acceptance test plans for AI workflows.
Build evaluation datasets and automated evaluation harnesses to assess LLM output accuracy, relevance, groundedness, and reliability.
Define and track success metrics and evaluation criteria for AI-driven workflows.
Identify, document, and triage defects, hallucinations, and performance issues; conduct root-cause analysis and remediation.
Perform regression testing when models, prompts, or platform components change.
Partner with the AI Platform Architect to implement designs, evaluate tools, and provide feedback on platform capabilities and limitations.
Prototype solutions and proofs of concept for government use cases and present results to customers and stakeholders.
Troubleshoot integrations across Azure, ServiceNow, and government networks with ServiceNow developers and Azure cloud engineers.
Contribute to design reviews, engineering standards, and technical documentation.
Support technical screening of AI/ML talent when requested.
Support monitoring, logging, and alerting for AI workflows and agents.
Manage prompt and model versioning and support CI/CD deployments of AI services.
Monitor production output quality, model drift, and data quality, and report findings.
Track model and compute usage costs and recommend optimizations.
Build and operate within DISA STIGs, IL-5 requirements, and Department of the Air Force security policies.
Implement CUI handling and data classification controls in AI workflows.
Provide engineering evidence and documentation to cybersecurity teams for ATO and continuous monitoring.