Senior AI Software Engineer

GD Information Technology•Sterling, VA

7d•Onsite

About The Position

GDIT is seeking a Senior AI Software Engineer to develop, modernize, integrate, and sustain an on-premises AI-powered enterprise chatbot platform supporting the U.S. Department of State in Sterling, Virginia. The engineer will build software solutions using large language models (LLMs), retrieval-augmented generation (RAG), vector databases, document-processing technologies, and modern AI inference frameworks. The role spans backend development, model-serving integration, data processing, automated testing, and deployment, with a focus on scalable containerized architecture, system performance, reliability, security, and user experience. The engineer will collaborate with AI/ML engineers, data engineers, infrastructure specialists, cybersecurity teams, and government stakeholders to deliver secure and reliable AI capabilities in classified and air-gapped environments. Work includes integrating AI services and document workflows, modernizing platform deployments, developing and testing software, and supporting security and operational requirements for government systems.

Requirements

At least a Secret clearance; no sponsorship. The posting also lists Secret as the clearance currently required and the ability to obtain Top Secret.
U.S. citizenship required.
Bachelor’s degree in Computer Science, Software Engineering, Artificial Intelligence, Computer Engineering, or a related technical field, or equivalent practical experience.
Professional software engineering experience, including backend application development. The posting’s overview lists 7+ years of related experience; the Required Qualifications section lists 5+ years of professional software engineering experience.
Strong proficiency in Python and experience developing production-quality software.
Experience developing RESTful APIs and integrating distributed application services.
Hands-on experience integrating LLMs or generative AI into software applications.
Understanding of transformer-based models, inference APIs, prompt engineering, and LLM application development.
Experience with RAG concepts, including document ingestion, embeddings, vector search, and retrieval pipelines.
Experience with relational databases such as PostgreSQL.
Familiarity with Podman or Docker and experience with Linux-based development and application deployment.
Proficiency with Git, automated testing, debugging, and software development best practices.
Experience troubleshooting applications across multiple interconnected services.
Strong analytical, problem-solving, communication, and technical documentation skills.
Ability to work effectively with cross-functional teams and government stakeholders.

Nice To Haves

Experience with OpenWebUI configuration, integration, custom Functions, Tools, or application extensions.
Hands-on experience with vLLM, Ollama, Hugging Face Transformers, or other LLM inference technologies.
Experience serving or integrating open-weight LLMs in on-premises environments.
Experience with NVIDIA GPUs, CUDA, GPU memory management, and AI inference optimization.
Experience with PostgreSQL pgvector, vector similarity search, and semantic retrieval.
Familiarity with embedding models, rerankers, and cross-encoder architectures.
Experience with Docling, Tesseract, or document intelligence technologies.
Experience implementing production RAG pipelines and evaluation frameworks.
Familiarity with agentic AI frameworks, tool calling, and Model Context Protocol (MCP).
Experience developing or maintaining containerized microservice architectures.
Familiarity with Redis, PgBouncer, and asynchronous background processing.
Experience with Red Hat Enterprise Linux and Windows Server.
Experience with FastAPI, Pydantic, pytest, and related Python technologies.
Experience using Splunk for application monitoring and troubleshooting.
Familiarity with software supply-chain security, vulnerability scanning, and container hardening.
Experience supporting federal government systems, classified networks, or air-gapped deployments.
Knowledge of federal cybersecurity requirements, RMF, and ATO processes.

Responsibilities

Design, develop, test, and maintain Python applications, microservices, and APIs for an on-premises chatbot.
Extend and integrate OpenWebUI with enterprise AI services and backend systems, including custom Functions, Tools, and mission-specific integrations.
Integrate approved LLMs through OpenAI-compatible APIs and inference frameworks such as vLLM and Ollama; develop reusable components for model access, prompt management, response processing, and application workflows.
Collaborate with AI/ML engineers to improve inference performance, response quality, and application reliability, including support for document-understanding, OCR, and vision-language use cases.
Design, implement, and optimize RAG workflows for enterprise document search and question answering.
Build document-ingestion pipelines, including document chunking, metadata extraction, embedding generation, retrieval, and integration with tools such as Docling, Tesseract, and vision-language models.
Work with PostgreSQL and pgvector for vector search, semantic retrieval, and document indexing; develop APIs and background services for ingestion and knowledge-base management.
Implement testing and evaluation approaches for RAG accuracy, retrieval relevance, and answer quality.
Integrate and maintain inference services using vLLM, Ollama, and compatible model-serving frameworks; support approved LLMs, embedding models, rerankers, and vision-language models.
Assist with inference configuration and performance optimization, troubleshoot model and integration issues, and develop benchmarking and performance-testing utilities.
Support migration of DS Chat components to containerized services; build and maintain container images with Podman or Docker and Podman Compose configurations.
Integrate platform services including OpenWebUI, vLLM, Docling, PostgreSQL, pgvector, PgBouncer, and Redis; develop deployment, upgrade, recovery, health-check, logging, and service-management automation.
Troubleshoot container networking, authentication, storage, GPU access, and service connectivity; coordinate with infrastructure and cybersecurity teams on deployment consistency and security.
Research and prototype agentic AI capabilities, controlled tool-calling, multi-step workflows, and integrations with authorized enterprise APIs and services, including exploration of Model Context Protocol (MCP).
Design safeguards for AI-driven actions, including secure execution boundaries, permissions, audit logging, approval mechanisms, human oversight, and prevention of unauthorized actions.
Apply software engineering practices such as modular architecture, code reviews, documentation, and version control; develop automated unit, integration, regression, and performance tests.
Support CI/CD in disconnected and classified environments; troubleshoot production issues using logs, metrics, traces, and diagnostics; integrate operational monitoring with tools such as Splunk.
Assist with vulnerability remediation, dependency management, software supply-chain reviews, and container security assessments; produce technical documentation, deployment procedures, architecture diagrams, and operational runbooks.
Participate in Agile activities including sprint planning, technical design reviews, and demonstrations.
Design for secure on-premises, air-gapped government environments; implement authentication integration, role-based access controls, API authorization, data protection, and authorized internal network communications.
Support offline software packaging and deployment using preapproved dependencies, container images, and model artifacts; collaborate on security findings and RMF activities, and provide documentation and evidence for TRB/CCB reviews and ATO processes.
Follow approved procedures for handling sensitive and classified information.