LLM Inference & GPU Systems Consultant

Delan Associates • Charlotte, NC

4d • Onsite

About The Position

We are seeking an AI Infrastructure Runtime Engineer to build and maintain large-scale on-prem LLM infrastructure. This is an enterprise private GenAI environment running on NVIDIA H200 GPU clusters and an OpenShift AI deployment ecosystem. You will manage production inference internally, including self-hosting open-source LLMs like Llama. We are focused exclusively on inferencing; this role involves no model training infrastructure or fine-tuning pipelines.

Requirements

• 8+ years experience working as an LLM Systems Engineer or AI Infrastructure Runtime Engineer.
• 8+ years hands-on experience with NVIDIA H200 clusters and runtime optimization techniques (KV Cache, prefill/decode).
• Proficiency in OpenShift AI and GPU orchestration tools like RunAI.
• Strong experience with modern inference frameworks, specifically vLLM and TensorRT-LLM.
• Proven track record managing the Hugging Face deployment lifecycle.

Responsibilities

• Drive extreme runtime efficiency and optimization for the token generation pipeline.
• Specifically manage prefill/decode optimization and KV cache management.
• Deploy and manage inference engines including vLLM and TensorRT-LLM.
• Optimize GPU throughput tuning, batching strategies, and latency optimization.
• Manage workload orchestration using RunAI and Kubernetes GPU orchestration.
• Oversee the complete Hugging Face model lifecycle, including model onboarding, deployment, and retirement.
• Operate and maintain the OpenShift AI ecosystem as the primary container platform for GenAI workloads.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed