LLM Engineer

UNC-Chapel Hill • Chapel Hill, NC

4d

About The Position

The LLM Engineer will design, build, and operate large language model-powered tools and applications across the institution. Working within the CAIO office and in close collaboration with the University Library, the engineer will contribute to two flagship initiatives: PromptLab, UNC’s shared generative AI service built on LibreChat, and Carolina Source, a proposed institutional knowledge platform with a governed retrieval layer and federated applications.

Requirements

• Bachelor’s degree in Computer Science, Data Science, Information Science, or a related field
• Demonstrated experience building and deploying production-grade LLM -powered applications
• Proficiency in Python and relevant frameworks including LangChain, LlamaIndex, Hugging Face Transformers, or equivalent
• Experience with RAG architectures, vector databases, and embedding models
• Familiarity with major LLM APIs including OpenAI, Anthropic, Google, and open-source alternatives
• Working knowledge of cloud platforms (Azure, AWS , or GCP ) and containerization (Docker, Kubernetes)
• Experience with web application development using JavaScript, TypeScript, Node.js, or React
• Experience with platform operations including monitoring, incident investigation, upgrades, and recovery

Nice To Haves

• Master’s degree in Computer Science, Artificial Intelligence, Computational Linguistics, Information Science, or a related field
• Experience with LibreChat, LiteLLM, or similar open-source generative AI platforms
• Experience designing or implementing enterprise search and knowledge retrieval platforms (Azure AI Search, Elasticsearch, OpenSearch, or equivalent)
• Familiarity with data governance frameworks, access control design, and permission-based retrieval architectures
• Experience building federated application architectures on top of shared knowledge or data platforms

Responsibilities

• Design, build, and operate large language model-powered tools and applications.
• Contribute to PromptLab, UNC’s shared generative AI service built on LibreChat.
• Contribute to Carolina Source, a proposed institutional knowledge platform with a governed retrieval layer and federated applications.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior