Expert RAG Engineer

Salvo Software

3d•Remote

About The Position

Design, build, and own the retrieval-augmented generation systems behind AI products. This senior, hands-on role is responsible for architecting production-grade RAG pipelines end to end, including ingestion, chunking, embeddings, hybrid retrieval, reranking, evaluation, and serving open-source LLMs such as Ollama and llama.cpp in cloud, on-premises, and air-gapped environments. The role defines RAG architecture standards and makes technical decisions about retrieval quality, latency, and cost, while providing technical guidance to AI Developers. It works with engineering, product, and client stakeholders to turn complex knowledge sources into reliable, measurable, and secure AI features. The position also supports deployment and scaling of RAG services, AI-service CI/CD, observability, and event-driven ingestion and re-indexing. It requires hands-on ownership across the stack, from retrieval and model serving to infrastructure troubleshooting.

Requirements

6+ years of experience in backend, machine learning, or infrastructure work.
2+ years building production RAG systems, including end-to-end architecture, hybrid retrieval, reranking, chunking, indexing, and evaluation frameworks.
Expert-level Python programming.
Deep knowledge of embeddings, vector databases, and metrics-driven evaluation.
Hands-on experience serving and optimizing open-source LLMs, including Ollama and llama.cpp, with quantization.
Experience with GPU/CUDA troubleshooting and performance tuning.
Strong proficiency in Microsoft Azure, AWS, Docker, Kubernetes, Linux, CI/CD pipelines, and Model Context Protocol (MCP).
Strong communication, technical leadership, and ownership skills.
Ability to define architectural standards and collaborate effectively in a distributed, remote environment.

Nice To Haves

Azure DevOps experience is preferred; GitHub Actions and GitLab CI are also used.

Responsibilities

Design and own end-to-end RAG architectures, from document ingestion to grounded, cited responses.
Build advanced retrieval strategies, including hybrid dense and sparse search, reranking, query rewriting and expansion, metadata filtering, and multi-step or agentic retrieval.
Define chunking and indexing strategies for structured and semi-structured sources, including XML/XSD, Office formats, PDFs, and scanned documents.
Select, benchmark, and fine-tune embedding models, and manage vector databases at scale.
Build evaluation frameworks for retrieval and generation quality, including recall@k, MRR, nDCG, faithfulness, and answer relevance; run evaluations continuously as part of CI/CD.
Reduce hallucinations using grounding, citations, guardrails, and confidence handling.
Implement permission-aware and multi-tenant retrieval so users can access only authorized data.
Develop Model Context Protocol (MCP) integrations that expose RAG systems and internal data sources to LLMs and agents.
Optimize LLM serving with Ollama and llama.cpp, including GGUF quantization, GPU/CUDA tuning, batching, caching, and offline or air-gapped deployments.
Set RAG architecture standards, reference implementations, and best practices for the AI department.
Review designs and code, and provide technical guidance to AI Developers.
Translate client and product requirements into retrieval strategies with clear quality, latency, and cost targets.
Evaluate emerging techniques such as GraphRAG, long-context versus retrieval trade-offs, and agentic RAG, and recommend what to adopt.
Deploy and scale RAG services on Microsoft Azure and AWS using Docker and Kubernetes.
Maintain CI/CD pipelines for AI services.
Build observability for RAG systems, including tracing, retrieval diagnostics, and latency, cost, and quality dashboards.
Support event-driven ingestion and re-indexing pipelines using Apache Kafka.
Troubleshoot across the stack and investigate root causes rather than patching symptoms.
Collaborate with engineering, product, and client stakeholders in a distributed, remote environment.