AI Platform Engineer

ZipStaff•Cary, NC
•Remote

About The Position

ZipStaff is hiring an AI Platform Engineer for a client described as a Fortune 10 healthcare and pharmacy leader. The engineer will build and operate a production agentic AI platform that other engineers can build on, and own its reliability, security, and cost. This is a fully remote contract-to-hire role based in Cary, NC. Applicants must be U.S. citizens or green card holders; sponsorship is not available.

Requirements

  • U.S. citizen or green card holder; sponsorship is not available.
  • At least 8 years in software or platform engineering, with hands-on production ownership of AI or ML infrastructure.
  • Experience building and operating agentic or LLM systems in production, not only prototyping them.
  • Experience with LLM and agent orchestration using LangGraph, LangChain, LlamaIndex, CrewAI, AutoGen, or Google Agent Development Kit (ADK).
  • Model-serving and gateway experience with LiteLLM, Portkey, Kong AI Gateway, TrueFoundry, or Vertex AI Model Garden, including routing, fallback, rate limiting, token and usage capture, serving open-weight models through vLLM, TGI, Ollama, or Vertex endpoints, and LoRA/QLoRA adapters.
  • RAG experience with Vertex Vector Search, Pinecone, Weaviate, pgvector, or Elasticsearch.
  • Evaluation and LLMOps experience with RAGAS, DeepEval, Promptfoo, or OpenAI Evals, and experiment tracking with MLflow, LangSmith, Langfuse, Arize Phoenix, or Braintrust.
  • Guardrails and AI security experience with NeMo Guardrails, Guardrails AI, Model Armor, or Lakera.
  • Observability experience, including OpenTelemetry tracing for LLM and agent calls.
  • Experience with Kubernetes and containers.
  • Terraform, CI/CD, and SRE practice for production AI services.
  • Strong Python skills.

Nice To Haves

  • Go or Java for platform services.
  • GCP experience, including GKE, Vertex AI, or Cloud Run.

Responsibilities

  • Build and operate a production agentic AI platform, including orchestration, model gateway, agent registry, retrieval, guardrails, evaluation, and observability.
  • Implement production multi-agent orchestration, including state management and retry and fallback design.
  • Operate model-serving and gateway capabilities, including routing, fallback, rate limiting, token and usage capture, and serving open-weight models.
  • Build retrieval-augmented generation systems, including vector stores, embedding models, chunking, hybrid and semantic search, reranking, and grounding-quality evaluation.
  • Implement agent infrastructure, including Model Context Protocol (MCP), tool and function-calling contracts, agent registries, permission enforcement, and separation of reasoning from acting agents.
  • Design evaluation and LLMOps practices, including evaluation harnesses, offline and online evaluation, regression suites, drift and hallucination detection, and experiment tracking.
  • Apply guardrails and AI security measures, including defenses against prompt injection, jailbreaks, and data leakage, plus PII detection and redaction in prompts and responses.
  • Instrument observability for LLM and agent calls, including token, latency, and cost telemetry, and monitor quality signals that feed alerting.

Benefits

  • Health insurance
  • Dental insurance
  • Vision insurance
  • 401(k) with matching
  • Paid time off
  • Referral program
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service