Cortex Real-Time Inference Engineering Lead

Conch Technologies•Concord, CA
•Onsite

About The Position

The Real-Time Inference Engineering Lead designs, builds, and operationalizes scalable, low-latency model-serving platforms for real-time AI and predictive decisioning use cases. The role leads the architecture, deployment, performance optimization, resiliency, and operational governance of online inference services across cloud and on-premises environments. The successful candidate combines AI platform engineering, distributed systems expertise, and operational excellence to deliver resilient, high-performance inference platforms that enable enterprise-scale real-time AI solutions.

Requirements

  • Experience with online inference and model-serving architectures.
  • Experience with real-time APIs and distributed systems design.
  • Experience with Kubernetes, container orchestration, and service mesh technologies.
  • Experience with autoscaling, capacity management, and workload optimization.
  • Experience with performance engineering, load testing, and latency optimization.
  • Experience with monitoring, observability, logging, tracing, and SLO management.
  • Experience with CI/CD, DevOps, and Infrastructure-as-Code practices.
  • Experience with reliability engineering, fault tolerance, and resiliency patterns.
  • Experience with cloud and on-premises platform operations.
  • Python, Java, Go, or similar backend development experience.
  • Experience with MLOps platforms and enterprise AI deployment frameworks.
  • Hands-on experience with real-time recommendation, fraud, risk, personalization, or predictive analytics platforms.

Nice To Haves

  • Familiarity with GPU-based inference, model optimization, and multi-cloud deployments.

Responsibilities

  • Design and implement highly available, low-latency model-serving architectures for real-time inference workloads.
  • Develop and standardize deployment patterns for scalable AI/ML services across Kubernetes-based platforms.
  • Lead API-based inference service design, integration, and lifecycle management.
  • Optimize model-serving performance through latency tuning, caching strategies, autoscaling, and resource management.
  • Establish monitoring, observability, SLOs/SLAs, alerting, and operational runbooks for production services.
  • Drive load testing, capacity planning, resiliency engineering, and disaster recovery readiness.
  • Integrate inference platforms with CI/CD pipelines to enable automated deployments and controlled releases.
  • Partner with Data Science, Platform Engineering, MLOps, and Infrastructure teams to ensure reliable production model operations.
  • Govern operational best practices, security, reliability, and performance standards for enterprise AI deployments.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service