About The Position

The Backend Software Engineer, ML Inference will own the design, development, and scaling of the core systems that serve Prior Labs' foundation models. The role involves making architectural decisions, establishing engineering practices, and shaping the technical direction of production model serving. TabPFN performs learning in the forward pass and handles datasets with millions of rows, creating engineering challenges around fast, reliable, and cost-effective inference. The engineer will work directly with AI researchers to translate advanced models into scalable production systems. The position offers autonomy and the opportunity to specialize in inference optimization or core backend systems, or to grow into a technical leadership role. Models are served through a managed API, within customer VPCs, and as self-hosted open weights.

Requirements

  • At least 3 years of professional experience in machine learning and backend engineering, with a proven track record of managing production infrastructure.
  • Experience deploying and operating machine learning models in production, with a strong understanding of how models behave and fail under real traffic.
  • Strong hands-on experience designing and building production-grade APIs and backend services.
  • Significant experience building and operating services on cloud platforms; GCP is preferred.
  • Strong hands-on experience with Infrastructure as Code (IaC), using tools such as Terraform.
  • Significant experience with containerization and orchestration technologies such as Docker and Kubernetes.
  • Proficiency in Python.

Nice To Haves

  • Experience building or managing infrastructure specifically for machine learning, including MLOps, model serving frameworks, feature stores, or data pipelines.
  • Deep experience with inference optimization, including quantization, compilation, dynamic batching, caching strategies, and the interplay with distributed computing.
  • Experience with PyTorch and FastAPI.
  • Contributions to relevant open-source projects.

Responsibilities

  • Own the end-to-end inference serving path, including latency, throughput, batching, memory behavior on large inputs, and cost per prediction.
  • Design robust, scalable, and secure backend systems and production-grade APIs for serving foundation models.
  • Develop high-quality, maintainable code for core backend services.
  • Design, deploy, and manage cloud infrastructure with a focus on reliability, monitoring, observability, and cost-efficiency.
  • Implement secure, GDPR-compliant systems for data storage, access control, usage tracking, and quota management.
  • Promote high standards for testing, CI/CD, documentation, and security within the engineering team.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service