About The Position

Microsoft Foundry helps developers and organizations build, deploy, and operate AI solutions at scale. The Foundry inference team builds infrastructure to make model training and inference faster, more reliable, and more efficient for customers. The team is hiring Software Engineer II and Senior Software Engineers to own a serving component from design through production, reducing customer latency and improving serving efficiency.

Requirements

  • Bachelor's degree in Computer Science or a related technical field and at least 2 years of technical engineering experience coding in languages including C, C++, C#, Java, JavaScript, or Python, or equivalent experience.
  • Ability to meet Microsoft, customer, and/or government security screening requirements, including passing the Microsoft Cloud Background Check upon hire or transfer and every two years thereafter.

Nice To Haves

  • At least 5 years of technical engineering experience coding in languages including C, C++, C#, Java, JavaScript, or Python, or equivalent experience.
  • Proven coding experience in C++, C#, Rust, Go, Java, or Python, with networking and concurrency fundamentals.
  • Proven experience with software engineering fundamentals, debugging skills, and clear cross-team communication.
  • Experience shipping and operating gateways, model-serving runtimes, or accelerator software.
  • Experience with containerized services, Kubernetes, and serving runtimes such as vLLM, SGLang, or TensorRT-LLM.

Responsibilities

  • Profile execution, memory, and dependency bottlenecks.
  • Improve cache-aware routing, admission control, continuous batching, or KV-cache management within the assigned area.
  • Validate time-to-first-token, tail latency, and cost per token through load and failure testing.
  • Build secure, tenant-isolated systems with automated testing, observability, reliable recovery, and safe releases.
  • Participate in operational support and improve services using production feedback.
  • Preserve model quality and numerical correctness.
  • Use reproducible benchmarks to improve throughput, latency, and cost.
  • Pursue near-linear throughput scaling where workloads permit, across agreed GPU and node configurations.
  • Build provider-aware serving stacks and integrate with supported training and post-training stacks for OpenAI, Anthropic, Microsoft AI (MAI), and open-source models.

Benefits

  • Certain roles may be eligible for benefits and other compensation; further information is available through Microsoft's corporate pay and benefits link.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service