Staff Software Engineer

Coupang•Seattle, WA
•$152,000 - $261,000

About The Position

Coupang is hiring a Staff Backend Engineer to work with platform and product leaders on complex infrastructure challenges. The role will architect and build the company’s next-generation AI Inference Gateway, supporting mission-critical machine learning and generative AI workloads at scale across cloud and on-prem environments. It involves developing scalable, secure, reliable, high-performance platform services and providing technical direction across teams.

Requirements

  • At least 6 years of professional software development experience.
  • At least 4 years of experience designing and operating large-scale distributed systems in production.
  • Strong hands-on programming expertise in one or more of Go, Java, or Python.
  • Proven track record building highly available, mission-critical platform or infrastructure services.
  • Experience designing API platforms, service gateways, service mesh, or large-scale networking infrastructure.
  • Deep understanding of microservices architecture, distributed systems design, and cloud-native technologies.
  • Experience with Kubernetes and containerized workloads in production environments.
  • Experience operating services on AWS, Azure, or GCP.
  • Strong understanding of observability, reliability engineering, capacity planning, and production operations.

Nice To Haves

  • Experience building AI/ML inference platforms, LLM gateways, model serving infrastructure, or GPU-accelerated workloads.
  • Expertise in Kubernetes ecosystem technologies such as Gateway API, Ingress Controllers, service mesh (Istio, Linkerd, Envoy), and platform networking.
  • Experience with inference serving frameworks such as vLLM, Triton Inference Server, TensorRT-LLM, Ray Serve, KServe, SGLang, or similar technologies.
  • Strong understanding of high-performance networking, gRPC, HTTP/2, streaming protocols, API gateways, and service proxy architectures.
  • Experience designing large-scale traffic management systems, including routing, retries, circuit breaking, rate limiting, and request prioritization.
  • Experience optimizing latency, throughput, and resource utilization for CPU and GPU workloads.
  • Familiarity with GenAI and LLM ecosystems, including model deployment, prompt routing, RAG systems, model observability, and AI governance.
  • Experience with distributed data systems such as Kafka, Cassandra, Redis, MongoDB, or similar technologies.
  • Strong understanding of concurrency, synchronization, asynchronous programming, and non-blocking I/O.

Responsibilities

  • Architect and build the AI Inference Gateway platform for large-scale machine learning and generative AI workloads.
  • Design and develop request routing, model endpoint abstraction, traffic shaping, load balancing, failover, caching, and policy enforcement for inference services.
  • Drive the technical vision and roadmap for scalable, secure, reliable AI inference infrastructure across cloud and on-prem environments.
  • Develop infrastructure components in Go, Java, or Python, emphasizing performance, resiliency, and operational excellence.
  • Design multi-tenant capabilities such as authentication, authorization, quota management, cost attribution, rate limiting, and governance controls.
  • Partner with ML Platform, Model Serving, Data, and Product Engineering teams to enable AI workload deployment and operation.
  • Lead architecture and design reviews, raise engineering standards, and mentor senior engineers across teams.
  • Optimize performance, latency, throughput, and infrastructure efficiency for large-scale inference workloads.
  • Define observability standards for metrics, tracing, logging, and SLO-based operations.
  • Investigate production issues, lead root-cause analysis, and implement long-term architectural solutions.
  • Collaborate with engineering leaders to establish common platform standards and enable AI innovation across Coupang.

Benefits

  • Medical, dental, vision, and life insurance, plus AD&D insurance.
  • Flexible Spending Accounts (FSA) and Health Savings Account (HSA).
  • Long-term and short-term disability coverage.
  • Employee Assistance Program (EAP).
  • 401K plan with company match.
  • 18–21 days of paid time off per year, based on tenure.
  • 12 public holidays.
  • Paid parental leave.
  • Pre-tax commuter benefits.
  • Free electric car charging station.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service