Backend Engineer _ AI Gateway- Bilingual (Korean/ English)

Woongjin•Plano, TX

About The Position

We are building an AI Gateway — a high-performance, secure intermediary layer that routes, inspects, and governs all traffic between enterprise clients and LLM/AI service providers. This role requires deep expertise in network programming, async Python, and cloud infrastructure to design and operate a latency-sensitive, policy-driven gateway that handles massive concurrent streaming connections at scale.

Requirements

3+ years of Async Python & Network Programming experience, including asyncio, aiohttp/anyio, low-level TCP/UDP sockets, WebSocket & SSE servers, TLS handshake, connection-pooling, keep-alive, and zero-copy buffering.
Experience managing chunked transfer, bidirectional streaming, back-pressure, and high-throughput data paths.
Working familiarity with HTTP/2 & HTTP/3, gRPC, and custom protocol parsers.
3+ years of experience with FastAPI-based backend development, including Pydantic, dependency injection, and OpenAPI docs.
Experience with Docker, AKS/EKS/GKE deployment, HPA/VPA scaling, rolling updates, and basic service-mesh (Istio/Envoy).
Production use of Redis Streams, RabbitMQ, or Kafka.
Experience with PostgreSQL advanced query tuning, partitioning, and PgBouncer pooling.
Experience with HashiCorp Vault (dynamic secrets, transit encryption).
Experience with cloud networking and security, including L4/L7 load balancing, TLS termination, reverse-proxy (NGINX/Envoy), mTLS, and automated cert lifecycle (ACME/Let’s Encrypt).
Experience with Terraform modules and GitHub Actions/GitLab CI pipelines with container image scanning and canary/blue-green deployments.

Nice To Haves

Integration with CASB/DLP/WSS or similar edge-security platforms.
Centralized logging & audit trails.
Configuring Azure AD/Entra ID, Okta, SAML 2.0 or OIDC SSO flows.
Managing Conditional Access & SCIM provisioning.
Building prompt-engineering workflows, PII anonymisation, and monitoring Azure OpenAI (or comparable) usage, rate limits & cost.
Custom webhook/REST integrations, circuit-breaker patterns, rate-limiting, auto-retry and health-checking.
End-to-end monitoring with Datadog, Prometheus/Grafana or Azure Monitor.
Proactive cost-optimisation and 24/7 incident response (SLA/SLO).
Bilingual Korean.

Responsibilities

Design, develop and test production-grade REST/Streaming APIs using FastAPI.
Deploy and manage containerized applications on Kubernetes (AKS/EKS/GKE).
Implement asynchronous messaging and event-driven designs using Redis Streams, RabbitMQ, or Kafka.
Manage PostgreSQL databases, including query tuning and pooling.
Implement security measures using HashiCorp Vault for key management.
Configure and manage cloud networking and security, including load balancing and TLS termination.
Develop and maintain infrastructure as code using Terraform.
Build and manage CI/CD pipelines using GitHub Actions or GitLab CI.
Integrate with enterprise security gateways (CASB/DLP/WSS).
Configure IAM and SSO solutions (Azure AD/Entra ID, Okta, SAML 2.0, OIDC).
Build prompt-engineering workflows, PII anonymization, and monitor AI/LLM usage.
Implement API Gateway operations including webhooks, rate-limiting, and health-checking.
Set up end-to-end monitoring with tools like Datadog or Prometheus/Grafana.
Perform cost optimization and participate in 24/7 incident response.