Software Engineer, Platform

Arena Intelligence, Inc. • Bay Area, CA

1d • Hybrid

About The Position

Arena Intelligence is looking for a Software Engineer, Platform to build the core infrastructure that sits beneath our online evaluation systems — the AI gateways, automated arena runtimes, and serving layers that make real-world model evaluation possible at scale. This is a critical part of the Arena Service. Arenas are live, online systems: they route traffic across frontier models from many providers, handle bursty and unpredictable load, need to fail gracefully when upstream models do, and have to remain fair and consistent under all of it. The team exists to build foundational infrastructure for users that scales, is reliable, and makes the complexities of operating this infrastructure at scale disappear. The role requires a practitioner who's shipped this kind of infrastructure before and knows where the sharp edges are. The AI gateway is currently in private, gated launch with individual developers as primary users, with enterprise use cases to follow. Near-term work includes shipping direct model access and new infrastructure features, and potentially collecting usage traces and building leaderboards shared with lab partners. The role is for an early member of the infrastructure team, working closely with researchers, engineers, and product leadership. The work is zero-to-one in places and scale-it-up in others. The company moves fast and stays rigorous. This is a hands-on individual contributor role, not a tech lead or SRE function.

Requirements

• ~5+ years of backend engineering experience, with meaningful time spent on distributed systems, infrastructure, or developer-facing platforms.
• Proven delivery on meaningful backend work (for Senior level).
• Extensive, deep experience owning backend systems end to end (for Staff level).
• Strong proficiency in Go.
• Experience with LLM provider APIs (OpenAI, Anthropic, Google, etc.) and a working understanding of their challenges: streaming, token management, rate limits, model-specific quirks.
• A product-oriented mindset, considering the developer experience of APIs and asking 'why' before 'how'.
• Comfort with ambiguity and a startup environment where scope is fluid and context shifts.

Nice To Haves

• Cloud infrastructure experience (AWS, GCP, or Azure), Kubernetes, Terraform, and database systems like Postgres and Redis.
• Experience building API gateways, proxies, or developer tools (Bifrost, Kong, Envoy, Tyk, or custom).
• Background in AI/ML infrastructure, model serving, inference, or evaluation frameworks.
• Experience building enterprise-ready features: SSO, RBAC, audit logs, multi-tenancy.
• Experience building billing infrastructure around systems like Stripe, Metronome, and Orb.
• Familiarity with the modern AI infra stack (vLLM, LiteLLM, LangChain, etc.).

Responsibilities

• Build API-based products from the ground up.
• Design and implement low-latency, high-reliability APIs for leaderboards, models, and arenas.
• Solve hard streaming problems, handling SSE/streaming responses across heterogeneous providers, including partial failure recovery, mid-stream fallback, and consistent response normalization.
• Ship enterprise-grade infrastructure, including rate limiting, authentication, usage metering, cost attribution, audit logging, and SOC 2 compliance.
• Build deep observability by instrumenting infrastructure with distributed tracing, latency breakdowns, token-level usage tracking, and real-time dashboards.
• Build AI-centered products by integrating with the core evaluation platform, Arena data, and customer-specific benchmarks.
• Collaborate with the research team to turn novel ideas into full-featured products.
• Contribute to the backend of the Leaderboards and Evals platforms when needed, helping unify public and private data architectures.

Benefits

• Competitive compensation and equity aligned to markets.
• Comprehensive health and wellness benefits, including medical, dental, vision, and additional support programs.
• The opportunity to work on cutting-edge AI with a small, mission-driven team.
• A culture that values transparency, trust, and community impact.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed