Autonomous Platform (Horizon AI)

STN•San Francisco Bay Area, CA

About The Position

Horizon AI is building an operating system for how organizations run themselves, beginning with autonomous data centers. Its AI agents monitor STN’s GPU infrastructure, reason about conditions, and take actions when risks are bounded and outcomes can be verified. Higher-risk decisions are presented to humans, with the goal of expanding autonomous action based on evidence. Data centers are the initial proving ground for technology intended to generalize to other settings. The Principal-level role is a hands-on senior engineering position responsible for making the platform dependable enough to support autonomous action and for maintaining a broad view of its architecture. The role also serves as technical deputy, making decisions when the Chief Architect is unavailable. The initial priority is hardening agent dispatch reliability so the platform can safely expand the actions agents are trusted to perform. The current platform has dispatch reliability gaps, limited failure visibility, no caching or event-streaming layer, synchronous service communication, single-instance deployment, no infrastructure-as-code, and runtime concurrency limits. The successful candidate will address these foundations, determine and execute an appropriate orchestration and scaling approach, and own cross-cutting engineering practices. The role also rotates through Systems and SRE embeds.

Requirements

  • A long record of shipping and hardening complex distributed systems as a hands-on senior-level engineer.
  • Experience taking a rapidly built system to production dependability, including idempotency, circuit breaking, backpressure, and observable degradation.
  • Breadth across Python and Java, with comfort working on backend services, data infrastructure, and deployment.
  • Sound judgment about orchestration and scaling grounded in operating systems.
  • Ability to reduce novel problems to working code quickly.

Nice To Haves

  • Production-scale Kubernetes experience.
  • Event-streaming experience.
  • Experience with CI at scale.

Responsibilities

  • Improve dispatch reliability with circuit breakers, appropriate retry semantics, health-aware routing, and error handling that surfaces failures.
  • Add a caching layer and an event-streaming backbone.
  • Choose and implement an orchestration approach, evaluating the existing setup, Kubernetes, or an intermediate option.
  • Introduce infrastructure-as-code and dependable deployment practices.
  • Refactor toward per-session isolation so concurrent agent runs are safe.
  • Own cross-cutting platform concerns, including service scaffolding standards, testing strategy, CI, and platform observability.
  • Rotate through Systems and SRE embeds.
  • Serve as technical deputy and make technical decisions when the Chief Architect is unavailable.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service