Staff Site Reliability Engineer
About The Position
Join the Cloud Infrastructure Team as a technical leader driving reliability, automation, and scalability across the systems running Sight Machine's platform. You'll operate at the intersection of classic SRE discipline which include IaC, CI/CD, observability, incident response and the emerging demands of running agentic AI systems in production: LLM gateways, agent orchestration, and the operational patterns that come with non-deterministic workloads. This is a senior level IC role. You'll help set and drive technical direction for infrastructure and reliability practices across teams, mentor senior engineers, and be a primary escalation point for the org's hardest systems problems while still being hands-on with code, infrastructure, and incidents. Success requires deep technical range, sound judgment on risk vs. customer impact, and the ability to influence architecture decisions across Development Engineering without formal authority.
Requirements
- Demonstrated experience designing, building, or operating agentic AI/LLM-based systems in production, held to the same quality-first, test-driven rigor as traditional infrastructure code, not just prototype-grade work
- Embody a quality-first and security-first culture in all that you do
- 10+ years of experience with Kubernetes/Docker in at least one top-tier cloud provider (Azure, GCP, AWS), including production-scale multi-tenant or multi-cluster environments
- 10+ years coding experience (Python, Go, Java, or similar) with a track record of building tools/platforms used by other engineers, not just scripts
- 10+ years with IaC and CI/CD tooling (Terraform/OpenTofu, FluxCD or similar GitOps tooling, Jenkins/GitHub Actions)
- Strong, provable Linux and networking fundamentals (TCP/IP and application-layer)
- Practical experience integrating or operating LLM/agentic AI systems in a production context this can be API-based orchestration, LLM gateways, or agent frameworks
- A track record of authoring technical documentation (design docs, ADRs, runbooks) that other engineers actually use
- Demonstrated mentorship of other engineers, without needing formal management authority to do it
- Strong bias for action over endless planning, hands-on, has made mistakes, learned from them, and can weigh risk vs. customer impact under pressure
- Clear, empathetic communicator, comfortable pushing back on architecture decisions across teams
- Operational experience with monitoring/alerting systems (Prometheus, Grafana, Loki, Sentry, Signoz or equivalents)
- Deep understanding of cloud performance, able to diagnose and resolve bottlenecks others can't
Nice To Haves
- Experience with elements of our current tech stack are a plus: Kubernetes, FluxCD, Terraform, Helm Charts, Prometheus, Elasticsearch, Python, Java, Kafka, Postgres, and Jenkins
- Previous experience or a keen interest in industrial IoT, analytics, or manufacturing a plus
Responsibilities
- Champion an agentic-AI-first engineering mindset: identify where AI-driven automation and agent-based tooling can replace manual toil, and hold that work to the same quality, testing, and reliability bar as any other production system
- Evolve reliability practices for meeting reliability SLO’s, error budgets, drive incident postmortems to systemic (not just symptomatic) fixes, and lead reliability reviews for new services before they hit production
- Troubleshoot and resolve the org's most complex, cross-layer systems problems CI/CD, container orchestration, networking, OS, cloud resources, databases, and increasingly, agentic AI/LLM orchestration layers
- Design, build, and operate the infrastructure supporting agentic AI workloads, LLM gateway routing, agent orchestration frameworks, monitoring of non-deterministic/AI-driven services, and the operational tooling needed to run them reliably at scale
- Architect and instrument monitoring, alerting, and observability infrastructure for critical services, with an eye toward what "critical" means for AI-driven systems specifically
- Author and continuously improve operational runbooks and automation, increasingly incorporating agentic/AI-assisted tooling (e.g., automated triage, AI-assisted incident response) where it measurably reduces toil
- Design and build internal platforms and developer tooling that other engineers build on top of
- Participate in on-call coverage and help evolve the program as we scale including escalation paths and reducing avoidable pages through better automation
- Bring a startup mindset of daily engagement: staying close to what's breaking, what customers are hitting, and where the team needs help, even outside a formal ticket or rotation
- Mentor senior and mid-level engineers; act as a technical sounding board across teams
- Proactively identify and drive cross-team initiatives that improve stability, reliability, and availability, this is expected to be self-directed, not assigned
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
