Staff Software Engineer - Observability

Cerebras Systems•Sunnyvale, CA

16h

About The Position

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud inference services. This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation. Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference. About the team The Cerebras Inference team’s mission is to deliver the world’s most performant, secure, and reliable enterprise-grade AI service. We build and operate large-scale distributed systems that power AI inference at unprecedented speed and efficiency. Join us to help scale inference and accelerate AI. We’re looking for a Software Engineer focused on Observability to build and evolve the systems that give us deep visibility into large-scale, performance-critical production systems. You’ll design and implement metrics, logging, tracing, and alerting infrastructure that enables fast debugging, high reliability, and confident operation of complex distributed systems. This role sits at the intersection of platform engineering, distributed systems, and reliability. This is not a dashboards-only role — you’ll be writing production software, shaping internal platforms, and working closely with engineers across the stack.

Requirements

Strong experience in backend or systems software engineering
Proficiency in one or more of: Go, C++, Rust, Java, Python
Solid understanding of: Distributed systems
Networking fundamentals
Concurrency and performance tradeoffs
Hands-on experience with: Metrics, logs, and distributed tracing
Production monitoring and alerting
Familiarity with tools such as: OpenTelemetry
Prometheus
Grafana
Datadog / Elastic / Jaeger / Tempo (or similar)
Experience designing: High-signal alerts
Scalable telemetry pipelines
Service-level indicators and objectives

Nice To Haves

Experience in high-performance computing, AI/ML systems, or inference platforms
Hardware-aware observability (accelerators, GPUs, custom hardware)
Prior SRE or platform engineering background
Experience debugging large-scale production incidents
Building internal developer platforms or shared libraries

Responsibilities

Design and implement observability instrumentation across services and platforms
Build and maintain telemetry pipelines for metrics, logs, and traces at scale
Develop internal observability platforms, libraries, and tooling
Define and operationalize SLIs, SLOs, and alerting strategies
Partner with engineers to make systems debuggable by design
Reduce MTTR by enabling fast root-cause analysis during incidents
Create clear, actionable dashboards and alerts that reflect real system health
Balance telemetry signal vs cost, noise, and performance impact
Improve the developer experience around observability and debugging