Senior Engineering Manager, Observability Engineering

Temporal Technologies•Seattle, WA

About The Position

Temporal is seeking a Senior Engineering Manager to lead its Observability Engineering team. This role owns both the internal telemetry platform used to operate Temporal’s multi-tenant service and the customer-facing observability capabilities that help customers debug workflows. The position calls for a leader experienced in running observability at scale, managing telemetry cost and cardinality, and taking accountability for live-site outcomes on a reliability-critical platform. The manager will set the team’s technical roadmap and delivery priorities; lead, hire, and develop engineers; and guide technical decisions. They will own the ingestion, storage, query, and visualization path for high-cardinality telemetry across a multi-tenant, multi-cloud fleet, as well as alerting and incident-response tooling. In partnership with Product and SDK teams, they will help shape the metrics, traces, and workflow-level diagnostics Temporal exposes to customers, including through direct work with design partners. The role also owns observability unit economics, cardinality budgets, tool consolidation, and vendor relationships. The manager will partner across Engineering, Product, Security, and Support, communicate technical tradeoffs and risks to senior leaders and other partners, and use incidents and customer feedback to drive lasting improvements.

Requirements

15+ years in software and/or infrastructure engineering, including 7+ years of people management, with demonstrated ownership of delivery and live-site outcomes.
Proven experience leading engineering teams that build and operate observability, monitoring, or telemetry platforms at scale, with strong operational practices.
Demonstrated success building and operating production observability platforms, including hiring, coaching, performance management, and delivery.
Experience hiring, leveling, and managing performance.
Strong distributed-systems expertise and hands-on judgment to guide architecture and execution.
Experience operating multi-tenant infrastructure on which other organizations’ production workloads depend.
Strong leadership and coaching skills, with the ability to grow engineers and build a healthy, accountable, high-ownership team, including through reorgs, acquisitions, or platform migrations.
Ability to set priorities, deliver incremental improvements with incomplete information, make tradeoffs explicit, and adapt plans to changing customer and operational needs.
Command of observability building blocks—metrics, logs, traces, profiling, and alerting—and the tradeoffs between them at scale.
Knowledge of time-series and telemetry data fundamentals, including cardinality control, retention and downsampling, query performance, and hot/cold storage tiering.
Live-site reliability expertise, including SLA/SLO definition, on-call, incident management, and postmortem-driven continuous improvement.
Ability to review and improve technical artifacts, including design documents and code reviews, across a distributed-systems codebase.
Excellent communication skills, including the ability to partner across engineering, product, and leadership and translate customer feedback into a roadmap.
Bachelor’s degree in Computer Science or a related field, or equivalent practical experience.

Nice To Haves

Advanced degree.
Experience applying AI/LLM techniques to triage, anomaly detection, or incident summarization, with sound judgment about their benefits and limitations.
Deep OpenTelemetry experience, including collector architecture, semantic conventions, and instrumentation strategy.
Production experience with the open-source observability stack, including Prometheus, Mimir/Thanos/Cortex, Grafana, Loki, Tempo, or ClickHouse.
Experience building observability as a product for external developers, not only as internal tooling.
Kubernetes internals and multi-cloud delivery experience across AWS, GCP, and Azure.
Experience with vendor consolidation or migration programs, such as commercial APM/SaaS to open source or the reverse, with measurable cost and reliability outcomes.
Developer experience or engineering productivity background, treating internal engineers as customers with a measurable experience.
Prior experience at a developer infrastructure or open-source-based company where the community is part of the feedback loop.

Responsibilities

Own the observability team’s technical roadmap and delivery, aligned with broader engineering priorities.
Evaluate build, buy, and open-source options, balancing reliability, customer needs, and cost.
Lead, hire, and grow a high-ownership engineering team; guide technical decisions through design and code reviews, coach engineers, and remove obstacles to their work.
Own the ingestion, storage, query, and visualization path for high-cardinality telemetry across a multi-tenant, multi-cloud fleet.
Enable self-serve capabilities that support a growing Engineering organization.
Own alerting and incident-response tooling end to end; improve detection quality and routing, create clear and actionable runbooks, and make on-call sustainable.
Turn incident reviews into lasting reliability improvements.
Partner with Product and SDK teams on customer-facing metrics, traces, and workflow-level diagnostics.
Work directly with design-partner customers and incorporate customer feedback into the roadmap.
Own observability unit economics, including cost per ingested series, query, and namespace; set cardinality budgets, consolidate tools where appropriate, and manage vendor relationships against value criteria.
Partner with Engineering, Product, Security, and Support to resolve competing priorities and deliver shared outcomes.
Explain technical tradeoffs and risks clearly to senior leaders and non-specialist partners.
Set priorities and deliver incremental improvements amid incomplete information, adapting plans as customer and operational needs change.
Define and support live-site reliability practices, including SLAs/SLOs, on-call, incident management, and postmortem-driven continuous improvement.
Review and raise the bar on technical artifacts, including design documents and code reviews, across a distributed-systems codebase.