Director of Observability & Developer Excellence

Comcast • Reston, VA

5d

About The Position

Comcast’s Technology + Product (T+P) organization works at the intersection of media and technology. Our innovative teams are continually developing and delivering products that transform the customer experience. About the Role: We are seeking a Director of Observability to lead observability as a core pillar of our Developer Experience organization. This role focuses on empowering engineers with fast, actionable feedback about their systems—so they can build, debug, and operate software with confidence. You’ll be part of team that dives observability strategy, platforms, and practices that enable developers to understand system behavior, detect issues early, and resolve problems independently. Will also be responsible for leveraging AI to make observability services much easier to use. Success in this role means observability is intuitive, standardized, and seamlessly integrated into how developers design, ship, and run services.

Requirements

• 15+ years of experience in software engineering, platform engineering, SRE, or related roles
• 5+ years leading technical teams or large cross-functional initiatives
• Deep expertise in observability fundamentals: metrics, logs, traces, alerting, SLIs/SLOs, and incident analysis
• Strong hands-on experience building open-source observability platforms at scale
• Proven expertise with OpenTelemetry (instrumentation, collectors, pipelines, and standards) in production environments
• Experience operating and scaling VictoriaMetrics (or VictoriaMetrics cluster) for high-cardinality, high-volume metrics workloads
• Strong understanding of cloud-native and distributed systems (Kubernetes, microservices, service meshes)
• Demonstrated ability to design developer-first platforms that emphasize self-service, usability, and fast feedback

Nice To Haves

• Experience standardizing observability using vendor-neutral, open standards
• Familiarity with complementary open-source tooling (e.g., Clickhouse, Elastic, Grafana, Prometheus, Tempo, Loki, Jaeger)
• Experience migrating from proprietary observability tools to open-source or hybrid models
• Strong product mindset applied to internal developer platforms
• Experience balancing developer experience, system reliability, and cost optimization at scale

Responsibilities

• DevEx Strategy & Leadership Define the observability vision aligned with developer productivity and cognitive load reduction. Lead and grow the team delivering developer‑first observability platforms and tooling.
• Developer-Centered Observability Deliver low‑configuration, self‑service observability with clear standards for metrics, logs, traces, dashboards, and alerts. Ensure developers can quickly determine service health, changes, and root causes across all workflows.
• Platform Architecture & Tooling Build scalable cloud/hybrid observability platforms with vendor‑neutral instrumentation (e.g., OpenTelemetry). Streamline and standardize tooling for usability, consistency, and cost efficiency.
• Reliability Enablement Partner with SRE/Infra to strengthen reliability while enabling teams to own SLIs/SLOs. Improve incident detection, debugging speed, and learning through better telemetry and tooling.
• Collaboration & Influence Embed observability early in system and service design through cross‑functional partnership. Communicate health, maturity, and tradeoffs clearly while influencing org-wide standards.
• Governance & Measurement Set guardrails for telemetry quality, retention, and privacy while maintaining team autonomy. Drive adoption through documentation, golden paths, training, and DevEx‑focused success metrics.

Benefits

• Medical & Dental
• 401(k) Savings Plan
• Generous paid time off
• Life Milestones - from adoption assistance, childcare resources, pet insurance, and more, Comcast supports you at all life stages.
• Courtesy Services - We offer all of our full-time employees in serviceable areas free digital TV and internet.
• Discount tickets for Universal Resorts, including theme park tickets and onsite hotel rooms.

What This Job Offers

• Job Type: Full-time
• Career Level: Director