Observability & Scale Engineer

Kyndryl•Frisco, TX
•$115,080 - $218,520

About The Position

The Observability & Scale Engineer owns observability and monitoring, utilization analysis, demand insight, capacity forecasting, scaling triggers, and recommendations for NVIDIA AI infrastructure and workloads. The role implements and maintains telemetry, establishes operational measures, and delivers dashboards and alerts for platform teams and leaders. It also analyzes workload demand and capacity trends, develops evidence-based forecasts and recommendations, and partners with technical, financial, and customer teams on planning. The engineer validates monitoring coverage during deployments, maintains baselines throughout the lifecycle, and improves data quality and automated reporting.

Requirements

  • Experience in infrastructure monitoring, observability, performance analysis, or capacity management.
  • Hands-on experience with Prometheus, Grafana, or comparable monitoring platforms.
  • Experience analyzing time-series data and translating trends into operational recommendations.
  • Working knowledge of Linux, containers, GPU infrastructure, and distributed platforms.

Nice To Haves

  • Experience with NVIDIA DCGM and GPU telemetry.
  • Experience with Run:ai Scheduler, Kubernetes scheduling, queue analysis, or AI workload demand.
  • Experience with Python, APIs, data analytics, and dashboard automation.

Responsibilities

  • Implement and maintain monitoring using DCGM, Prometheus, Grafana, and related telemetry sources.
  • Define service health, GPU, fabric, storage, queue, workload, and utilization measures.
  • Build dashboards, alerts, thresholds, and operational views for platform teams and leaders.
  • Analyze utilization, queue depth, workload demand, bottlenecks, and capacity trends.
  • Develop forecasts, scaling triggers, and evidence-based capacity recommendations.
  • Partner with platform operations, FDE, architecture, finance, and customer teams on demand and growth planning.
  • Validate monitoring coverage during deployment and maintain baselines through the lifecycle.
  • Improve data quality and automate recurring performance and capacity reporting.

Benefits

  • Medical and dental coverage
  • Disability benefits
  • Retirement benefits
  • Paid leave
  • Paid time off
  • Eligibility for Kyndryl’s discretionary annual bonus program, subject to plan terms
  • Employee learning programs and access to industry certifications
  • Company-wide volunteering and giving platform
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service