About The Position

NVIDIA’s AI Infrastructure Engineers build systems, tools, and data infrastructure for AI cloud services, helping engineering teams innovate while identifying, tracking, and mitigating technical risks. The Senior Data Engineer, Operational Excellence will combine operational excellence practices with hands-on data engineering expertise, leading high-severity incident response and building the pipelines and analysis needed to identify reliability improvements.

Requirements

  • At least 5 years of experience in data engineering, incident management, or a closely related engineering role.
  • Bachelor’s degree or equivalent experience.
  • Strong SQL and Python data engineering skills, including experience building and maintaining pipelines for operational reporting.
  • Proven ability to analyze incident trends and present findings that leadership uses to prioritize reliability work.
  • Hands-on experience with incident management platforms such as PagerDuty, Rootly, or ServiceNow and dashboarding tools such as Tableau, Grafana, or Databricks.
  • Experience facilitating blameless postmortems or operational reviews and holding teams accountable for follow-up actions.
  • Clear, calm communication with engineers, leadership, and stakeholders during high-pressure incidents.
  • Hands-on experience using AI tools to accelerate analysis, debugging, and documentation.

Nice To Haves

  • Experience applying statistical or machine learning methods to operational data, such as anomaly detection, forecasting, or clustering incident text to find related failures.
  • Formal experience serving as an incident commander.
  • Background building or contributing to internal reliability platforms for incident tracking, telemetry, or service cataloging.
  • Experience operating in globally distributed, 24/7 environments with strict SLOs tied to external customer impact.
  • Track record of running incident simulations or game days that measurably improved team readiness.
  • Familiarity with distributed systems, cloud infrastructure, or GPU-based services sufficient to follow root cause discussions in depth.
  • Experience using LLMs and prompt engineering to automate incident summaries, trend detection, or knowledge retrieval.

Responsibilities

  • Serve as incident commander during high-severity incidents, coordinating responders across teams through mitigation and recovery.
  • Build and maintain data pipelines that collect incident, alert, and change data into a clean, queryable source for reliability reporting.
  • Analyze incident trends to identify recurring failure patterns, regressions in TTD, TTM, and TTR, and services contributing most to customer impact.
  • Convert trend findings into prioritized recommendations for engineering leadership and track corrective actions through completion.
  • Strengthen operational excellence practices, including runbooks and on-call readiness, based on data findings.
  • Run postmortem reviews and operational health forums, presenting incident data and driving clear ownership and follow-up.

Benefits

  • Equity
  • Benefits
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service