Principal Data and Platform Engineer, DGX Cloud

NVIDIA•Santa Clara, CA

About The Position

NVIDIA's DGX Cloud organization is seeking a lead expert in data and platform engineering to guide the development and management of the DGX Cloud Data Platform. The organization connects data across cloud service providers, NVIDIA Cloud Partners, and internal systems, and develops shared data functions that help teams publish, discover, join, and use reliable data in operational workflows. The Principal Data and Platform Engineer will turn complex operational needs into reliable platform services and shared datasets. The role shapes technical approaches for ingestion and orchestration, data agreements, quality monitoring, cataloging, and managed access; leads delivery and adoption across teams; sets production standards; and resolves technical issues across organizational boundaries. The primary implementation environment is Python, SQL, Databricks, and Spark.

Requirements

Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent experience.
More than 16 years of progressively responsible experience developing and operating production software, data platforms, databases, or distributed systems.
A sustained record of defining technical strategy and architecture for shared platforms spanning multiple teams, with accountability for delivery, adoption, and production outcomes.
Demonstrated judgment balancing reliability, performance, security, cost, and maintainability through testing and evaluation of alternatives.
Extensive hands-on experience with distributed data processing, analytical databases, production ETL, streaming systems, or backend platforms handling substantial data volumes.
Strong understanding of dependencies and failure modes across compute, storage, networking, and cloud services.
Strong software engineering fundamentals and production proficiency in Python or another backend or systems language, with willingness and ability to work primarily in Python and SQL.
Experience designing reusable frameworks and personally implementing and debugging critical code paths.
Deep SQL and data modeling expertise, including query execution, incremental processing, schema evolution, and consistency.
Experience connecting independently managed systems through shared identifiers, documented relationships, and reliable data contracts, with clear ownership and compatibility standards.
Ability to reason about idempotency, replay, late-arriving data, partial failure, and correctness across system boundaries.
Experience establishing production practices adopted by multiple teams, including CI/CD, data validation, observability, incident response, secure deployment, and recovery.
Demonstrated ability to lead complex production investigations and major migrations, using logs, metrics, traces, query plans, profiles, and controlled experiments to identify root causes and prevent recurrence.
Demonstrated success influencing technical direction across organizational boundaries, aligning engineering and business leaders around shared goals, and developing engineers for broader technical responsibilities.

Nice To Haves

Extensive experience developing and managing Databricks, Apache Spark, PySpark, Spark SQL, Delta Lake, or Unity Catalog platforms, including common features applied across engineering teams.
Experience delivering self-service data publishing, cataloging, lineage, data contracts, or certification systems with demonstrated adoption.
Experience combining operational data from various cloud providers and internal systems, including entity resolution, inventory reconciliation, lifecycle history, and coverage measurement.
Experience designing and operating Kafka or comparable streaming systems, including partitioning, consumer behavior, offset management, backpressure, replay, and schema compatibility.
Experience enabling AI agents to discover and use governed data, or building production agentic systems with tool integration, evaluation, permissions, observability, and failure recovery.

Responsibilities

Define the architecture and multiyear technical roadmap, including system boundaries, shared interfaces, and engineering standards across ingestion, processing, storage, and data serving.
Balance immediate operational needs with long-term reliability, maintainability, and cost.
Lead delivery across platform engineers, data producers, and consumers; translate ambiguous requirements into designs and milestones, resolve dependencies, and guide implementation through production adoption.
Build and operate reusable batch and streaming ingestion capabilities for service provider and internal data, including supported patterns for APIs, change data capture, streams, database connectors, and files.
Establish ingestion orchestration, retries, backfills, and recovery patterns.
Establish shared identifiers, data contracts, consistent modeling, and automated quality and certification capabilities in partnership with source owners.
Define dataset meaning, keys, supported joins, historical coverage, compatibility, and ownership; make quality failures actionable and dataset limitations visible to consumers.
Guide cataloging, lineage, and governed access capabilities so people and AI agents can discover datasets, understand definitions and relationships, and follow approved access procedures.
Make onboarding and operations repeatable through shared libraries, reference implementations, deployment automation, and self-service workflows; measure improvements in onboarding effort, dataset reuse, and manual toil.
Own production readiness and reliability standards, including SLOs, testing, observability, runbooks, rollback, and recovery.
Lead critical investigations across pipelines, applications, SQL engines, Spark, storage, networks, and cloud services.
Partner with security and infrastructure teams on service identities, least privilege, secrets management, environment isolation, auditability, and upgrades and migrations that preserve data access and production service.
Mentor senior engineers and technical leads, lead design reviews, and resolve technical disagreements through evidence and clear tradeoffs.
Partner with leadership on priorities, progress, operational risks, and investment decisions.