Principal Data and Platform Engineer, DGX Cloud

NVIDIA•Santa Clara, CA

About The Position

NVIDIA’s DGX Cloud organization is seeking a lead expert in data and platform engineering to guide the development and management of the DGX Cloud Data Platform. The platform connects data across cloud service providers, NVIDIA Cloud Partners, and internal systems, and provides shared capabilities that help teams publish, discover, join, and use reliable data in operational workflows. The engineer will turn complex operational needs into reliable platform services and shared datasets, shape technical approaches across ingestion and orchestration, data agreements, quality monitoring, cataloging, and managed access, and lead adoption and delivery across teams. The role is intended to reduce repeated integration work, improve confidence in operational data, and make the platform easier to adopt and operate. The primary implementation environment is Python, SQL, Databricks, and Spark. The role defines platform architecture, a multiyear technical roadmap, system boundaries, shared interfaces, and engineering standards across ingestion, processing, storage, and serving. It leads delivery with platform engineers, data producers, and consumers; develops reusable batch and streaming ingestion capabilities; establishes shared identifiers, data contracts, modeling, quality, and certification practices; and guides cataloging, lineage, and governed access. It also makes onboarding and operations repeatable through shared libraries, reference implementations, deployment automation, and self-service workflows. The Principal Engineer owns production readiness and reliability standards, leads critical cross-system investigations, and partners with security and infrastructure teams on secure, auditable access and operational changes. The role mentors senior engineers and technical leads, leads design reviews, resolves technical disagreements using evidence and tradeoffs, and partners with leadership on priorities, progress, risks, and investment decisions.

Requirements

Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent experience.
More than 16 years of progressively responsible experience developing and operating production software, data platforms, databases, or distributed systems.
A sustained record of defining technical strategy and architecture for shared platforms spanning multiple teams, with accountability for delivery, adoption, and production outcomes.
Demonstrated judgment balancing reliability, performance, security, cost, and maintainability through testing and evaluation of alternatives.
Extensive hands-on experience with distributed data processing, analytical databases, production ETL, streaming systems, or backend platforms handling substantial data volumes.
Strong understanding of dependencies and failure modes across compute, storage, networking, and cloud services.
Strong software engineering fundamentals and production proficiency in Python or another backend or systems language, with the ability and willingness to work primarily in Python and SQL.
Experience designing reusable frameworks and personally implementing and debugging critical code paths.
Deep SQL and data-modeling expertise, including query execution, incremental processing, schema evolution, and consistency.
Experience connecting independently managed systems through shared identifiers, documented relationships, and reliable data contracts, with clear ownership and compatibility standards.
Ability to reason about idempotency, replay, late-arriving data, partial failure, and correctness across system boundaries.
Experience establishing production practices adopted by multiple teams, including CI/CD, data validation, observability, incident response, secure deployment, and recovery.
Demonstrated ability to lead complex production investigations and major migrations, using logs, metrics, traces, query plans, profiles, and controlled experiments to identify root causes and prevent recurrence.
Demonstrated success influencing technical direction across organizational boundaries, aligning engineering and business leaders around shared goals, and developing engineers for broader technical responsibilities.

Nice To Haves

Extensive experience developing and managing Databricks, Apache Spark, PySpark, Spark SQL, Delta Lake, or Unity Catalog platforms, including common features applied across engineering teams.
Experience delivering self-service data publishing, cataloging, lineage, data contracts, or certification systems with demonstrated adoption.
Experience combining operational data from various cloud providers and internal systems, including entity resolution, inventory reconciliation, lifecycle history, and coverage measurement.
Experience designing and operating Kafka or comparable streaming systems, including partitioning, consumer behavior, offset management, backpressure, replay, and schema compatibility.
Experience enabling AI agents to discover and use governed data, or building production agentic systems with tool integration, evaluation, permissions, observability, and failure recovery.

Responsibilities

Define the architecture and multiyear technical roadmap, including system boundaries, shared interfaces, and engineering standards across ingestion, processing, storage, and data serving.
Balance immediate operational needs with long-term reliability, maintainability, and cost.
Lead delivery across platform engineers, data producers, and consumers; translate ambiguous requirements into designs and milestones, resolve dependencies, and guide implementation through production adoption.
Build and operate reusable batch and streaming ingestion capabilities for service-provider and internal data.
Establish supported patterns for APIs, change data capture, streams, database connectors, and files, including orchestration, retries, backfills, and recovery.
Establish shared identifiers, data contracts, consistent modeling, and automated quality and certification capabilities.
Partner with source owners to define dataset meaning, keys, supported joins, historical coverage, compatibility, and ownership; make quality failures actionable and dataset limitations visible to consumers.
Guide cataloging, lineage, and governed-access capabilities so people and AI agents can discover datasets, understand definitions and relationships, and follow approved access procedures.
Make onboarding and operations repeatable through shared libraries, reference implementations, deployment automation, and self-service workflows; measure onboarding effort, dataset reuse, and manual toil.
Own production readiness and reliability standards, including SLOs, testing, observability, runbooks, rollback, and recovery.
Lead critical investigations across pipelines, applications, SQL engines, Spark, storage, networks, and cloud services.
Partner with security and infrastructure teams on service identities, least privilege, secrets management, environment isolation, auditability, and upgrades and migrations that preserve data access and production service.
Mentor senior engineers and technical leads, lead design reviews, and resolve technical disagreements through evidence and clear tradeoffs.
Partner with leadership on priorities, progress, operational risks, and investment decisions.