Principal Infrastructure Engineer

Nextdata•San Francisco, CA

About The Position

The future of data lies in decentralization, and the concept of a data mesh is the proven approach for implementing this at Enterprise scale. We’re here to make it a reality. Nextdata OS is a data-mesh-native platform built to meet the challenge of decentralizing data at scale. We are inventing a new way for developers to work with data and share it responsibly via data product containers. Our vision is to build a world where AI/ML and analytics are powered by decentralized, responsible, and equitable data ownership, across boundaries of organizations, technology, and most importantly boundaries of trust. Our purpose is to change the experience of creating, sharing, discovering, and using data forever, to be connected, fast, and fair based on data mesh principles. Our technology is designed to empower data developers, users and owners with a delightful experience where data products are a first-class primitive, with trust built-in. We are here to accept the reality that the world of data is complex and messy; data models are out-of-date the moment they are created; data is owned across trust boundaries; data is stored on different platforms; data is used in many different modes and most importantly data can't protect itself. We recognize that past approaches to tackle these complexities with centralized data collection, modeling and governance are ineffective at best and pathologically unfair at worst. Our mission is to reimagine the world of data with you. As a Principal Infrastructure Engineer, you will help lead and build out the automation for provisioning and managing the Nextdata OS in multiple clouds. Given that we are in the early, your work will shape the future of how data mesh will get adopted by the industry. You will apply your knowledge of having deployed large scale distributed systems and data and ML/analytics infrastructure to join the founding engineering team to deliver a self-service and secure OS platform for the data product developers of the future. You will contribute to the overall Nextdata OS design and own the design and implementation of the deployment of the OS in PaaS, SaaS and customer cloud environments. You will write scalable and modular infrastructure-as-code and review others’ code to ensure the overall code quality. You will own the quality of the code you have written, even (especially!) when it is running in production. You will work with customers to understand their requirements, collaborate with the product team, and co-create the Nextdata OS with the rest of the engineering team.

Requirements

Bachelor's Degree in Computer Science, IT, Engineering, or related field.
10+ years of experience as an Infrastructure Engineer.
Experience working for a start-up and leading and infrastructure team.
Experience with designing, developing, and maintaining robust CI/CD automation for Kubernetes native applications in multiple clouds and on customer cloud environments.
Successfully dealt with the complexities of PaaS and SaaS deployment models for a large number of customers.
Experience with provisioning and managing data and ML/analytics ecosystem on multiple clouds, such as Snowflake, Databricks, AWS/Azure/GCP data warehouses and ML platforms.
Successfully worked on minimizing infrastructure costs for deployments.
Led or contributed significantly to infrastructure security audits.
Experience troubleshooting system issues, providing timely resolution to minimize downtime.
Solid understanding of containerization technologies, specifically Docker.
Expert knowledge of Kubernetes, with experience in managing and operating Kubernetes clusters across multiple environments.
Proficiency in working with AWS EKS, Google GKE, Azure AKS
Strong experience in network architecture, particularly in cloud environments (AWS VPC, Azure VNet, GCP VPC).
Proven experience with Infrastructure as Code (IaC) tools such as Terraform or CloudFormation.
Familiarity with monitoring and logging tools such as Prometheus and ELK stack.
Excellent problem-solving skills, attention to detail, and the ability to work in a fast-paced, dynamic, team-oriented environment.
Experience with service meshes like Istio or Linkerd.
Familiarity with cloud security practices and tools.

Responsibilities

Help lead and build out the automation for provisioning and managing the Nextdata OS in multiple clouds.
Shape the future of how data mesh will get adopted by the industry.
Apply knowledge of having deployed large scale distributed systems and data and ML/analytics infrastructure to join the founding engineering team to deliver a self-service and secure OS platform for the data product developers of the future.
Contribute to the overall Nextdata OS design and own the design and implementation of the deployment of the OS in PaaS, SaaS and customer cloud environments.
Write scalable and modular infrastructure-as-code and review others’ code to ensure the overall code quality.
Own the quality of the code written, even when it is running in production.
Work with customers to understand their requirements.
Collaborate with the product team.
Co-create the Nextdata OS with the rest of the engineering team.