Senior Software Engineer, Dataplane Management

Astronomer•San Francisco, CA

19h•$180,000 - $210,000•Remote

About The Position

Astronomer empowers data teams to bring mission-critical software, analytics, and AI to life and is the company behind Astro, the industry-leading unified DataOps platform powered by Apache Airflow®. Astro accelerates building reliable data products that unlock insights, unleash AI value, and powers data-driven applications. Trusted by more than 800 of the world's leading enterprises, Astronomer lets businesses do more with their data. To learn more, visit www.astronomer.io . About this role: At Astronomer, we’re redefining how companies run Apache Airflow at scale. Our R&D organization is home to some of the most innovative minds in cloud infrastructure and open-source software. We’re looking for a Senior Software Engineer to join our Data Plane Management team, whose mission is to efficiently operate, and develop new features for, the cloud compute foundation that our customer Airflow deployments are hosted upon. This is a particularly fascinating role for someone who loves the challenge of managing a large scale fleet of 500+ Kubernetes clusters with a true software engineering mindset. This team owns the compute & network primitives in our Astro platform and develops the APIs that our customers use to stand up their environments. The team’s roadmap includes new self-service features such as failover cluster support, VPC/VNet peering, Transit Gateway wiring, metric/log shipping, and more.

Requirements

Experience building and operating SaaS infrastructure, or experience managing a large scale internal compute platform.
Software engineering expertise with Golang, or similar languages with a desire to learn Golang.
A passion for reliability and operational excellence.
A low tolerance for toil, alert fatigue, and other nonsense.
Production experience with a container based orchestration system (Kubernetes preferred, but not essential).
Understanding of how to build with security and isolation in mind, so that Astronomer’s managed platform can securely integrate with any customer environment and ensure strict isolation between customer workloads.
Strong communication skills, both written and verbal, with experience in working with a globally distributed team in delivery.
Proactive approach to identifying and addressing issues, with a focus on ownership and accountability.
Experience as part of an on-call rotation - this role involves periodic on-call for the services and system we own.

Nice To Haves

Experience working on a SaaS/PaaS product across multiple cloud providers, with a ‘single tenant data plane’ model.
Programmatically administered Kubernetes in multiple clouds.
Experience designing systems for resiliency, scale and security.
A familiarity with Non-Abstract Systems design is a major plus.
A passion for finding and addressing inefficiencies in code, infrastructure costs, tooling and processes.
Experience with our particular tech stack components and technologies (deep breath): Calico/Azure CNI/Cilium, PostgreSQL/Aurora/CloudSQL/etc, OpenTelemetry, Prometheus/ Chronosphere, Splunk, Istio, Karpenter, Falco,
Experience of the innards and quirks of AWS, GCP and Azure.
Experience with Apache Airflow.

Responsibilities

Own key endpoints and features of our flagship product, Astro, to extend our offering to more complex customer networking options.
Work across domains to develop diverse features for our core infrastructure (ie: workload identity, multitenancy, cross region disaster recovery, cloud resource management, private network access).
Evolve our fleet orchestration system to enable us to safely make (and roll back) changes across our infrastructure, and to enable us to scale from the 500+ clusters we have now to thousands in the future.
Gain experience working in a multi-cloud environment by working with managed Kubernetes offerings and network/auth primitives from AWS, Azure, and GCP.
Deepen your operational knowledge of Kubernetes-based workloads by managing the data pipelines of many of the largest companies in the world.

Stand Out From the Crowd

Upload your resume and get instant feedback on how well it matches this job.

Upload and Match Resume