Staff Software Engineer - Data Services
About The Position
At Cloudera, we empower people to transform complex data into clear and actionable insights. With as much data under management as the hyperscalers, we're the preferred data partner for the top companies in almost every industry. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises. At Cloudera, our Data Services Pillar is the heart of data innovation. We don’t just work with technology; we build it. Our mission is to empower data practitioners by creating seamless, enterprise-grade experiences for data engineering, warehousing, streaming, operational databases, and AI. We are seeking a Staff Software Engineer with deep expertise in data infrastructure and a strong focus on Apache Airflow to lead the orchestration and workflow engineering experience within Cloudera Data Engineering (CDE). In this high-impact, Individual Contributor (IC) role, you will be a key technical driver shaping how thousands of enterprise customers worldwide design, schedule, and monitor complex, multi-tenant data pipelines. You will help execute the architectural vision for scaling Airflow natively within Kubernetes across multi-cloud and on-premises environments, ensuring it seamlessly integrates with Spark and modern Lakehouse (Iceberg) architectures.
Requirements
- BS or MS in Computer Science or a related technical field.
- 10+ years of professional software engineering experience, with a heavy emphasis on data infrastructure, workflow orchestration, and distributed systems.
- Deep, production-proven experience architecting, scaling, and extending Apache Airflow (e.g., custom operators, hooks, providers, Celery/Kubernetes executors, and security models).
- Strong understanding of Python (essential for Airflow development) and at least one compiled language such as Java, Scala, C++, or Go.
- Deep expertise in cloud-native architectures, specifically running stateful and stateless workloads on Kubernetes (EKS, AKS, GKE) and/or private cloud environments (OpenShift, Rancher).
- Strong understanding of large-scale systems design, scheduling algorithms, resource allocation, and performance tuning.
- Exceptional communication skills, an open-minded attitude, and a passion for clean code, technical quality, and mentorship.
Responsibilities
- Drive the multi-year technical roadmap and architectural vision for enterprise-grade Apache Airflow within Cloudera Data Engineering.
- Design and optimize highly available, secure, and multi-tenant Airflow environments capable of managing complex dependencies and scheduling workflows across thousands of distributed cluster nodes.
- Gain and apply deep technical knowledge across the CDE stack, with a primary focus on extending Airflow, integrating it with Spark and Iceberg, and contributing back to the open-source community where applicable.
- Foster a culture of technical excellence through hands-on technical mentorship, rigorous design reviews, and robust architectural guidance for distributed workflow execution.
- Collaborate closely with product management, engineering leaders, and cross-functional partners to deliver critical features that make workflow automation seamless and intuitive for CDE customers.
Benefits
- Generous PTO Policy
- Support work life balance with Unplugged Days
- Flexible WFH Policy
- Mental & Physical Wellness programs
- Phone and Internet Reimbursement program
- Access to Continued Career Development
- Comprehensive Benefits and Competitive Packages
- Paid Volunteer Time
- Employee Resource Groups
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
