Staff Data Platform Engineer

WEXWEXUS • San Jose, CA

1d • $140,600 - $173,100 • Remote

About The Position

The Data Platform Engineering Team serves as the backbone of the enterprise data architecture, connecting Data Engineering, Infrastructure, and Operations. The team owns multi-cloud infrastructure across AWS and Azure, Apache Airflow orchestration, big data frameworks, and containerized environments to deliver a robust, production-grade platform.

The Staff Software Engineer will serve as a core platform engineer and DevOps specialist, taking ownership of orchestration and containerization while advancing CI/CD automation, observability, data governance, and cloud cost optimization. The role spans platform architecture and implementation, production operations and incident management, roadmap planning, and technical mentorship.

The listed base pay range is $140,600 to $173,100. Actual pay varies based on qualifications, skills, competencies, and proficiency. Non-sales roles are typically eligible for a quarterly or annual bonus, subject to the applicable plan. The company describes its benefits as comprehensive and market competitive.

Requirements

• At least 6 years of hands-on professional experience in Data Platform Engineering, DevOps, or Site Reliability Engineering supporting big data environments.
• Deep production experience managing, tuning, dynamically scaling, and troubleshooting Apache Airflow infrastructure, including Celery/Kubernetes Executors, DAG parsing performance, and dynamic configurations.
• Expert-level skills in Kubernetes, Helm, Terraform, Docker, ArgoCD, and GitHub Actions, including custom runner configurations.
• Demonstrated experience configuring production alerting, metrics collection, and log aggregation using Grafana, Prometheus, and Loki.
• Deep operational and configuration experience with Spark, AWS EMR, Snowflake, Azure Synapse, dbt, and Apache Kafka.
• Hands-on experience with AWS (including EC2, S3, VPC, IAM, and EKS) and/or Azure, with a demonstrated history of cloud cost optimization.
• Experience deploying or managing data cataloging tools such as DataHub, Amundsen, or similar metadata management platforms.
• Strong programming skills in Python, Bash, Go, or SQL.
• Ability to take complex architectural requirements from concept through production-grade deployment in a high-paced environment.
• Automation-first approach, including replacing manual operational tasks with code, automated tests, CI/CD checks, and resilient self-healing infrastructure.

Responsibilities

• Design, deploy, scale, and maintain highly available Apache Airflow clusters using Helm, Kubernetes, and Terraform for enterprise ETL/ELT workflows.
• Drive infrastructure-as-code and GitOps practices using Terraform, Helm Charts, and ArgoCD; automate platform deployments, manage self-hosted GitHub runners, and enforce GitOps workflows.
• Architect and manage GitHub Actions CI/CD pipelines for deployment of data pipelines, infrastructure components, and DAGs.
• Provision and manage scalable Kubernetes EKS/AKS clusters, Docker containers, and cloud infrastructure across AWS and Azure.
• Build and maintain monitoring, logging, and alerting systems using Grafana, Prometheus, Loki, and centralized log management solutions.
• Build scalable data infrastructure supporting Apache Spark, AWS EMR, Snowflake, Azure Synapse, Apache Kafka, and dbt.
• Deploy and maintain data discovery and metadata tooling such as DataHub for governance, schema management, and cataloging.
• Monitor, audit, and optimize compute and storage costs across AWS and Azure infrastructure.
• Build internal tools, CLI utilities, and workflow templates to improve productivity for data engineers, analytics engineers, and data scientists.
• Own data platform modules from architectural design through deployment, production operations, and incident management.
• Define and enforce engineering standards for code quality, design patterns, testing, data lineage, and security, including access controls, IAM, and dynamic schema management.
• Partner with data leads, product managers, and business stakeholders to identify infrastructure gaps, establish a 1–2 year platform roadmap, and prioritize initiatives.
• Serve as a data infrastructure subject matter expert and mentor junior and mid-level data platform engineers.

Benefits

• Health insurance
• Dental insurance
• Vision insurance
• Retirement savings plan
• Paid time off
• Health savings account
• Flexible spending accounts
• Life insurance
• Disability insurance
• Tuition reimbursement
• Quarterly or annual bonus eligibility for non-sales roles, subject to the applicable plan

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed