Staff Software Engineer, Data Infrastructure

ZocDoc, Inc. • New York, NY

63d • Hybrid

About The Position

As a Staff Software Engineer (Data Infrastructure), you'll lead the design and development of the software that underpins our data platform: ingestion frameworks, execution services, orchestration, metadata, data governance, and developer experience. Your focus is building APIs, libraries, and services that make data producers/consumers effective, while optimizing reliability, performance, and spend on AWS. You'll enjoy this role if you are… A builder of platforms and frameworks (not just point pipelines). Comfortable with distributed systems abstractions (compute scheduling, storage layout, back‑pressure, retries, idempotency). Excited by lakehouse tech and modern data contracts, and you want to create self‑service for hundreds of use cases.

Requirements

• 8+ years building backend/platform software with Python/Scala/Java and strong SQL; proven track record designing distributed systems.
• Deep experience with Spark (Databricks or EMR/EKS) and AWS data services; solid grasp of scheduler/executor behavior and performance tradeoffs.
• Hands‑on data warehouse optimization (Snowflake ideal; others welcome).
• Experience building platform APIs/SDKs that other engineers adopt; excellent collaboration and technical leadership.

Nice To Haves

• Experience at petabyte‑scale data platforms, distributed big data compute, or lakehouse engines (Delta/Iceberg).
• Familiarity with metadata/governance tech (Unity Catalog, Collibra, Lake Formation).

Responsibilities

• Design and ship platform services for ingestion, transformation, orchestration, and metadata (e.g., service‑backed interfaces for Dagster/Airflow, lineage, quality, and data contracts).
• Build execution & scheduling capabilities for Spark/SQL jobs (queuing, prioritization, retries, resource isolation on EMR/EKS/Databricks), focusing on throughput and developer experience.
• Implement lakehouse features (Delta/Iceberg): schema evolution, partitioning, compaction, vacuum, snapshotting, ACID guarantees, and table‑format governance
• Optimize Snowflake and other warehouses: cost controls, query profiling/pruning, workload isolation, RBAC; expose safe self‑service patterns.
• Deliver SDKs, CLIs, and templates that standardize how teams build reliable data products; enable CI/CD for data and contract testing.
• Work across AWS (S3, EMR/EKS, Glue/Athena, Lambda, Kinesis/MSK) with IaC (Terraform) and strong observability (Datadog/CloudWatch).

Benefits

• Flexible, hybrid work environment at our convenient Soho location
• Unlimited Vacation
• 100% paid employee health benefit options (including medical, dental, and vision)
• Commuter Benefits
• 401(k) with employer funded match
• Corporate wellness programs with Headspace and Peloton
• Sabbatical leave (for employees with 5+ years of service)
• Competitive paid parental leave and fertility/family planning reimbursement
• Cell phone reimbursement
• Catered lunch everyday along with beverages and snacks
• Employee Resource Groups and ZocClubs to promote shared community and belonging
• Great Place to Work Certified

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Industry: Professional, Scientific, and Technical Services
• Education Level: No Education Listed
• Number of Employees: 501-1,000 employees