Staff Software Engineer, Data Infrastructure

ZocdocNew York, NY
64d$179,444 - $270,750

About The Position

As a Staff Software Engineer (Data Infrastructure), you’ll lead the design and development of the software that underpins our data platform: ingestion frameworks, execution services, orchestration, metadata, data governance, and developer experience. Your focus is building APIs, libraries, and services that make data producers/consumers effective, while optimizing reliability, performance, and spend on AWS.

Requirements

  • 8+ years building backend/platform software with Python/Scala/Java and strong SQL.
  • Proven track record designing distributed systems.
  • Deep experience with Spark (Databricks or EMR/EKS) and AWS data services.
  • Hands-on data warehouse optimization experience, preferably with Snowflake.
  • Experience building platform APIs/SDKs that other engineers adopt.

Nice To Haves

  • Experience at petabyte-scale data platforms, distributed big data compute, or lakehouse engines.
  • Familiarity with metadata/governance tech such as Unity Catalog, Collibra, Lake Formation.

Responsibilities

  • Design and ship platform services for ingestion, transformation, orchestration, and metadata.
  • Build execution & scheduling capabilities for Spark/SQL jobs focusing on throughput and developer experience.
  • Implement lakehouse features such as schema evolution, partitioning, compaction, vacuum, snapshotting, ACID guarantees, and table-format governance.
  • Optimize Snowflake and other warehouses for cost controls, query profiling/pruning, workload isolation, and RBAC.
  • Deliver SDKs, CLIs, and templates that standardize how teams build reliable data products.
  • Work across AWS services with Infrastructure as Code (IaC) and strong observability.

Benefits

  • Flexible, hybrid work environment at our convenient Soho location.
  • Unlimited Vacation.
  • 100% paid employee health benefit options (including medical, dental, and vision).
  • Commuter Benefits.
  • 401(k) with employer funded match.
  • Corporate wellness programs with Headspace and Peloton.
  • Sabbatical leave (for employees with 5+ years of service).
  • Competitive paid parental leave and fertility/family planning reimbursement.
  • Cell phone reimbursement.
  • Catered lunch every day along with beverages and snacks.
  • Employee Resource Groups and ZocClubs to promote shared community and belonging.
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service