Senior Software Engineer - Analytics Data Platform Lakehouse
About The Position
The Analytics Data Platform Lakehouse team builds and operates the foundations that power data engineers, applied AI, and product teams—managing millions of tables on their behalf and simplifying operations from maintenance and observability to governance, for both internal and customer-facing use cases. If you're excited by the intersection of petabyte data processing scale, open-source query engines, and building platforms with real product stakes, this is the team for you. At Datadog, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them.
Requirements
- You have a BS/MS/PhD in Computer Science, Engineering, or a related field, or equivalent professional experience
- You have deep, production-grade experience with one or more of Apache Iceberg, Trino, or Apache Spark, ideally demonstrated through significant open-source contributions: merged PRs, committer status, or PMC membership on projects
- You have built or operated large-scale distributed data systems
- You have a solid grasp of query planning, columnar file formats (Parquet, ORC), and table format internals (snapshots, manifests, partition evolution)
- You are fluent in Java, Scala or Go and comfortable with Python for pipeline tooling
- You have experience deploying and running data infrastructure on Kubernetes in cloud environments
Responsibilities
- Design, build, and operate core components of our lakehouse platform, including Apache Iceberg table management (data compaction, data layout optimization, materialized view scheduling…) and Iceberg catalog
- Drive adoption of open table formats across internal teams, owning the integration of Trino, Spark and other query engines (DuckDB, Puppygraph…) with our Iceberg-based at petabyte scale
- Build observability for managed iceberg tables, to identify query performance bottlenecks, cost drivers and contribute fixes back to upstream open-source projects (Iceberg, Trino, Spark, Open Lineage) where relevant
- Build self-serve tooling and abstractions that allow data engineering teams to reliably run thousands of pipelines per day against our lakehouse
- Collaborate with data engineers, analysts, and infrastructure teams to define the roadmap for our lakehouse architecture and shape how Datadog manages analytic data at scale
Benefits
- competitive salary and equity package
- variable compensation
- healthcare
- dental
- parental planning
- mental health benefits
- a 401(k) plan and match
- paid time off
- fitness reimbursements
- a discounted employee stock purchase plan
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
