Senior Data Engineer

Polymarket•New York, NY

About The Position

Polymarket is seeking a Senior Data Engineer to enhance our data platform, supporting the company's rapid growth. The data layer is critical, processing every trade, position, and resolution to power user-facing features like leaderboards, PnL, activity feeds, and volume analytics, all while maintaining sub-100ms latency targets at crypto scale. This is a high-ownership role where you will be responsible for driving entire workloads end-to-end, from design and architecture to delivery. Your goal is to create a robust and reliable analytics and serving data layer that the rest of the company can depend on without concern for issues like schema drift, migration parity gaps, latency regressions, or cost overruns. You will be the key person ensuring the data layer can meet performance and cost requirements for complex queries.

Requirements

5+ years of data engineering experience on production systems serving real users at scale.
Deep knowledge of OLTP/OLAP split architectures and understanding of when to use row stores, column stores, or both.
Expertise in columnar warehouses (ClickHouse strongly preferred; Snowflake, BigQuery, Redshift, or Apache Pinot accepted if fundamentals are solid).
Experience with data lake technologies: Parquet, Iceberg (or Delta/Hudi), compaction strategies, and S3 layout discipline.
Proficiency in streaming pipeline technologies: Kafka, understanding of exactly-once vs. at-least-once semantics, backpressure, consumer-group patterns, and schema evolution.
Strong data modeling fundamentals: star/snowflake schemas, SCD patterns, CDC, idempotent event sourcing, and tradeoffs between dimensional modeling and event logs.
Experience with PostgreSQL at scale, including partitioning, index design, autovacuum/bloat remediation, query planning, and CDC triggers vs. logical replication.
SQL fluency at warehouse scale, including window functions, CTEs, dictionary-based enrichment, and dialect specifics.
Understanding of distributed systems concepts: consistency models, event ordering, replay semantics, write-once vs. mutable state, and reorg handling.

Nice To Haves

EV M indexing experience (rindexer, subgraphs, or comparable).
Rust programming experience for indexer and validation tooling.
Domain knowledge in DeFi, prediction markets, or order-book systems.
Experience with observability and SLOs (Prometheus metrics design, dashboard discipline, alert-fatigue avoidance).
Python for SQL tooling, ad-hoc analysis, and migrations.
Track record of shipping a platform migration or greenfield data stack under a hard deadline.

Responsibilities

Own the OLAP analytics layer, managing the entire lifecycle from raw event ingestion to business aggregates and API-serving views.
Design and manage materialized views, refresh cadences, dictionary catalogs, query planning, and cost optimization for the columnar warehouse.
Partner on the OLTP serving layer, focusing on high-write serving tables that support product APIs, including partition strategy, indexing, trigger pipelines, autovacuum tuning, and bloat monitoring to meet sub-100ms read-path requirements.
Shape streaming and data lake infrastructure by defining Kafka topic schema contracts, evolving the S3 lake layout with modern table formats, and contributing to data correctness validation tooling.
Design data models for event-sourced, append-mostly data with chain-reorg semantics, including derivative analytics like PnL and position tracking.
Formalize ownership boundaries between upstream ingestion and downstream analytics.
Coordinate with other teams, including the warehouse-owning team and downstream consumers (frontend, notifications, third-party integrators), to negotiate schema contracts.