Senior Data Engineer

Chicory

12d • $160,000 - $185,000 • Hybrid

About The Position

Chicory is hiring a Senior Data Engineer to own and expand the data platform behind our advertising, campaign, product, and financial reporting. You’ll work with high-volume, real-time ad events, partner APIs and files, and internal business systems using GCP, BigQuery, Dataflow, Cloud Composer, Python, SQL, and Superset. The role covers the full data stack, from ingestion and orchestration through warehouse design, data marts, and BI, giving you the opportunity to see how data moves from our advertising platform into reports used across the company. As Chicory’s Senior Data Engineer, you will have the autonomy to define the next version of our data lake and warehouse architecture and lead its implementation. You’ll remain hands-on, building pipelines and data models, establishing clear metric definitions, and improving how teams access and use data. You will work closely with backend and cloud engineers, as well as Campaign Management, Finance, Product, and leadership. Your work will directly support campaign performance, revenue reporting, publisher payments, and product decisions. This role offers broad technical ownership and the opportunity to establish the engineering practices that will support Chicory’s data systems and future data team.

Requirements

• Demonstrated success designing, rearchitecting, and operating a production analytical data platform that includes a data lake or durable raw layer, a cloud data warehouse, governed data models, and business-facing data marts
• Deep hands-on knowledge of BigQuery, including schema and table design, partitioning and clustering, query plans and performance, workload patterns, permissions, retention, and cost management
• Strong Python and advanced SQL skills, including the ability to write maintainable, tested production code and review work across ingestion, transformation, orchestration, and data-serving layers
• Hands-on production experience building batch and streaming systems with Apache Beam and Dataflow and with Apache Airflow and Cloud Composer, including retries, idempotency, dependency management, duplicate and late-arriving events, schema evolution, replay, backfills, monitoring, and failure recovery
• Strong data-modeling judgment across dimensional, event, and domain models, with experience creating stable facts, dimensions, slowly changing dimensions, semantic definitions, and reusable analytical datasets
• Track record of establishing automated data quality, reconciliation, observability, lineage, alerting, service-level objectives, incident response, and operational runbooks for critical data products
• Experience enabling BI and self-service analytics through Superset or a comparable platform, including curated datasets, governed metrics, access patterns, dashboard performance, and stakeholder education
• Applied understanding of data governance, least-privilege access, privacy and sensitive-data handling, auditability, retention, and the controls required for financial or revenue-impacting data
• Experience with software-engineering practices for data systems, including Git, automated testing, CI/CD, code review, infrastructure as code such as Terraform, and clear technical documentation
• Demonstrated ability to operate as an autonomous senior technical owner: assess an unfamiliar environment, make architectural decisions, sequence migrations, resolve ambiguity, lead incidents, and communicate tradeoffs with both engineers and non-technical stakeholders

Responsibilities

• Map how ad requests, bids, impressions, clicks, campaign metadata, spend, revenue, publisher payments, partner files, and product events move from their source into BigQuery and Superset
• Choose and implement the new BigQuery structure, including raw, staging, core, and reporting datasets; table grain and keys; naming; partitioning and clustering; incremental loads; and ownership
• Build and maintain batch and streaming ETL and ELT pipelines with Python, SQL, Apache Beam, and Dataflow using data from Pub/Sub, Cloud Storage, APIs, databases, and files
• Build and maintain Cloud Composer and Apache Airflow DAGs with clear dependencies, useful retries and alerts, and tasks that can be rerun safely
• Choose and implement a version-controlled SQL transformation workflow using Dataform, dbt, or an equivalent tool, then move important calculations out of scheduled queries and Superset dashboards
• Build the BigQuery tables and data marts used for campaign delivery, spend and revenue reconciliation, publisher payments, partner reporting, product analytics, and leadership reporting
• Investigate mismatches between source systems, partner reports, BigQuery, and Superset; correct the logic, repair affected data, and document why the numbers changed
• Add tests and alerts for late or missing data, duplicate records, unexpected schema changes, broken joins, and totals that do not reconcile
• Maintain the Superset datasets that power business reporting, review expensive or confusing queries, and make sure shared metrics come from the same maintained model
• Review BigQuery usage and cost, then improve the tables, queries, schedules, storage, or retention rules responsible for avoidable spend
• Manage data access with GCP IAM, limit access to sensitive fields, and document retention or deletion rules where they apply
• Use Git, automated tests, code review, deployment pipelines, and infrastructure as code so changes can be reviewed, repeated, and rolled back
• Work directly with Engineering, Campaign Management, Finance, Product, Sales, and leadership to define calculations, verify new tables, and decide when old reports can be retired
• When a pipeline fails or a report looks wrong, lead the investigation and recovery. Leave runbooks, diagrams, model descriptions, and metric definitions that another engineer can follow
• Build upon team and company culture and act as a champion of the Chicory Principles

Benefits

• Comprehensive healthcare benefits, including dental + vision plans
• Flexible paid time off policy for vacation, sick days, mental health days, and more!
• 14 paid holidays + flexibility to take paid time off for any other holidays you observe
• Employer-paid membership to One Medical
• Free ClassPass membership
• Annual learning & development stipend
• Flexible remote and hybrid work style arrangements
• Fun and friendly office in NYC stocked with free snacks
• Virtual and in-person outings and fun activities with our amazing team
• Dog-friendly workplace
• Competitive compensation package, including base salary + stock options
• Flexible Spending Account (FSA) + Health Spending Account (HSA)
• 401k retirement plan
• Commuter benefits
• Discounted pet insurance

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed