Sr. Analytics Data Platform Engineer

DoubleVerify•New York, NY

31d•$107,000 - $212,000

About The Position

You will join the Data & Analytics Platform (DAP) team within the Pinnacle engineering organization. The DAP team owns and operates two data consolidation platforms — Quantum (contract-driven, next-gen) and Analytics 2.0 (SQL-driven, legacy) — that ingest, transform, and serve billions of records daily from social platforms, measurement systems, and third-party partners. The data powers Looker dashboards, customer-facing reports, and downstream APIs used across DoubleVerify's product suite.

Requirements

Bachelor's degree or foreign equivalent in Computer Science, Data Engineering, or a related field.
5+ years of experience in a Data Engineering or related role.
Strong SQL skills — advanced querying, performance tuning, window functions, and complex transformations at scale.
Proficiency in Python — building libraries, data processing scripts, and automation tooling (experience with Pydantic, Jinja2, or similar templating frameworks is a plus).
Deep experience with Snowflake — schema design, Snowpipe, streams, tasks, materialized views, clustering, and query optimization.
Experience with dbt (data build tool) — building and maintaining models, macros, custom materializations, and incremental strategies.
Experience with orchestration tools — Airflow / Cloud Composer, DAG design, scheduling, and monitoring.
Experience with cloud platforms — GCP (GCS, BigQuery, Cloud Composer, Kubernetes) or equivalent.
Strong understanding of data warehousing concepts — dimensional modeling, star/snowflake schemas, slowly changing dimensions, fact/aggregate table design, and data consistency patterns.
Experience with CI/CD pipelines — GitLab CI, Flyway migrations, or similar deployment automation.
Experience with AI-assisted development tools — Claude Code, Cursor, GitHub Copilot, or similar AI coding assistants. Experience building or contributing to AI agent context files (AGENTS.md), skills, or meta-repo patterns is a strong plus

Nice To Haves

Experience building or working with contract-driven / configuration-driven data platforms where pipelines are generated from declarative specifications (YAML, JSON schemas).
Experience with Looker / LookML — building semantic models, exploring, aggregating awareness, and dashboard development.
Experience with Kafka — schema registries, topic management, and streaming data integration.
Experience with data quality and observability frameworks — automated testing, watermarking, data integrity validation, and SLA monitoring.
Experience with Terraform or infrastructure-as-code for managing cloud resources.
Familiarity with data mesh principles — federated data ownership, data products, and self-service platform design.

Responsibilities

Design and maintain the YAML-based "Contract" system that allows users to define data entities, transformations, and SLOs without writing low-level orchestration code.
Develop the translation engine that converts user contracts into automated dbt models, Airflow DAGs, and Snowflake objects.
Transition the platform from static configuration files to a dynamic, API-first architecture, enabling programmatic creation of data artifacts.
Build tooling and guardrails that allow business units to deploy their own data solutions while maintaining global standards for governance and security.
Optimize the "translation" layer to ensure that generated jobs are efficient, cost-effective, and leverage the full power of the Snowflake/dbt stack.
Act as the "Product Manager" for your platform, gathering feedback from internal users to simplify the data development lifecycle.
Design and build data pipelines that process billions of records a day across consolidation, semantic, and externalization layers using the DV Internal Data Platform — a self-service, contract-driven architecture where pipelines are defined via YAML contracts and automatically deployed to Snowflake, Airflow, and Looker.
Develop and extend the Contract Interpreter — a Python library (Pydantic, Jinja2) that reads contract driven platform based YAML and generates dbt models, Airflow DAGs, and environment configurations for each deployment environment (dev, stg, prod).
Lead new initiatives and integrations with the world's largest social platforms (YouTube, TikTok, Meta, Snapchat, Reddit, Netflix, etc.) to measure ad performance end-to-end.
Build and maintain the semantic layer — design LookML models, explores, and views that translate consolidated data into customer-ready analytics through Looker.
Implement and maintain observability — build monitoring, alerting, watermarking, and data consistency checks to ensure pipeline reliability and data freshness at scale.
Leverage AI agents and tooling — contribute to and use the team's AI agent workspace (meta-repo with AGENTS.md context files, skills, and MCP integrations) to accelerate development, automate workflows, and encode institutional knowledge for AI-assisted engineering.
Design schema evolution and data migration strategies — manage schema versioning, backward compatibility, incremental vs. full-refresh deployments, and large-scale data backfills.
Work in multi-functional agile teams with end-to-end responsibility for product development and delivery — from contract definition to customer-facing data.
Collaborate directly with engineers from partner platforms on API development and data integration specifications.
Train and mentor a team of software engineers.