Principal Data Scientist

Zillow

1d • $173,000 - $290,900 • Remote

About The Position

Zillow Group is seeking a Principal Data Scientist to serve as a measurement and impact partner in CoreTech. This senior individual-contributor role is embedded with the Data & Infrastructure Platform team and partners closely with Product leadership, especially the Connections Platform (CP). CP supports conversational experiences, lead funnels, follow-up workflows, and shared agent services, increasingly powered by LLM-based and agentic systems. The role may also take on difficult measurement problems across the broader CoreTech organization.

The position will make the impact of platform work measurable and actionable, define meaningful platform metrics, build rigorous evaluation methods for LLM and agentic systems, and operationalize measurement in partnership with engineering. A central challenge is estimating causal impact when clean A/B testing is unavailable, including for experiences that span sessions and channels or roll out broadly at once. The role will also establish an evaluation feedback loop that distinguishes genuine quality and business outcomes from activity and adoption.

The Principal Data Scientist will set measurement standards for platforms used across Zillow, help shape Data & Infrastructure Platform OKRs and executive reviews, and communicate findings and trade-offs to senior leaders. The role is categorized as remote. Remote employees do not have a permanent corporate office workplace and must identify their chosen physical work location. U.S. employees may live in any of the 50 states, with limited exceptions.

Requirements

• 10+ years of experience in Data Science, UX Research, or Applied Science.
• Meaningful experience measuring products whose customers are developers, internal teams, or platform users.
• Experience leading measurement of platform products where adoption, partner teams, and downstream outcomes matter.
• Experience designing evaluations for complex systems where quality is difficult to define, using methods such as labeled test sets, human review, or LLM-as-a-judge.
• Experience owning the design and analysis of production A/B tests and causal inference methodology, with the judgment to select appropriate methods for difficult questions.
• Deep experience designing primary metrics, KPIs, OKR metric definitions, and leading- and lagging-indicator systems.
• Ability to distinguish measurement problems from instrumentation problems and address the appropriate issue.
• Ability to work directly with engineers on event taxonomies, methodology, and schemas.
• Proficiency writing Python and SQL and experience shipping repeatable, reliable, documented instrumentation.
• Ability to communicate rigorous measurement narratives to VPs or Senior Directors and surface relevant assumptions.
• Individual-contributor approach: write code, model data, prototype evaluations and dashboards, and raise the rigor and repeatability of team practices.

Nice To Haves

• PhD and 4+ years, MS and 6+ years, or BS and 8+ years of experience in Statistics, Econometrics, Data Science, User Experience Research, Machine Learning, Operations Research, or a related quantitative field.
• Direct experience measuring complex, production user-facing systems, including conversational, LLM-based, or agentic products.
• Fluency with modern evaluation methods, including LLM-as-a-judge and offline or online evaluation harnesses, or adjacent evaluation experience.
• Experience measuring the impact of AI or agentic tooling on real outcomes, separating adoption and activity from productivity or quality gains and assessing ROI against spend.
• Experience working in a Platform-as-a-Product model with internal teams as customers.
• Fluency with quasi-experimental and causal inference methods such as difference-in-differences, synthetic control, interrupted time series, and switchback or interleaved designs, including explicitly stating assumptions and limitations.

Responsibilities

• Define and operationalize measurement strategy for the Connections Platform and broader Data & Infrastructure Platform team, from raw events through executive dashboards and causal attribution.
• Design evaluations for CP’s LLM-based and agentic systems, define quality signals, and build evaluation frameworks, including LLM-as-a-judge and tools such as Braintrust.
• Own causal inference and experimentation, designing and analyzing A/B tests and selecting suitable quasi-experimental methods when randomization is unavailable.
• Choose and apply methods such as switchbacks, difference-in-differences, synthetic control, and interrupted time series, stating assumptions and producing defensible impact estimates.
• Advise teams on what to measure and experiment on, challenge vanity metrics, identify missing leading indicators, and select signals that reflect customer experience.
• Connect adoption of agents and platform investments to business and revenue outcomes, not only activity, speed, and quality measures.
• Partner with engineering individual contributors to define events and schemas and build reliable pipelines and instrumentation for trusted quality measurements.
• Translate complex measurement findings and trade-offs into clear narratives for VP- and C-level partners.
• Set standards for rigorous, repeatable data science in platform teams and measure platforms as products, including their adoption, quality, and impact.
• Assess multidimensional product changes affecting consumers, partners, and developers and carrying business and revenue stakes.

Benefits

• Competitive base salary.
• Eligible for equity awards based on factors such as experience, performance, and location.
• Remote work flexibility through Cloud HQ, subject to identifying a physical work location and applicable location exceptions.

What This Job Offers

• Job Type: Full-time
• Career Level: Principal
• Education Level: No Education Listed