Sr. Data Scientist, Search & NLP Analytics

Realtor.com Careers•Austin, TX

20h•Hybrid

About The Position

As a Senior Data Scientist on our Search & Algorithms team, you'll own NLP analytics for our next-generation semantic search experiences. You'll apply NLP and LLM-based techniques to a proprietary corpus of millions of real consumer search interactions—extracting signals like buyer vs. seller intent and conversational patterns to improve how our models rank and personalize results. You'll also prototype data products that turn those signals into features product and engineering can ship. This is an AI-native role: LLM evaluation tooling and AI-assisted workflows are core to how you'll work, not afterthoughts.

Requirements

5+ years in data science, with 2+ years focused on NLP, text analytics, or search analytics.
Hands-on NLP experience: query classification, intent detection, semantic similarity, entity extraction, or LLM-based evaluation.
Strong Python skills and proficiency with NLP libraries (Hugging Face, spaCy, scikit-learn, or similar).
Experience analyzing conversational and LLM-generated data using observability or evaluation tooling (e.g., Langfuse, Braintrust, or similar platforms for tracing, scoring, and evaluating AI outputs).
Robust SQL skills and comfort working with large-scale clickstream or query log datasets.
Solid experimentation fundamentals: A/B testing, power analysis, and statistical inference.
Ability to build rapid prototypes and translate analytical findings into product-ready recommendations.
Excellent communication skills—able to present NLP and AI concepts clearly to product and executive stakeholders.
Bachelor's or Master's degree in Computer Science, Statistics, Linguistics, Information Science, or a related field.

Nice To Haves

Experience with search ranking analytics, query log analysis, or information retrieval measurement.
Hands-on experience with LLM APIs (OpenAI, Claude, Gemini, or similar) and prompt engineering for evaluation, scoring, or analytical tasks.
Experience working with LLM observability platforms (Langfuse, Braintrust, or similar) to instrument, trace, and evaluate AI product performance at scale.
Prior work in consumer-facing search products or two-sided marketplace analytics.
Experience with Snowflake or similar cloud data warehouses.
Ph.D. in Computer Science, Computational Linguistics, NLP, or a related field.

Responsibilities

Measure and evaluate semantic search models using NLP and LLM-based evaluation frameworks on real user queries—surfacing buyer vs. seller context, exploration vs. transaction signals, and conversational patterns that inform ranking and personalization.
Prototype data products that leverage conversational query data and behavioral signals to drive personalized search experiences, translating NLP insights into actionable features for product and engineering teams.
Design tests for search model changes; define primary and guardrail metrics; communicate results clearly to both technical and non-technical stakeholders.
Instrument and analyze conversational AI data using LLM observability and evaluation tooling—tracking model outputs, scoring response quality, and surfacing patterns across large volumes of AI-generated interactions to inform model and product improvements.
Partner with search product and engineering leadership to define success metrics, shape the analytics roadmap for search, and present findings to senior leadership.
Work in an AI-native way: apply agentic AI workflows, LLM APIs, and AI-assisted development to accelerate your work and help set the standard for how our team uses AI.
Actively evaluate and adopt new tooling as the space evolves.