Senior/Staff Applied Scientist

Glean • Palo Alto, CA

234d • $175,000 - $230,000

About The Position

Glean is building a world-class Data Organization composed of product data science, applied science, data engineering and business intelligence groups. This is an applied science role based in our Palo Alto headquarters. It covers a subset of: Ranking & RAG: Define KPIs, create data foundations and build visualizations to measure the efficacy of Glean’s world-leading document indexing, retrieval, ranking infrastructure, which powers its search and LLM-powered products, as well as other ways of content personalization such as ranking agents in an agent library. Conduct rigorous empirical analyses rooted in sophisticated techniques and a deep understanding of our ranking stack to identify & prototype ways to improve ranking at Glean. ML ENG teams would iterate & deploy these techniques into production. A/B Experimentation: Collaborate with product data science and engineering teams to identify techniques, tooling and process improvements in online A/B experimentation to assist rigorous decision making across all relevant product domains. Develop and maintain our A/B experimentation platform based on stakeholder feedback. Write code or identify vendors to deploy these techniques to production in a scalable manner that’s easy to use by engineering, product data science, product management and design teams. Generative AI evaluation: End to end evaluation of various hero use cases like document/URL uploads, including evaluation set generation, coming up with evaluation criteria and methods to interpret the results. Breaking down end to end evaluations into more granular evaluation of various tasks & skills including but not limited to content summarization/analysis/generation, multi-step reasoning & strategizing, tool selection & use, coding & system design. Design, development and ownership of best practices, tools and processes across various evaluation problems, e.g. query intent classification, standardizing the use of best statistical principles to handle LLM stochasticity, industry benchmarking.

Requirements

• Masters/PhD degree in Statistics, Mathematics, Computer Science, or another quantitative field.
• 5+ years of experience for Masters degree holders, 3+ years for PhD holders.
• Strong background in statistics and/or machine learning.
• Proficient in Python and able to maintain an internal source-controlled library.
• Concise and precise in written and verbal communication.
• Proficient in SQL and the modern data stack.
• Strong at defining product KPIs and analyzing raw data for strategic insights.

Nice To Haves

• Experience in B2B SaaS.
• Experience working on ranking and A/B experimentation platforms.
• Passionate about using AI to improve productivity for data teams and non-data professionals.

Responsibilities

• Define KPIs, create data foundations and build visualizations to measure the efficacy of Glean’s document indexing and retrieval infrastructure.
• Conduct empirical analyses to identify and prototype ways to improve ranking.
• Collaborate with product data science and engineering teams to improve A/B experimentation processes.
• Develop and maintain the A/B experimentation platform based on stakeholder feedback.
• Evaluate various use cases for generative AI, including document/URL uploads.
• Design and develop best practices and tools for evaluation problems.

Benefits

• Competitive compensation
• Medical, Vision and Dental coverage
• Flexible work environment and time-off policy
• 401k
• Company events
• Home office improvement stipend upon joining
• Annual education stipend
• Wellness stipend
• Healthy lunches and dinners provided daily

What This Job Offers

• Education Level: Master's degree