Senior/Staff Applied Scientist

Glean•Mountain View, CA

17h•$180,000 - $330,000•Hybrid

About The Position

Glean is building a world-class Data Organization composed of product data science, applied science, data engineering and business intelligence groups. This is an applied science role based in San Francisco office. You will: Collaborate with product data science and engineering teams to identify techniques, tooling and process improvements in online A/B experimentation to assist rigorous decision making across all relevant product domains. Develop and maintain our A/B experimentation platform based on stakeholder feedback Write code or identify vendors to deploy these techniques to production in a scalable manner that’s easy to use by engineering, product data science, product management and design teams Conduct end to end evaluation of various hero use cases like document/URL uploads, including evaluation set generation, coming up with evaluation criteria and methods to interpret the results. Break down end to end evaluations into more granular evaluation of various tasks & skills including but not limited to content summarization/analysis/generation, multi-step reasoning & strategizing, tool selection & use, coding & system design. Design, develop and own best practices, tools and processes across various evaluation problems, e.g. query intent classification, standardizing the use of best statistical principles to handle LLM stochasticity, industry benchmarking.

Requirements

Masters/PhD degree in Statistics, Mathematics or Computer Science, or another quantitative field
5+ years of experience as a Masters degree holder, 3+ as a PhD degree holders
Strong in statistics and/or machine learning. Experience in applying these skills into tangible improvements in products, internal tools, and processes in a pragmatic way that puts business urgencies first.
Very proficient in Python, e.g. proficient enough to maintain an internal source-controlled library used by dozens of others.
Concise and precise in written and verbal communication. Technical documentation is your strong suit.
Proficient in SQL and the modern data stack (e.g. source-controlled dbt pipelines for ETL/ELT).
Strong at defining good product KPIs/guardrail metrics, dashboarding and analysis of raw data to derive strategic insights.
Experience in B2B SaaS.
Experience working on ranking, developing, and maintaining A/B experimentation platforms and/or ML measurement problems.
Passionate about using AI to improve the productivity of data teams as well as non-data professionals trying to derive more value of their company’s data.

Responsibilities

Collaborate with product data science and engineering teams to identify techniques, tooling and process improvements in online A/B experimentation to assist rigorous decision making across all relevant product domains.
Develop and maintain our A/B experimentation platform based on stakeholder feedback
Write code or identify vendors to deploy these techniques to production in a scalable manner that’s easy to use by engineering, product data science, product management and design teams
Conduct end to end evaluation of various hero use cases like document/URL uploads, including evaluation set generation, coming up with evaluation criteria and methods to interpret the results.
Break down end to end evaluations into more granular evaluation of various tasks & skills including but not limited to content summarization/analysis/generation, multi-step reasoning & strategizing, tool selection & use, coding & system design.
Design, develop and own best practices, tools and processes across various evaluation problems, e.g. query intent classification, standardizing the use of best statistical principles to handle LLM stochasticity, industry benchmarking.