Senior Applied AI/ML Scientist

Sprout Social

1d•$168,000 - $277,200•Remote

About The Position

Sprout Social is looking for a Senior Applied AI/ML Scientist to join its AI, Data, and Intelligence Business Unit and drive advancements to our suite of agentic and ML capabilities that are the foundation of Sprout Intelligence. Sprout Social empowers businesses worldwide to harness the power of social media. Processing over one billion social messages daily, our platform serves essential insights to over 30,000 brands. We are weaving AI throughout our products, and this role sits at the center of making that AI trustworthy: as we ship agents across our product suite, the hardest and most valuable problem is knowing whether an agent is actually good, and making it better. That is the work of this role.

Requirements

5+ years of applied AI/ML experience, with a strong track record in the evaluation, measurement, and quality of AI/ML or LLM systems that shipped and drove measurable impact.
Demonstrated experience designing evaluation frameworks, building and calibrating judges or automated evaluators, and establishing their reliability, for ML or LLM/agentic systems.
Strong judgment in model selection and validation: assessing whether one model or approach is genuinely better than another, and why.
Rigorous experimentation and statistical skills: A/B testing, power analysis, and sound measurement design.
Foundational ML and modeling knowledge (Python, common ML frameworks, transformers and embeddings) sufficient to reason about and evaluate the systems you assess.
SQL and comfort with large datasets.
Ability to work closely with product, design, and engineering, and to translate technical quality questions into terms stakeholders can act on.

Nice To Haves

Direct experience evaluating LLM-driven or agentic products in production (prompt quality, judge design, hallucination and quality measurement, tool-calling reliability).
Experience defining a quality bar or evaluation methodology that other people and teams then built against.
Rapid prototyping with new AI/ML techniques to assess their applicability.
Familiarity with eval tooling and observability (e.g. Datadog, Phoenix, or similar) and MLOps practices.
Experience influencing product roadmaps with evidence about AI quality and feasibility.
Awareness of AI/ML model security, bias mitigation, and responsible-AI practices.

Responsibilities

Own end-to-end agent and agentic feature development, including defining requirements, definitions of success, evals, guardrails, the necessary tools; and working with engineers, designers, and PMs to execute.
Lead model selection, development and validation: rigorously assess whether a model or configuration is genuinely better, and make well-reasoned adoption decisions.
Design and run rigorous experimentation (A/B testing and other methods) to measure the real impact of AI features on customers.
Set and evolve the quality bar across agents, and partner with engineers who build and run the agents and the eval infrastructure so your judgment scales across many systems rather than one.
Identify opportunities to improve the capabilities of our agent platform, ranging from prompt iteration, reusable judges, context window management, tool use, adopting new state of the art models.
Communicate AI capabilities, limitations, and quality clearly to product, design, engineering, and leadership, so decisions are grounded in evidence.

Benefits

Premium BCBSIL medical, dental (high/low plans), and vision (Eyemed) insurance for you and your eligible dependents.
Full, free access to Modern Health for you and your dependents, including coaching, therapy sessions, and digital wellness resources.
401(k) plan with a 50% company match on your first 6% of contributions (a 3% total match).
100% employer-paid Life and Disability insurance for your peace of mind.
A flexible PTO policy, supplemented with additional company-wide Rest & Recharge days throughout the year.
Up to 16 weeks of paid leave for new parents to support you in expanding your family.
A $1,000 USD annual Lifestyle Spending Account to spend on your physical, mental, and financial well-being.
A one-time $550 USD stipend to set up your home office, plus a monthly $50 USD stipend for internet.
16 hours of paid volunteer time annually, plus a $100 annual match for your charitable donations.
Access to pre-tax commuter benefits, subsidized child/eldercare (Care.com), discounted pet insurance (Figo), and no-cost personalized financial wellness support through Your Money Line.