Senior Data Scientist — Model Evaluation

SumerSports

9h • $135,000 - $165,000 • Remote

About The Position

The Senior Data Scientist will be focused on model evaluation and quality for our football ML models. You'll contribute to the work that sits between research and production, making sure the models we ship measure what we think they measure. We believe good research is less about having the perfect idea and more about building a rigorous system for testing ideas. Ideas are plentiful. The real work is designing experiments, learning quickly, and following the evidence.

Requirements

• Strong statistical foundations. Calibration (reliability and resolution), bootstrapping, inter-rater reliability.
• Track record contributing to model evaluation on a production ML team.
• Production ML pipeline experience. You've seen schema drift, label noise, and leakage.
• You build tooling, not just consume it.
• Self-starter. Takes strategic direction and shapes their own path.
• AI-native. Builds skills, loops, and tooling to accelerate evaluation throughput.
• Stays current with research. Papers, releases, new techniques.
• Domain absorption capacity. You've gotten fluent on a complex domain (sports, finance, biotech, medical) on a prior job

Nice To Haves

• Football or sports analytics exposure.
• Experience evaluating tracking-data models.
• Skill scores for probabilistic forecasts (Brier, etc.).
• Deep-learning fluency to read training code.
• Human-in-the-loop ML experience.

Responsibilities

• Contribute to model evaluation across our football models. Drive one or two threads end-to-end.
• Build and extend the tooling we use to test, compare, and ship models.
• Work with football subject-matter experts to design evaluation that reflects how the models will actually get used.
• Catch silent regressions before they ship.
• Pushback when a stakeholder asks for a metric that doesn't measure what they think.
• Build the experimentation harness that lets the team test many ideas fast.

Benefits

• Competitive Salary and Bonus Plan
• Comprehensive health insurance plan
• Retirement savings plan (401k) with company match
• Remote working environment
• A flexible, unlimited time off policy
• Generous paid holiday schedule - 13 in total including Monday after the Super Bowl

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed