Staff Data Scientist, AI Evaluation

Syllo
•$160,000 - $200,000•Remote

About The Position

Syllo is seeking a Senior Data Scientist to evaluate and improve the quality and performance of its AI systems. This hands-on role sits at the intersection of data science, engineering, and applied AI. The role works directly with production data and systems and partners closely with Engineering, Product, and Analytics. The position develops rigorous ways to measure system behavior, runs experiments, investigates issues, and turns complex data into actionable decisions.

Requirements

  • Master's degree in computer science, data science, machine learning, or a related field with substantial hands-on experience, or 2–4 years of relevant professional experience.
  • Strong Python and SQL skills and experience working directly with production data.
  • Comfort navigating production code and data systems not previously built by the candidate.
  • Strong foundation in applied statistics, experimentation, and quantitative analysis.
  • Practical experience working with and evaluating modern AI/ML systems.
  • Experience diagnosing complex issues involving data, models, or production systems.
  • Ability to turn ambiguous questions into structured investigations and clearly communicate the results.

Nice To Haves

  • Experience building AI/ML evaluation frameworks, benchmarks, or monitoring systems.
  • Experience with information retrieval, search, or large-scale information systems.
  • Familiarity with modern cloud data platforms and production data infrastructure.
  • Experience building internal analytics or observability tooling.

Responsibilities

  • Develop quantitative methods and tooling to evaluate AI system quality and performance.
  • Design and run experiments to compare approaches, understand tradeoffs, and measure the impact of changes.
  • Build repeatable processes to identify regressions, anomalies, and changes in system performance.
  • Extract, transform, and analyze production data to support technical and product investigations.
  • Diagnose complex issues involving data, models, and production systems, identifying root causes and potential solutions.
  • Partner with Engineering to improve measurement of system quality, reliability, and performance.
  • Build tools and dashboards that make system performance easier to understand and monitor.
  • Clearly communicate findings, tradeoffs, and recommendations to technical and non-technical stakeholders.

Benefits

  • Health insurance
  • Equity
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service