AI Data Scientist
About The Position
Ro is building a team focused on shipping LLM-powered products across the patient experience, clinical operations, and internal tooling. We're hiring an Applied AI Scientist to help measure, evaluate, and improve our AI systems. You'll answer one of the most important questions in applied AI: "Is this actually working?" You'll design evaluations, analyze production behavior, run experiments, and partner closely with engineers to improve the quality of AI-powered features. You'll work on real production systems while learning modern evaluation techniques from experienced teammates.
Requirements
- 1–4 years in data science, analytics, applied ML, or a closely adjacent role.
- Strong Python and SQL skills.
- Hands-on experience building or evaluating LLM-powered applications through work, research, school, hackathons, or side projects.
- You’re curious about how AI systems work beyond the model itself, including retrieval, prompting, evaluation, and production behavior.
- You’re comfortable communicating analytical findings clearly to engineers, product managers, and operational stakeholders.
- You are excited about learning rapidly in a field where best practices continue to evolve.
Nice To Haves
- Experience with evaluation tooling, A/B testing frameworks, production model monitoring, healthcare, or other operations-heavy environments.
Responsibilities
- Build evaluation datasets, rubrics, and synthetic test cases for LLM-powered features across patient and internal workflows.
- Analyze production logs to identify model failures, hallucinations, quality issues, and operational bottlenecks.
- Design and run experiments end-to-end, including hypothesis development, dataset creation, evaluation, analysis, and recommendations.
- Track key product and operational metrics including resolution rate, handle time, touches to resolution, latency, and quality and identify opportunities for improvement.
- Partner with engineers to validate improvements and productionize successful experiments.
- Help build or integrate tooling and dashboards that make AI performance easy to understand and monitor.
Benefits
- Competitive equity and benefits package
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Education Level
No Education Listed
