Senior Data Scientist, Applied AI
About The Position
The Senior Data Scientist, Applied AI independently owns the development and evaluation of AI solutions for an assigned use case, working within the team’s established evaluation framework and practices. The role uses professional judgment to choose technical approaches, diagnose issues, and implement solutions within the assigned project, escalating decisions that affect the team’s evaluation strategy, span multiple projects, or involve material risk. Success is measured by the quality and reliability of the systems and evidence delivered for the use case. This is an individual-contributor role with no people-management responsibilities. The Senior Data Scientist collaborates with engineering, product, and domain partners to deliver assigned projects and supports the reliability of owned workflows. Definitive Healthcare is a healthcare intelligence company headquartered outside Boston, operating across North America, Europe, and India and serving more than 2,400 customers. The company describes its culture as collaborative and inclusive, and highlights employee community and affinity-group programs.
Requirements
- Bachelor’s degree.
- At least 5 years of relevant experience in data science, machine learning, applied AI, software engineering, or a related field.
- Hands-on experience building and evaluating LLM applications.
- Strong Python programming, software engineering and testing, and analytical problem-solving skills.
- Strong knowledge of LLMs, embeddings, semantic search, retrieval systems, and RAG architectures.
- Strong hands-on experience with Databricks, including Delta Lake, Unity Catalog, Lakebase, Lakeflow Jobs, and MLflow, for building and deploying scalable, production-grade data and AI solutions.
- Ability to design representative evaluations for an assigned use case, analyze errors, and use results to improve a system.
- Ability to weigh quality, reliability, latency, and cost when recommending an approach for an assigned project.
- Clear communication and effective collaboration with technical and nontechnical partners.
- Ability to operate independently within the team’s established evaluation framework and exercise judgment on technical approach, diagnosis, and issue resolution within an assigned project.
Nice To Haves
- Master’s degree in a quantitative or technical field.
- Experience with agent frameworks and AI evaluation platforms.
- Experience with healthcare data.
Responsibilities
- Independently build, test, and improve LLM, RAG, and tool-using agent workflows for an assigned use case using maintainable, tested Python code.
- Select and apply evaluation criteria, baselines, and test datasets within the team’s established evaluation approach.
- Compare models, embeddings, and retrieval methods for the assigned use case based on quality, reliability, latency, and cost; document recommendations and tradeoffs.
- Investigate retrieval misses, incorrect answers, and workflow failures; determine root causes and implement and validate fixes within the assigned project.
- Build and update test cases, benchmark datasets, and regression checks for the assigned use case.
- Monitor and investigate performance changes in owned workflows and recommend and implement project-level improvements.
- Communicate findings and technical tradeoffs to engineering, product, and domain partners.
- Escalate decisions involving changes to the team’s evaluation approach, significant business risk, sensitive data, material cost commitments, or production release approval to the Manager or appropriate owner.
Benefits
- Medical coverage
- Dental coverage
- Vision coverage
- Unlimited paid time off
- 401(k) plan with employer contribution
- Employees may be eligible for a company bonus or commission plan, depending on the position
- DefinitiveCares community service group
- Employee-led affinity groups
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
