AI Data Scientist

Beth Israel Lahey Health

21h

About The Position

The AI Data Scientist role is designed for individuals who are interested in applying data science, machine learning, and generative AI to healthcare and operational challenges. This position emphasizes fundamental understanding of generative AI and large language models (LLMs), including how models are trained, finetuned, evaluated, and applied responsibly. The role focuses on model development, experimentation, evaluation, and analytical reasoning. The AI Data Scientist will work as part of a multidisciplinary team and receive mentorship while developing skills across the AI and data science lifecycle, from data exploration and modeling to evaluation and communication of results.

Requirements

Bachelor’s degree in Data Science, Computer Science, Artificial Intelligence, Statistics, Mathematics, or a related quantitative field required.
0–2 years of related experience, including internships, research, academic projects, or entry level roles.
Hands on experience through coursework, capstone projects, labs, or internships involving data analysis or AI modeling.
Foundational understanding of data science, machine learning, and generative AI concepts.
Basic familiarity with large language models (LLMs) and common generative AI use cases.
Experience using Python for data analysis, modeling, or experimentation.
Understanding of model evaluation concepts, including metrics and qualitative assessment approaches.
Strong analytical and problem-solving skills with attention to detail.
Ability to explain technical concepts clearly in both written and verbal form.
Willingness to learn new tools, methods, and AI techniques in a collaborative environment.

Nice To Haves

Relevant coursework or academic projects in machine learning, NLP, or AI strongly preferred.
Coursework or project experience in natural language processing (NLP) or generative AI.
Exposure to LLM finetuning methods or experimentation with pretrained models.
Familiarity with LLM evaluation techniques, such as human review, test datasets, or scoring frameworks.
Experience with data visualization tools (e.g., matplotlib, seaborn, Power BI, Tableau).
Awareness of AI ethics, fairness, and responsible AI practices.
Interest in healthcare analytics or working with regulated data.

Responsibilities

Support the development and evaluation of machine learning and generative AI models for healthcare analytics, automation, and decision support.
Build a foundational understanding of large and small language models (LLMs), including basic architecture concepts, embeddings, inference behavior, and limitations.
Assist with prompt design, prompt testing, and prompt optimization to improve accuracy, relevance, and consistency of LLM outputs.
Participate in LLM finetuning activities (such as instruction tuning or parameter efficient finetuning) under guidance and within established workflows.
Help design and execute model evaluation approaches, including quantitative metrics and qualitative review of LLM outputs (accuracy, completeness, bias, hallucinations).
Contribute to retrieval of augmented generation (RAG) projects by assisting with data preparation, document evaluation, and grounding assessments.
Perform exploratory data analysis, data validation, and feature engineering for predictive and descriptive modeling tasks.
Collaborate with senior data scientists, data engineers, clinicians, and business stakeholders to translate problems into data science and AI modeling work.
Create visualizations, written summaries, and presentations to communicate insights and model performance to technical and nontechnical audiences.
Follow Responsible AI practices, including awareness of fairness, bias, privacy, transparency, and appropriate AI use in healthcare.
Participate in team meetings, mentoring sessions, and ongoing learning activities related to AI and data science.