Applied Scientist IV

Oracle

84d

About The Position

The Principal Applied Scientist will play a key role in the evaluation of LLM and LMM models, with a focus on creating innovative solutions to real-world problems. This role requires a deep understanding of machine learning techniques, including foundation model evaluation, traditional NLP/CV metrics, LLMaaJ techniques, human evaluation, confidence estimation, agentive application evaluation, and RAG application evaluation. The successful candidate will have experience in conducting in-depth research, producing production-ready code, and collaborating with cross-functional teams to integrate evaluation capabilities into various applications and products.

Requirements

• Deep understanding of machine learning techniques.
• Experience with foundation model evaluation, traditional NLP/CV metrics, LLMaaJ techniques, human evaluation, confidence estimation, agentive application evaluation, and RAG application evaluation.
• Experience in conducting in-depth research.
• Proficiency in programming languages, such as Python.
• Experience with machine learning frameworks, such as TensorFlow or PyTorch.
• PhD in Computer Science, Mathematics, Statistics, Physics, Linguistics, or a related field, with a dissertation or thesis centered on machine learning techniques, preferred.

Nice To Haves

• Master's or Bachelor's degree with relevant experience will also be considered.

Responsibilities

• Conduct in-depth research on foundation model evaluation.
• Produce production-ready code for handoff to engineering counterparts.
• Help build and mentor a high-performing team of scientists and engineers.
• Work closely with cross-functional teams to integrate evaluation capabilities into various applications and products.
• Identify new opportunities for evaluation and explore emerging technologies.
• Stay up-to-date with industry trends and advancements in evaluation.
• Design and execute experiments, research new algorithms, and find new ways of optimizing risk, profitability, and customer experience.

What This Job Offers

• Career Level: Senior
• Education Level: Ph.D. or professional degree
• Number of Employees: 5,001-10,000 employees