Research Scientist, LLM Evaluation & Post-Training

Centific

1d • $150,000 - $300,000 • Hybrid

About The Position

As a Research Scientist, LLM Evaluation & Post-Training, you will be at the frontier of how evaluation design, measurement strategy, and feedback signals drive model improvement across Centific’s AI platform products. This is a high-impact individual contributor and collaborative research role that sits at the intersection of applied ML research, enterprise AI product development, and customer-facing scientific consulting. You will lead research programs that define next-generation evaluation-driven post-training workflows, develop rigorous benchmark frameworks, and partner directly with leading AI organizations to deliver credible, actionable model improvement insights. This role offers the opportunity to shape Centific’s internal research agenda, build reusable scientific assets, and publish at top-tier venues.

Requirements

• MS or PhD in Computer Science, Machine Learning, Statistics, Applied Mathematics, AI, or a related quantitative field (PhD strongly preferred).
• 5+ years of relevant experience in applied ML research or research science, with substantial work in LLMs or foundation models (graduate research counts).
• Demonstrated experience with LLM evaluation, benchmarking, alignment, post-training, or model quality research.
• Strong foundation in experimental design, statistical analysis, and scientific reasoning for ML systems.
• Strong Python coding skills for research experimentation, data processing, evaluation pipelines, statistical analysis, and visualization.
• Hands-on experience with modern ML frameworks (PyTorch, Hugging Face, JAX/TensorFlow).
• Ability to evaluate and compare human and automated evaluation methods, including tradeoffs in cost, reliability, validity, and scalability.
• Experience designing reproducible evaluation studies across datasets and model versions.
• Strong written and verbal communication skills; able to present nuanced technical conclusions, assumptions, and limitations clearly to both research and non-technical audiences.

Nice To Haves

• Hands-on experience running fine-tuning or post-training experiments (SFT, preference optimization, RLHF/RLAIF-style workflows).
• Experience with multimodal evaluation (text-image, audio, video) and long-context benchmarking in real-world settings.
• Experience designing multi-turn, interactive, or agentic evaluation protocols.
• Publications and/or open-source benchmark contributions in LLM evaluation, post-training, alignment, or related areas at top venues (NeurIPS, ICML, ICLR, ACL, EMNLP, etc.).
• Experience in customer-facing applied research, technical consulting, or cross-functional product/research collaboration.
• Familiarity with safety, trustworthiness, and governance considerations in GenAI evaluation.

Responsibilities

• Define and execute a rigorous research agenda focused on LLM evaluation and post-training, with emphasis on evaluation-driven model improvement.
• Design experiments to study how evaluation methodologies impact fine-tuning and post-training outcomes.
• Develop and validate comprehensive evaluation frameworks for LLM and multimodal systems, covering benchmark and task design, scoring methods, judge/model-assisted evaluation, human evaluation protocols, and robustness/stress testing.
• Lead research on frontier evaluation domains including long-context, cross-modal, and dynamic multi-turn evaluations.
• Study effectiveness and limitations of existing techniques and propose improved methodologies with clear validity and scalability tradeoffs.
• Analyze model behavior and failure patterns; generate actionable recommendations for model improvement and evaluation redesign.
• Translate findings into practical improvements for customer solutions and Centific’s internal platforms.
• Partner with Language Data Scientists to integrate human-in-the-loop and synthetic data/evaluation strategies, and with AI/ML Research Engineers to translate research methods into scalable evaluation and post-training pipelines.
• Engage with customer technical stakeholders at leading AI organizations to understand evaluation goals, review methodologies, and provide expert scientific recommendations.
• Serve as a credible technical peer to research and engineering leaders.
• Contribute to internal benchmark datasets, reusable evaluation frameworks, and research assets.
• Produce high-quality technical documentation, internal research reports, and client-facing materials explaining methods, results, assumptions, and limitations.
• Contribute to Centific’s position as a leader in LLM evaluation and post-training through publications, conference presentations, and open-source contributions.

Benefits

• Centific is an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, ancestry, citizenship status, age, mental or physical disability, medical condition, sex (including pregnancy), gender identity or expression, sexual orientation, marital status, familial status, veteran status, or any other characteristic protected by applicable law. We consider qualified applicants regardless of criminal histories, consistent with legal requirements.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Ph.D. or professional degree