Member of Technical Staff - Enterprise Model Evaluation

xAI•Palo Alto, CA

2d•Onsite

About The Position

The Model Evaluations team aims to design and implement xAI’s evaluations that shapes how we understand, measure and improve our model’s capabilities. You will work at the intersection of research and product to develop and implement model evaluations that give us high signal into merging model capabilities and robust evaluation infrastructure that enables fast iterations of our models. Your work will be essential to xAI’s mission of understanding the universe. You will collaborate closely with the training and product teams to ensure our models meet the highest standards before deployment. This is a technical leadership role where you will be expected to drive both the vision and implementation of our model evaluations.

Nice To Haves

Proven expertise in designing and implementing sophisticated evaluation frameworks for machine learning models, especially LLMs.
Experience with statistical analysis, experimental design, and benchmarking AI systems in real-world settings.

Responsibilities

Design and implement next-generation evaluation suites beyond traditional benchmarks, creating frameworks that capture real-world utility and performance of Grok in production environments.
Coordinate model evaluation efforts and collaborations to ensure comprehensive coverage and fast iterations.
Integrate Grok into production systems, gain deep insights into real-world environments, and ensure alignment with user needs and business objectives.
Partner with research teams to translate cutting-edge techniques and Grok models into production-ready implementations, optimizing for performance and impact.