AI Researcher

Mercor•San Francisco, CA

5h•$200,000 - $300,000•Onsite

About The Position

About Mercor Mercor is at the intersection of labor markets and AI research. We partner with leading AI labs and enterprises to provide the human intelligence essential to AI development. Our vast talent network trains frontier AI models in the same way teachers teach students: by sharing knowledge, experience, and context that can't be captured in code alone. Today, more than 30,000 experts in our network collectively earn over $1.5 million a day. Mercor is creating a new category of work where expertise powers AI advancement. Achieving this requires an ambitious, fast-paced and deeply committed team. You’ll work alongside researchers, operators, and AI companies at the forefront of shaping the systems that are redefining society. Mercor is a profitable Series C company valued at $10 billion. We work in-person five days a week in our new San Francisco headquarters. About the Role Silicon Valley’s top AI companies work with Mercor to find domain experts who can help train and evaluate their models. As a researcher at Mercor, you will be responsible for advancing the frontier of model evaluations to drive model improvements across the industry that create real world economic value. You will be frequently publishing impactful papers with industry leading collaborators, have ample resources to create high-impact datasets, and access to the frontier of evaluation and training data. You will work closely with Mercor’s Forward Deployed Research, Applied AI, and Operations teams, and have unmatched access to evaluate frontier models We are looking for an experienced AI researcher. A track record of LLM evaluation publications is preferred but publication experience in evaluation of other types of models or other AI related publications are of interest as well.

Requirements

PhD or M.S. and 2+ years of work experience in a computer science, electrical engineering, econometrics, or another STEM field that provides a solid understanding of ML and model evaluation.
Strong publication record in AI research, ideally in LLM evaluation. Dataset and evaluation papers are preferred.
Strong understanding of LLMs and the data on which they are trained and evaluated against.
Strong communication skills and ability to present findings clearly and concisely.
Familiarity with data annotation workflows.
Good understanding of statistics.
Willingness to work 6 days a week, with monday-friday in person in San Francisco

Responsibilities

Build benchmarks that measure real world value of AI models.
Publish LLM evaluation papers in top conferences with the support of the Mercor Applied AI and Operations teams.
Push the frontier of understanding data ROI in model development including multi-modality, code, tool-use, and more.
Design and validate novel data collection and annotation offerings for the leading industry labs and big tech companies.