Member of Technical Staff (Applied AI Research)

Artificial Analysis•San Francisco, CA

5h•Onsite

About The Position

Artificial Analysis is the leading independent AI benchmarking company, supporting labs, engineers, and enterprises in understanding AI capabilities and making critical decisions about their AI strategies. Our benchmarks are trusted by hundreds of thousands of users and are the go-to reference for leading AI labs, major publications, media, investors, and policymakers. We are a rapidly growing team, backed by prominent industry leaders. This role is at the forefront of AI measurement, where you will design evaluations that set the standard for how AI is measured. You will build novel benchmarks and datasets, evaluate every major model upon release, work with frontier labs on pre-release models, and publish analysis that shapes industry understanding. The goal is to become a world expert in modern AI through applied research with immediate industry impact.

Requirements

3+ years of relevant professional experience, across industry or research.
Intense interest in AI, a desire to become a world expert in the field.
Strong analytical and coding skills.
Proficiency in Python and data analysis.
Genuine, demonstrable interest and knowledge of Frontier AI.
Informed opinions about where AI is heading.
Fit one of the following profiles: AI and Machine Learning Backgrounds (ML Engineer, ML Researcher, Research Engineer, AI Engineer, Forward Deployed Engineer, Technical PM, or similar roles at AI companies or AI-focused teams) with hands-on experience with modern AI systems and understanding of how models work at a technical level.
Fit one of the following profiles: Strategy Consulting Backgrounds (Management Consultant, Associate, Engagement Manager, Data Scientist, or similar roles at firms like McKinsey, BCG, Bain, or equivalent) with strong preference for candidates with experience within a Data Analytics division such as QuantumBlack, AI by McKinsey, BCG X or equivalent. Must know how to structure ambiguous problems, build analytical frameworks, and communicate findings to senior stakeholders, with genuine technical interest in AI and the ability to code.
Fit one of the following profiles: Technical Product Management Backgrounds (Founding Engineer, Product Manager, Technical Co-founder, Head of Product, or generalist roles at early-stage AI companies) with experience building and shipping AI products in a fast-moving environment, operating across research, engineering, product and commercial work simultaneously.

Nice To Haves

Experience within a Data Analytics division such as QuantumBlack, AI by McKinsey, BCG X or equivalent (for Strategy Consulting Backgrounds).

Responsibilities

Design Frontier Evaluations: Conceive, build and ship novel evaluation methodologies that advance how AI capabilities are measured, and that stay ahead of what frontier models can do.
Build Evaluation Datasets and Infrastructure: Construct the datasets, harnesses and scoring systems behind our benchmarks, engineered for contamination resistance and repeatability at frontier scale.
Evaluate Every Major Model: Run our evaluation suite across frontier releases as they land, and own the integrity of the results the industry quotes.
Publish Influential Analysis: Produce the reports, indexes and data visualizations that shape how labs, enterprises and the broader industry understand AI progress.
Work with Frontier Labs on Pre-Release Models: Benchmark the leading labs’ systems, including pre-release and newly launched models, working directly with their research teams.
Become AI-Native: Embrace an AI-native workflow, using cutting-edge AI tools to generate leverage in a fast-changing industry and maintain our competitive edge in AI benchmarking.