Member of Technical Staff, Frontier Evals

Arcada Labs Incorporated•San Francisco, CA

6d•Onsite

About The Position

Intelligence is the product lab company behind DesignArena, a platform with over 5.5 million users in 8 months, whose leaderboard has been referenced by prominent figures in AI. The team is highly talented, backed by top-tier VCs and angel investors, and is one of the fastest-growing seed-stage startups in San Francisco. Intelligence works closely with leading frontier model providers like OpenAI to rigorously evaluate the capabilities of state-of-the-art multimodal models across various domains including design, web development, game development, image, video, audio, and slide generation, utilizing large-scale platforms they have built. Their flagship product, Design Arena, is a widely referenced benchmark for AI-generated visuals with over 5.3 million users globally. Prediction Arena was the first platform for autonomous model trading with real cash on real-world events, and Social Arena tested AI models' ability to grow and engage audiences on X as independent social media agents.

Requirements

Strong STEM background (Computer Science, Data Science, Statistics, Math, Engineering, Physics, or a related field).
Deep curiosity about frontier AI models.
Excited by understanding model behavior, discovering areas of failure, and building better ways to evaluate models.
Genuine thirst and intellectual drive to be on the frontier of AI development.
Fearlessness to roll up your sleeves, get your hands dirty, and do real work.

Responsibilities

Define how frontier AI models are measured.
Design new benchmarks.
Run experiments.
Analyze model behavior.
Build evaluation methodologies that become trusted signals for the industry.
Shape public leaderboards and evaluation tools shared with frontier labs.
Design genuinely hard and useful evaluations that measure frontier model performance on real-world tasks, and that become industry-leading gold-standards.
Investigate model failures and identify what they reveal about emerging capabilities.
Publish research, technical reports, and analyses that shape how frontier models are evaluated.