Platform Engineer, Intern

METR•Berkeley, CA

12d•Onsite

About The Position

METR is a nonprofit research organization focused on assessing AI capabilities, risks, and mitigations, particularly concerning AI R&D automation and misalignment. The organization has a history of conducting significant AI safety evaluations and aims to build a team of ambitious individuals to address critical challenges in AI risk. This role is for a Platform Engineering Intern who will contribute to building and maintaining METR's evaluation platform, which is crucial for their expanding research efforts. The intern will work directly with the infrastructure team, reporting to Mischa Spiegelmock, the team lead. The role involves learning independently with guidance, debugging and improving LLM evaluations, deploying and managing cloud infrastructure, building observability and monitoring systems, developing automation agents, and supporting AI researchers.

Requirements

Passionate about software, open source, and learning.
Works on personal projects for fun and has a track record of teaching themselves new things.
Deep familiarity with UNIX/Linux systems.
Some low-level programming experience (C, C++, or Rust).
Knowledge of LLMs, including transformers and embeddings, or experience with ML work.
Basic web development experience (full-stack, React, TypeScript).

Nice To Haves

AWS (IAM, ECS, Lambda)
PostgreSQL
Infrastructure-as-code (Terraform, Pulumi, CDK)
Kubernetes administration
Experience with LLM evaluation tooling like Inspect.

Responsibilities

Helping build and maintain the world's leading open-source LLM evaluation platform.
Debugging and improving LLM evaluations.
Deploying and managing cloud infrastructure.
Building robust observability and monitoring of our cloud platform.
Building agents to automate tasks.
Supporting AI researchers in their quests to challenge and confound the leading AI models.