Platform Engineer, Intern

METR•Berkeley, CA

15d•Onsite

About The Position

METR is a nonprofit research organization focused on assessing AI capabilities, risks, and mitigations, particularly concerning AI R&D automation and misalignment. The organization has a history of setting precedents in AI safety evaluations. They are seeking Platform Engineering Interns to help build and maintain their evaluation platform, which is crucial for their expanding operations. The interns will work directly with the infrastructure team and report to Mischa Spiegelmock, the team lead. The role involves learning independently with some guidance, debugging and improving LLM evaluations, deploying and managing cloud infrastructure, building observability and monitoring systems, automating tasks with agents, and supporting AI researchers.

Requirements

Passionate about software, open source, and learning.
Works on personal projects for fun and has a track record of teaching themselves new things.
Deep familiarity with UNIX/Linux systems.
Some low-level programming experience (C, C++, or Rust).
Knowledge of LLMs, including transformers and embeddings, or experience with ML work.
Basic web development experience (full-stack, React, TypeScript).

Nice To Haves

AWS (IAM, ECS, Lambda)
PostgreSQL
Infrastructure-as-code (Terraform, Pulumi, CDK)
Kubernetes administration
Experience with LLM evaluation tooling like Inspect

Responsibilities

Helping build and maintain the world's leading open-source LLM evaluation platform.
Debugging and improving LLM evaluations.
Deploying and managing cloud infrastructure.
Building robust observability and monitoring of our cloud platform.
Building agents to automate tasks.
Supporting AI researchers in their quests to challenge and confound the leading AI models.