AI Behavior Researcher - Agent Alignment
About The Position
Transluce is a fast-moving nonprofit research lab building the public tech stack for AI evaluation and oversight. We have contributed foundational research to the study of AI agents and their behaviors, and are using these to study emerging issues in the honesty and alignment of AI agents. As an AI Behavior Researcher, you will lead projects to design and develop automated evaluations of frontier AI systems that are technically sophisticated, scientifically valid, and concretely impactful. You will conduct novel analyses of behaviors related to agentic honesty and alignment. Example behaviors of interest include misreporting results, falsely claiming success, evaluation awareness, and memetic effects within AI swarms. As an early member of a highly collaborative team, you will learn and grow quickly, and work with our governance and infrastructure teams to scale your impact and technical reach.
Requirements
- Expertise on quantitative generative AI evaluation and measurement. Good intuition about how to systematize and operationalize complex concepts and to work backwards from possible failure modes of agents.
- Relevant experience designing and validating automated AI evaluation methods, such as LLM-as-a-judge systems or multi-turn benchmarks.
- Proficiency in Python to implement analysis and evaluation tooling.
- Meticulous, good experimental design, epistemic self-awareness and transparency.
- Ability to iterate quickly and balance between scrappiness and thoroughness based on the impact needs of a project.
- Strong communication skills, low ego, openness to giving and receiving feedback.
Nice To Haves
- Experience running automated evaluations at scale or in a production context.
- Experience conducting controlled human subjects experiments to validate automated evaluation methods.
- Experience in customer-facing, consulting, or forward-deployed roles translating ambiguous stakeholder needs into concrete deliverables.
- Experience and comfort using AI coding agents at work.
Responsibilities
- Develop novel, valid automated evaluations of AI agents' honesty and alignment.
- Write code to implement and run automated evaluations, such as environment simulators or LLM-as-a-judge pipelines.
- Design methods to improve the ecological of automated evaluations, and especially to measure which effects are increasing or decreasing as models become more capable.
- Collaborate with our governance team to deliver high-impact evaluations for public policy.
- Collaborate with scientists and research engineers to productionize best practices in AI behavior evaluation.
Benefits
- benefits
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Education Level
No Education Listed
