Software Engineer, Agent Optimization & Evaluation
About The Position
Reddit is building a next-generation AI developer platform: an agent stack intended to help employees build, test, review, and ship software efficiently while preserving control over quality, costs, and long-term AI strategy. The optimization harness measures and improves agents across the software development lifecycle through model benchmarking, regression detection, routing optimization, production feedback, and developer-productivity improvements. The Software Engineer, Agent Optimization & Evaluation will lead Reddit's agent optimization and evaluation platform. The role builds systems to measure and improve the effect of agents on engineering velocity, including Reddit-specific evaluation environments, intelligent model orchestration, and feedback loops informed by production usage. It is a hands-on role involving production code and ownership of new systems, with support from platform and infrastructure partners. The position reports to Kyle Maxwell and works with Platform Engineering, Infrastructure, and Developer Productivity.
Requirements
- 5–12+ years of engineering experience.
- Extensive experience building and shipping large-scale production software systems.
- Experience designing evaluation, experimentation, benchmarking, or measurement systems for AI, search, developer platforms, or other complex distributed systems.
- Strong understanding of modern LLMs, AI agents, and the engineering systems required to deploy them effectively.
- Experience building systems that optimize across multiple dimensions, including quality, latency, reliability, and cost.
- Excellent software engineering fundamentals and a passion for building production-quality systems.
- Ability to operate independently in ambiguous environments and drive technical direction for highly strategic initiatives.
- Strong cross-functional collaboration skills and experience working across engineering, infrastructure, and platform organizations.
Responsibilities
- Design and build Reddit's agent optimization harness to evaluate, benchmark, and improve AI agents across software engineering workflows.
- Develop Reddit-specific evaluation environments to measure agent performance against meaningful engineering tasks.
- Build systems for offline replay, automated evaluation, synthetic task generation, regression detection, and human-in-the-loop feedback.
- Design model-routing and orchestration strategies that optimize quality, latency, and cost.
- Define metrics and experimentation frameworks to measure developer productivity and engineering velocity.
- Build tooling to analyze agent behavior, investigate failures, and accelerate iteration.
- Partner with Platform Engineering, Infrastructure, and Developer Productivity to integrate agent capabilities into engineering workflows.
- Help shape Reddit's long-term strategy for a flexible, model-agnostic agent platform that gives the company control over its AI capabilities and economics.
- Write production code and build new systems with ownership and autonomy.
Benefits
- Comprehensive healthcare benefits and income replacement programs.
- 401(k) with employer match.
- Global benefit programs covering workspace, professional development, and caregiving support.
- Family planning support.
- Gender-affirming care.
- Mental health and coaching benefits.
- Flexible vacation and paid volunteer time off.
- Generous paid parental leave.
- Equity in the form of restricted stock units; depending on the position offered, the role may also be eligible for commission.
- Medical, dental, and vision insurance for U.S.-based employees.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Education Level
No Education Listed
