Backend Software Engineer, Cooperative AI (Evals)
About The Position
The Support Automation team at OpenAI uses AI to automate and improve work across the organization, developing reusable products for areas such as customer operations and engineering. The Backend Software Engineer will design and build evaluation infrastructure to measure the quality of OpenAI’s support automation. This is a technical, cross-functional role focused on robust systems and backend services that support how knowledge is created, accessed, and applied across OpenAI, with close collaboration with Data Science and Research partners to build evaluations at scale. The role includes designing reliable, reproducible, and extensible evaluation pipelines; building continuous evaluation monitoring and feedback systems; developing backend services and APIs for automation and knowledge systems; structuring data for downstream systems and AI workflows; and owning the development lifecycle of backend systems and internal platform capabilities. The work combines rapid iteration with attention to scale and maintainability.
Requirements
- At least 4 years of backend engineering experience at product-driven companies, excluding internships.
- Proficiency in backend technologies; the stated stack includes Python, FastAPI, and Postgres.
- Experience designing and scaling distributed systems, APIs, or data processing pipelines.
- Experience building AI agents or applications, including designing evaluations and improving performance through prompting or scaffolding.
- Familiarity with LLM evaluation methods and experience with patterns such as multi-agent workflows, tool use, or long context.
- A pragmatic mindset and comfort shipping iteratively while building toward a long-term vision.
Nice To Haves
- Experience creating production evaluations and/or measuring the performance of ML or LLM models at scale.
Responsibilities
- Design reliable, reproducible, and extensible evaluation pipelines.
- Build infrastructure for continuous evaluation monitoring, including regression and drift monitoring, robust golden datasets, and feedback loops to strengthen support automation.
- Design, build, and maintain backend services and APIs for intelligent automation and knowledge systems.
- Integrate and structure data across internal platforms and transform it for downstream systems and AI workflows.
- Collaborate with data, research, and engineering teams to integrate OpenAI models into high-leverage workflows.
- Own the full development lifecycle of new backend systems and internal platform capabilities.
- Build for scale and maintainability while rapidly iterating on new ideas.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Education Level
No Education Listed
