We’re looking for an AI Platform Engineer to evolve and extend our internal evaluation framework for assessing the quality of our AI-driven experiences at Khan Academy. This engineer will have worked with enough eval systems to quickly make sense of Khan's internal eval framework and recognize opportunities for improvement. This is largely a software development role, but domain experience with AI eval is essential for appreciating the hill-climbing and data science workflows we need to support. Soft skills will be important for gathering internal requirements, getting buy-in for changes, and then developing documentation and training materials. You’ll work closely with ML data engineers and platform developers to help internal teams adopt an eval-driven development process incorporating offline benchmark tests and online experiments. As a Platform Engineer focused on evaluation , you’ll be expected to: Be fluent in the range of offline and online evaluation strategies, and when to apply the techniques over the lifecycle of development Have intuitions about how to specify eval pipelines succinctly using declarative syntax Understand the role of stratified datasets and ground truth labeling Appreciate the range of eval scoring schemes from human raters to automated LLMs-as-judge We are a remote-first organization and we strive to build using technology that is best suited to solving problems for our learners. Currently, we build with Go, GraphQL, JavaScript, React & React Native, Redux and we adopt new technologies like LLMs when they’ll help us better achieve our goals. At Khan, one of our values is “Cultivate Learning Mindsets”, so for us, it’s important that we’re working with all of our engineers to help match the right opportunity to the right individual, in order to ensure every engineer is operating at their “learning edge”. Currently, we are focused on providing equitable solutions to historically under-resourced communities of learners and teachers, and guided by our Engineering Principles . You can read about our latest work on our Engineering Blog . A few highlights: Incremental Rewrites with GraphQL Our Transition to React Native Go + Services = One Goliath Project How Engineering Principles Can Help You Scale How to upgrade hundreds of React components without breaking production