Senior Efficacy Research and Psychometrics Analyst, Khan Kids (24 months fixed-term)

Khan Academy • San Mateo, HI

7d • Remote

About The Position

Khan Academy Kids is seeking a Senior Efficacy Research and Psychometrics Analyst for a 24-month fixed-term position to enhance the evidence supporting their assessments and learning content for children aged PreK through 2nd grade. This role involves leading the technical aspects of psychometrics and efficacy research, including managing external research partners, data analysis, and translating research findings into product enhancements. Additionally, the role will oversee analytics and reporting for Khan Academy Kids, maintaining a KPI dashboard and preparing regular reports for leadership, philanthropy, and school partnerships teams. The analyst will collaborate with a small, interdisciplinary team and external research organizations, requiring strong quantitative skills, ideally in psychometrics or educational measurement, with a passion for using evidence to improve early childhood learning experiences.

Requirements

• An advanced degree (PhD or Master's) in psychometrics, educational measurement, quantitative methods or a related field — or equivalent applied experience.
• Strong grounding in psychometric methods, including item analysis, reliability and validity evidence, growth modeling, and Item Response Theory (IRT).
• Proficiency with SQL and a statistical programming language (e.g., R or Python) for extracting, cleaning, analyzing, and visualizing data.
• Experience with (or eagerness to learn) business intelligence tools such as Looker, and comfort building dashboards and data pipelines.
• Familiarity with responsible data-sharing practices in education research, including student data privacy considerations (e.g., FERPA, COPPA) and work with pseudonymized and deidentified data.
• Excellent communication skills: able to translate complex psychometric findings into clear implications for product teams, funders, and school partners.
• Strong project-management and organizational skills, with the ability to coordinate multiple external partners, recurring deliverables, and data-sharing schedules with close attention to detail.
• Proven cross-cultural competency skills, demonstrating self-awareness, awareness of others, and the ability to adopt inclusive perspectives, attitudes, and behaviors to drive inclusion and belonging throughout the organization.
• Motivated by Khan Academy’s mission “to provide a free, world-class education for anyone, anywhere”.

Nice To Haves

• Experience with (or strong interest in) early childhood education and assessment of young learners is a plus.

Responsibilities

• Lead the psychometric research partnership, serving as the primary technical point of contact for external partners, supporting study design, data sharing, psychometric analyses, and interpretation of findings.
• Extract, clean, and securely share assessment data with research partners on a recurring schedule.
• Analyze and interpret psychometric findings, including item difficulty, score distributions, growth analyses, concurrent validity, and IRT-based scoring.
• Partner with product and engineering teams to translate findings into product improvements, such as updates to scoring models and assessment discontinuation rules.
• Collaborate with research partners to develop and publish technical documentation, including the technical manual for the PreK assessment.
• Recruit and manage research partner sites, including executing MOUs, aligning on consent procedures and participant incentives, and coordinating site visits for data collection.
• Partner with the legal team to draft and negotiate data-sharing agreements with districts and research organizations.
• Support efficacy research by coordinating Khan Academy Kids' participation in ongoing studies and contributing to the design of future studies.
• Own Khan Kids’s KPI dashboard, partnering with the engineering team to build and maintain it, including integrations with App Store Connect and Google Play Console.
• Maintain the dashboard for use by marketing, school partnerships, and philanthropy teams.
• Lead recurring reporting, including monthly and quarterly business reviews and periodic target setting.
• Field ad hoc data requests across teams, writing SQL queries to support targeted district outreach or grant reporting.

Benefits

• Competitive salaries
• Ample paid time off as needed
• 8 pre-scheduled Wellness Days in 2026
• Remote-first culture
• Generous parental leave
• An exceptional team that trusts you and gives you the freedom to do your best
• The chance to put your talents towards a deeply meaningful mission and the opportunity to work on high-impact products that are already defining the future of education
• Opportunities to connect through affinity, ally, and social groups
• 401(k) + 4% matching
• Comprehensive insurance, including medical, dental, vision, and life

What This Job Offers

• Job Type: Full-time
• Career Level: Senior