Manager, Testing Operations & Evaluation (Johns Hopkins Center for Talented Youth)

Johns Hopkins University•Baltimore, MD

44d•Hybrid

About The Position

Founded in 1979, the Johns Hopkins Center for Talented Youth (CTY) is a center for innovation dedicated to advancing the field of gifted education through research on testing, programs, and other supports for advanced learners. On the premise that advanced learners come from every community and every demographic, we aim to understand and inform how these students need to be nurtured to achieve their full potential as learners and global citizens. CTY also runs in-person and online academic programs for bright learners in order to cultivate a community of advanced learners and connect students in and outside the classroom. We are seeking a Manager, Testing Operations & Evaluation who will support the administration, evaluation, and improvement of CTY’s assessment programs. Reporting to the Director of Research and Assessment, this role oversees testing logistics, trains and supports proctors, manages secure test materials, and conducts applied data analyses that inform program effectiveness. The manager will also collaborate with senior leadership and researchers on studies of reliability, validity, and student outcomes. This is an excellent opportunity for someone with a background in psychometrics, quantitative psychology, or educational measurement who enjoys combining hands-on operations with meaningful data analysis and research collaborations.

Requirements

Master’s Degree in Psychology, Educational Measurement, Psychometrics, Quantitative Psychology, or related field.
Three years of experience (inclusive of doctoral-level training) in testing operations, assessment administration, or applied educational measurement. Experience supporting norming studies, reliability/validity research, or item bank development.
Additional education may substitute for required experience and additional related experience may substitute for required education beyond a high school diploma/graduation equivalent, to the extent permitted by the JHU equivalency formula.

Nice To Haves

Communication - Strong written and verbal skills, with ability to translate technical results into accessible reports for non-technical stakeholders.
Organization - Strong project management skills; ability to manage multiple priorities and meet deadlines.
Values - Commitment to diversity, equity, inclusion, and anti-racism in testing and evaluation practices.
Evaluation - Experience designing and implementing evaluation instruments, conducting multivariate analyses, and developing dashboards or data visualizations.
Testing Operations - Experience training and managing proctors, overseeing secure test administration, and maintaining compliance with testing standards.
Collaboration - Demonstrated success working with schools, families, or external stakeholders in test administration contexts.
Professional Growth - Interest in contributing to psychometric innovation, with potential to grow into higher-level leadership in testing and evaluation.
Proficiency in R or SPSS, with ability to write and edit scoring programs; familiarity with software such as Remark, Qualtrics, or similar survey/evaluation tools.

Responsibilities

Manage logistics for the administration of CTY-owned tests, including secure storage, inventory, and distribution of materials.
Train and support internal and external test administrators (proctors) to ensure compliance with CTY policies and the Standards for Educational and Psychological Testing.
Update and maintain CTY’s testing manuals, proctor training materials, and operational procedures.
Serve as vendor point of contact for CTY testing solutions
Use software tools (e.g., Remark, SPSS) to score assessments, process results, and generate reports.
Beta test new computer-based forms and assists with troubleshooting.
Collect and manage evaluation data on student progress and program effectiveness.
Conduct applied analyses (e.g., descriptive statistics, regression, basic psychometric checks) to inform program decision-making.
Develop evaluation instruments and surveys in collaboration with other relevant departments across the organization.
Maintain records, databases, and documentation related to testing and evaluation.
Prepare concise reports and data visualizations for CTY leadership and stakeholders.
Serve as a resource for families, schools, and contracted partners regarding testing logistics, score interpretation, and best practices.
Support communications with partners on license renewals and testing agreements.
Collaborate with senior leadership on validation studies, norming projects for current SCAT, and the development of new assessments.
Manage external testing projects of varying scale, including those with international organizations, and oversee contract staff as needed.
Assist in cross-functional projects and research initiatives as assigned.
Participate in ongoing training to stay current with testing best practices.
Stay current on CTY’s strategic priorities and orientation in the field.
Participate in academic conferences as budget allows.
Other duties as assigned.