Psychometrics and Quantitative Research Director

Center for Applied Linguistics•Washington, DC

5d•$114,300•Remote

About The Position

The Director, Psychometrics and Quantitative Research (PQR) is a critical member of the leadership team for the CAL-WIDA Partnership at CAL. The CAL-WIDA Partnership team works closely with WIDA, our primary client, to execute WIDA’s vision with respect to their testing programs. The CAL-WIDA Partnership is responsible for developing test content and conducting psychometric analyses for WIDA’s suite of assessments. Under the guidance of the Senior Director of the CAL-WIDA Partnership, the Director, PQR oversees and leads the work of the PQR team. This includes overseeing: operational and field test psychometric analyses; the preparation and quality control of annual technical reports; and the development of ancillary files including business rules and computer code that feeds into the psychometric analyses. The Director, PQR is also responsible for conceptualizing and overseeing special research activities related to WIDA’s suite of assessments, including verification studies, scale maintenance studies, and the like. The Director, PQR also provides leadership and guidance in quantitative and research methodologies to other researchers at CAL. The Director, PQR also assists the Senior Director in leading overall CAL-WIDA partnership activities and serves on CAL’s Extended Leadership Team.

Requirements

Ph.D. in Educational Measurement, Psychometrics, or related field
More than ten years of extensive experience in directing, designing, and conducting applied Psychometric analyses for high stakes operational testing programs.
Strong demonstrated experience overseeing psychometricians and analysts in successfully applying the Rasch model to a variety of psychometric issues.
Experience in leading analyses on performance and polytomously-scored tasks.
Demonstrated proficiency in using SPSS, Winsteps, Facets, R and other statistical and psychometric software, and mentoring other staff in the use of these packages.

Nice To Haves

Experience working within a high-stakes, standardized psychometric program in a K-12 U.S. Educational Context, and preferably in language testing.
Experience overseeing and directing a team of psychometricians and research analysts.
Experience communicating effectively in English with clients and internal and external stakeholders in high-stakes contexts.
Experience overseeing a team in producing timely error-free deliverables.
Demonstrated ability to oversee and lead a diverse, multi-disciplinary professional team of researchers and language testers.
Demonstrated ability to communicate both orally and in writing with a wide variety of audiences, including researchers, test developers, and client leadership teams.
Strong demonstrated experience with computerized adaptive assessments.
Experience in conducting standard setting study and/or alignment studies.

Responsibilities

Direct the work of the (PQR) team, working with the PQR Project Manager and PQR Team Manager to ensure all work flows smoothly with appropriate staff assignments and resourcing for the CAL-WIDA Partnership program
Direct the work of the PQR team in ensuring quality control of all deliverables
Provide oversight on the technical quality of language assessment and analysis activities conducted by the PQR team
Lead multiple technical operational psychometric and research projects for the CAL-WIDA Partnership program
Provide guidance and oversight to the PQR psychometric staff on technical aspects of their work
Lead, design, conduct, and document psychometric and statistical analyses for WIDA ACCESS and other WIDA Consortium-related projects
Investigate, evaluate, recommend, and implement new approaches to conducting psychometric and statistical analyses
Review and approve written research reports and technical documentation, such as the Annual Technical Report on WIDA ACCESS
Direct ongoing psychometric activities related to CAL’s BEST assessments, including input on study design, oversight of work to maintain accreditation of the BEST assessments, and providing input on technical reports.
Serve as the supervisor of record for CAL psychometricians, the PQR Project Manager, and the PQR Team Manager
Provide leadership in planning and implementing CAL-WIDA partnership activities
Assist with new project development and monitor current project budgets
Provide guidance to CAL researchers on proposal design, research methodology, and quantitative analytical approaches
Participate in writing quantitative-oriented portions of proposals as requested
Provide leadership and oversight to maintain quality of psychometric aspects of BEST assessments.
Develop and maintain a current list of professionals with expertise in psychometrics and quantitative methodology as potential contractors for CAL
Represent CAL PQR in WIDA Technical Advisory Committee meeting
Serve as CAL’s representative in WIDA Subcommittee and WIDA Board meeting, as needed
Other Duties as Assigned