Psychometrician

University of Waterloo•Waterloo, ON

39d

About The Position

We are seeking a Psychometrician to join a new project developing a screening assessment for reading skills in children from kindergarten through grade 3 for the British Columbia Literacy Screening Research Project (BCLSRP)—a large-scale, multi-year research initiative designed to rigorously study, follow, and inform the implementation of literacy initiatives across the province led by Canada Excellence Research Chair in Developmental Science, Dr. Sara Hart. The Psychometrician will lead the technical work involved in standardizing the screener, including item selection and refinement, field test design, ROC curve analysis, IRT modeling, scaling, equating, and norming. The successful candidate will also advise on data collection procedures to ensure that the resulting data support the planned psychometric analyses. This includes tasks such as psychometric modeling, statistical validation of assessment tools, longitudinal data analysis, and the preparation of analytical reports for internal and external stakeholders. The successful incumbent will provide methodological expertise, ensure data integrity, and support evidence‑based decision‑making within the research unit.

Requirements

PhD in quantitative psychology, educational measurement, psychometrics, or a related field; or a Master's degree with substantial applied experience in educational test development.
5+ years of demonstrated experience with mathematical, statistical, and analytical methods, such as ROC curve analysis, IRT and other measurement models.
Experience with relevant statistical software (e.g., R packages such as mirt or TAM, Mplus, flexMIRT, or similar)
Demonstrated experience using different norming approaches.
Demonstrated experience using measurement modeling and its application to assessment development.
Sensitivity to inequities in an educational context, e.g., knowledge of QuantCrit
Superior ability to create and work with statistical models.
Data analysis, management, manipulation, and interpretation.
Working knowledge of various reporting and data visualization tools.
Knowledge of SAS, Python, R, Mplus, or other statistical analysis or data transformation tools.
Demonstrated ability to acquire and use new technologies and techniques.
Demonstrated proficiency in managing competing priorities and time demands from multiple sources.
Ability to independently plan and implement multivariate statistical analyses.
Strong research skills, including an ability to think critically and analytically, to contribute to scientific papers, proposals for research funding, and to make presentations at scientific meetings.
Proven ability to work with data that is confidential in nature.
Ability to multi-task and to work quickly and efficiently under pressure.
Ability to stay focused on a task in a busy workplace.
Ability to maintain a creative and positive approach to support under tight timelines.
Excellent problem-solving skills and resourcefulness.
Excellent written and verbal communication.
Ability to work both independently and collaboratively.
Proven ability to be meticulous, organized, and detail oriented.

Nice To Haves

Familiarity with early literacy assessment or educational measurement contexts is an asset.
Familiarity with the BC educational context is an asset.

Responsibilities

Statistical Modeling, Data Analysis and Reporting · Lead psychometric analyses for a new early literacy screening tool, including dimensionality analyses, item calibration, and creation of benchmarks and different scaled scores.
Apply item response theory and receiver operating characteristic curve frameworks appropriate for early childhood assessment.
Advise on sampling, field testing, and data collection protocols.
Lead establishing reliability and validity evidence.
Develop scoring procedures and normative frameworks.
Prepare technical documentation of the screener and contribute to deliverables for the BC Ministry of Education and Child Care.
Contribute to peer-reviewed publications and presentations.
Collaborate with the project team, the French language screener team, and external partners.
Maintain current knowledge of standards for educational and psychological testing.
Code statistical models using methods to ensure reproducibility.
Monitor data trends as ongoing data collection is occurring.
Data Management and Knowledge Translation · Ensures that data is accurate, consistent, properly maintained and shared in compliance with relevant privacy protection, confidentiality, and other ethical principles.
Prepares final data files.
Identifies and addresses data integrity/reliability issues and uses data cleaning processes to achieve required data quality standards.
Works with large data sets to perform data mining and statistical analyses.
Transforms, synthesizes, and cleanses data; identifies opportunities to reduce duplication and errors and ensure data consistency; identifies data integrity issues and proposes data cleansing processes to develop clear and consistent data quality standards.
Facilitate disseminating research and findings by making presentations and scientific meetings.
Contribute to scientific papers and write technical reports.
Assume responsibility for the results published in peer-reviewed journals, including the accuracy of published data and assurance of scientific integrity.
Write documentation reports to accompany the datasets.
Communication and Consultation · Develops accurate graphs, reports, dashboards, and presentations to convey complex information understandably and compellingly.
Potential to take an active role in writing scholarly papers for submission to refereed journals.