Psychometrician

Universities of Wisconsin • Madison, WI

4d • $80,000 • Hybrid

About The Position

WIDA is an educational services organization that has been supporting, teaching, and assessing multilingual learners for over 20 years. We are dedicated to advancing language development and academic achievement for multilingual children in early childhood and grades K-12, providing educators globally with high-quality language standards, assessments, professional learning, and research. WIDA is proud to be part of UW-Madison's nationally ranked School of Education.

The Psychometrician position is within WIDA’s Assessment Department and reports to the Psychometrics Manager. This role is responsible for supporting the psychometric operational and research activities for WIDA’s language assessments, monitoring and evaluating operational psychometric projects and analyses conducted by vendors, and contributing to the authoring and review of research reports and psychometric deliverables. The position will be instrumental in supporting the psychometric operations and continuous improvement of the MODEL assessment program, including automated scoring initiatives. Additional duties include conducting and reviewing test security analyses for ACCESS for ELLs, supporting item bank maintenance and quality assurance, providing psychometric quality control for ACCESS and Alternate ACCESS assessment programs, and contributing to operational and research projects that ensure the validity, reliability, and fairness of WIDA assessments.

Requirements

• Minimum 1 year of experience in psychometrics theory, research, and statistics. Examples may include classical test theory, item response theory, scaling and equating, sampling, generalizability theory, linear modelling, or structural equation modeling.
• Experience with data manipulation and statistical/psychometrics analyses using Winsteps, SAS, R, or other related software packages.
• Familiarity with multistage adaptive testing, Rasch model, and graded response IRT models.
• Ability to write technical and research reports.
• Ability to work independently and collaboratively within a team.
• Experience collaborating with external vendors or partners.

Nice To Haves

• Programming skills, including proficiency in Python or similar programming languages used for psychometric modeling, statistical analysis, and assessment research.
• Experience applying artificial intelligence (AI) methods to assessment and measurement, such as natural language processing (NLP), automated scoring, machine learning, or related technologies.
• Test security related analysis, training or experience
• Experience working with K–12 educational assessments, language proficiency assessments, or large-scale standardized testing programs

Responsibilities

• Analyzes, researches, and proposes solutions to current and future psychometric data and problems and communicates solutions with internal and/or external stakeholders to support the development of educational resources
• Designs and implements psychometric tools to assess learner ability and/or enhance learning outcomes in collaboration with internal and external stakeholders and staff
• Serves as a subject matter expert and disseminates technical psychometric concepts across various mediums, including non-technical audiences
• Observes and evaluates learner performance and makes necessary changes in procedures
• Proctors psychological tests

Benefits

• generous vacation
• holidays and paid time off
• competitive insurance and saving accounts
• retirement benefits
• competitively priced health/dental/vision/life insurance
• tax-advantaged savings accounts
• participation in the nationally recognized Wisconsin Retirement System (WRS) pension fund

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: Ph.D. or professional degree