Psychometrician

University of Wisconsin Madison • Madison, WI

3d • $80,000 • Hybrid

About The Position

WIDA is an educational services organization that advances language development and academic achievement for multilingual children in early childhood and grades K-12. WIDA provides educators worldwide with high-quality language standards, assessments, professional learning, and research, and is part of UW-Madison's School of Education. The Psychometrician position is in WIDA’s Assessment Department and reports to the Psychometrics Manager. This role supports psychometric operational and research activities for WIDA’s language assessments, monitors and evaluates operational psychometric projects and analyses conducted by vendors, and authors and reviews research reports and psychometric deliverables. The position is key to supporting the psychometric operations and continuous improvement of the MODEL assessment program, including automated scoring initiatives. Additional responsibilities include conducting and reviewing test security analyses for ACCESS for ELLs, supporting item bank maintenance and quality assurance, providing psychometric quality control for ACCESS and Alternate ACCESS assessment programs, and contributing to operational and research projects that support the validity, reliability, and fairness of WIDA assessments. Employees in this position can expect benefits such as generous vacation, holidays and paid time off; competitive insurance and saving accounts; and retirement benefits. Remote work is an option on a voluntary basis, but the role is headquartered in Madison, WI, and requires occasional work-related travel. If working remotely out-of-state, travel back to UW-Madison would not be reimbursable, and other business travel will require cost comparisons between your home and Madison, WI. WIDA operates in a Windows-based environment and a Dell PC laptop will be issued for use in this role. Software and virtual connection tools critical to WIDA's work dictate that staff work on a PC.

Requirements

• Minimum 1 year of experience in psychometrics theory, research, and statistics. Examples may include classical test theory, item response theory, scaling and equating, sampling, generalizability theory, linear modelling, or structural equation modeling.
• Experience with data manipulation and statistical/psychometrics analyses using Winsteps, SAS, R, or other related software packages.
• Familiarity with multistage adaptive testing, Rasch model, and graded response IRT models.
• Ability to write technical and research reports.
• Ability to work independently and collaboratively within a team.
• Experience collaborating with external vendors or partners.

Nice To Haves

• Programming skills, including proficiency in Python or similar programming languages used for psychometric modeling, statistical analysis, and assessment research.
• Experience applying artificial intelligence (AI) methods to assessment and measurement, such as natural language processing (NLP), automated scoring, machine learning, or related technologies.
• Test security related analysis, training or experience
• Experience working with K–12 educational assessments, language proficiency assessments, or large-scale standardized testing programs

Responsibilities

• Analyzes, researches, and proposes solutions to current and future psychometric data and problems and communicates solutions with internal and/or external stakeholders to support the development of educational resources
• Designs and implements psychometric tools to assess learner ability and/or enhance learning outcomes in collaboration with internal and external stakeholders and staff
• Serves as a subject matter expert and disseminates technical psychometric concepts across various mediums, including non-technical audiences
• Observes and evaluates learner performance and makes necessary changes in procedures
• Proctors psychological tests
• Supports the psychometric operational and research activities associated with WIDA’s language assessments
• Supports the monitoring and evaluating of operational psychometric projects and analyses conducted by vendors
• Supports the authoring and review of research reports and psychometric deliverables
• Plays a key role in supporting the psychometric operations and continuous improvement of the MODEL assessment program, including automated scoring initiatives
• Conducts and reviews test security analyses for ACCESS for ELLs
• Supports item bank maintenance and quality assurance activities
• Provides psychometric quality control for ACCESS and Alternate ACCESS assessment programs
• Contributes to operational and research projects that support the validity, reliability, and fairness of WIDA assessments

Benefits

• Generous vacation, holidays and paid time off
• Competitive insurance and saving accounts
• Retirement benefits
• Competitively priced health/dental/vision/life insurance
• Tax-advantaged savings accounts
• Participation in the nationally recognized Wisconsin Retirement System (WRS) pension fund

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Ph.D. or professional degree