Psychometrician

National Registry of EMTs•Columbus, OH

4d•Onsite

About The Position

The Psychometrician is responsible for performing psychometric activities for examinations and assisting with improving psychometric methods and relevant issues in the evaluation of pre-hospital knowledge, skills, and abilities. The Psychometrician leads assigned projects and contributes to the dissemination of psychometric research via publications or conference presentations. The Psychometrician plays a critical role in advancing the organization’s mission by ensuring examinations are fair, defensible, and aligned with current EMS practice. Through data analysis, test development, and research, the role supports lifelong professional competence, strengthens partnerships, and promotes evidence-based decisions that enhance the credibility of certification outcomes.

Requirements

PhD in Education, Psychology, Measurement, or related field required.
Experience with Standards for Education and Psychological testing, and skill at translating those standards into practice.
Experience in software required for psychometric analyses, such as SAS, WINSTEPS, FACETS, R is required.
Knowledgeable of testing lifecycle: development, publication, administration, analysis, and reporting.
Working knowledge of measurement models, including Rasch.
Strong problem-solving and decision-making skills.
Effective communication skills
Ability to clearly document complex policies, procedures, work instructions and reports for internal and external audiences.

Nice To Haves

3 years of experience in related field is preferred.

Responsibilities

Ensures examination programs comply with the Standards for Educational and Psychological Testing and other applicable accreditation standards, balancing compliance, resources, and stakeholder impact.
Conducts and delivers accurate psychometric analyses and reports (e.g., item analysis, form analysis, equating, DIF, standard setting) in accordance with organizational policies, processes, and procedures.
Creates technical reports, supports examination rescoring inquiries, prepares leadership-request analyses, and coordinates with other psychometricians.
Produces analytical reports, job aids, and guidance to support examination development and improve efforts.
Collaborates with subject matter experts and examination developers on item development, examination design, and review activities, including participation in Practice (Job Task) Analysis projects and development of examination specifications.
Conducts and collaborates on research, development, and operational projects; communicates findings to stakeholders.
Supports project planning by identifying priorities, resource needs, timelines, and risks; advises on research, measurement, and statistical issues.
Contributes to internal initiatives that improve psychometric policies, processes, procedures, quality, and efficiency.
Serves as an organizational resource on psychometric and measurement topics, including responding to complex inquiries and delivering training.
Prepares materials for board and committee meetings and provides support to the Content Development and Examination teams.