Senior Psychometrician

National Board for Certification in Occupational Therapy • Gaithersburg, MD

44d • Remote

About The Position

The Senior Psychometrician is a member of the Research & Psychometrics team responsible for providing psychometric oversight, analysis, and consultation in support of assessments that measure the competency of occupational therapy professionals. This role supports NBCOT’s certification programs by performing operational psychometric work, including practice analysis and standard setting, and performing applied research to provide actionable insights for improving assessments and processes. The Senior Psychometrician operates with minimal supervision, plans and prioritizes work independently, and contributes to complex projects and workflows, recognized as a subject matter expert. This role exercises independent judgment in applying policies and solving routine to moderately complex problems, contributes to department initiatives with external influence, and delivers outcomes that have a direct impact on the organization’s reputation and audiences.

Requirements

• Doctoral degree in a field directly related to measurement, assessment, psychometrics, statistics, educational measurement, quantitative psychology, industrial-organizational psychology or similar, along with 3+ years of experience in a psychometrician role, or equivalent is required.
• Strong knowledge of psychometric theory and fundamental concepts in measurement and assessment (e.g., validity, reliability, classical test theory, item response theory).
• Knowledge of research methods, including survey design, data collection, and interpretation of research findings.
• Applied experience conducting job/practice analyses, standard-setting, and statistical and psychometric analyses to support assessment development, scoring, and evaluation.
• Expert knowledge of statistical and productivity software, such as R, SPSS, Winsteps, and Microsoft Excel.
• Able to perform expert level tasks including advanced statistical and psychometric analysis, conducting research studies, and facilitating psychometric meetings.
• Communication
• Analytical and critical thinking
• Data analysis and interpretation
• Attention to detail
• Research and evidence-based decision making
• Psychometric judgment
• Problem solving
• Technical writing
• Adaptability and learning agility
• Autonomy and initiative
• Planning and organization
• Accountability
• Collaboration

Nice To Haves

• Experience supporting certification, licensure, credentialing assessment programs is preferred.
• Experience with practice analysis studies, standard-setting activities, or exam development processes is preferred.
• Knowledge of machine learning and natural language processing is an asset.

Responsibilities

• Designs and conducts practice analysis and competency modeling studies to identify and define the knowledge, skills, competencies, and other requirements necessary for effective occupational therapy practice.
• Develops and validates competency frameworks and related models that support certification and assessment programs.
• Develops examination specifications and content outlines based on empirical practice analysis findings.
• Establishes passing standards using appropriate psychometric methods to support valid score interpretations and defensible classification decisions.
• Performs advanced psychometric and statistical analyses, including Rasch calibrations, DIF analyses, equating, scaling, and other related methodologies, to support assessment quality and fairness.
• Develops and implements scoring processes and algorithms and designs and generates score reports that provide meaningful feedback to test-takers and other relevant parties.
• Designs, programs, and implements AI-driven tools and solutions to support psychometric analysis, exam development, and operational efficiency across NBCOT’s assessment programs.
• Researches, develops, and evaluates innovative item types and assessment methods to enhance the measurement of knowledge, skills, abilities, and competencies relevant to occupational therapy practice.
• Explores emerging technologies and advances in assessment science to identify opportunities to strengthen NBCOT’s assessment programs and measurement practices.
• Conducts psychometric research to improve assessment instruments and practices and to support evidence-based decision-making.
• Prepares technical documentation and reports by summarizing complex psychometric and statistical information and clearly communicates findings to technical and nontechnical audiences, orally, in writing, and through data visualization.
• Designs, programs, and implements AI-driven tools and solutions to support psychometric analysis, exam development, and operational efficiency supporting NBCOT’s assessment programs.
• Manages large datasets and databases to support psychometric analyses, reporting, and research while ensuring data integrity, documentation, and reproducibility.
• Provides psychometric consultation on the design, development, interpretation, and use of assessments that measure the competencies of occupational therapy professionals.
• Works collaboratively with colleagues and provides education, guidance, and consultation to internal teams on measurement-related principles and practices, to support valid and fair certification processes, tools, and decisions.
• Contributes expertise to professional organizations and participates in conferences, mentoring emerging professionals, and supporting initiatives that advance the field of psychometrics and certification.
• Performs other job-related responsibilities as needed to support NBCOT’s mission, vision, and values.

Benefits

• 35-hour work week
• over 30 paid days off each year
• paid medical insurance
• matched 401K plan and discretionary profit-sharing contributions
• annual bonus and salary adjustments
• covered life insurance and short-term and long-term disability
• voluntary benefits: vision, term life, critical illness, accident, and pet insurance

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Ph.D. or professional degree