Senior Psychometrician

NBME•Philadelphia, PA

6d•Remote

About The Position

NBME®'s Psychometrics and Data Analysis (PADA) unit is seeking a Senior Psychometrician to join their team. This position is expected to make contributions in six primary areas: Applied research and development in support of modifications to examination designs, and analysis methods and procedures. Planning and execution of psychometric analyses, scoring, and score reporting for operational examination programs. Contribute to the educational measurement community via professional presentations and publications related to independent and organizational research agendas. Evaluate existing procedures for improvement opportunities, identify new procedures to increase accuracy and efficiency of exam processing. Designing and evaluating data visualization and graphical representation of data for stakeholder feedback, examinee feedback reports, and interactive data tools. Supporting AI-related research initiatives. Senior Psychometrician must effectively collaborate with a variety of internal staff including data analysts, test developers, information technologists, and program managers. In addition, external facing communication and engagement with clients and subject matter experts from a wide range of medical and health care fields is also necessary. Operational examination programs at NBME include the United States Medical Licensing Examination®, an extensive medical school examination program that supports the assessment needs of US and international medical schools, and numerous client examinations primarily in support of medical specialties. The variety of assessment programs creates a wealth of data and opportunities for applied and theoretical research that can impact practice. This role has been designated as primarily remote, meaning it requires little to no onsite attendance. While this role can be designated as remote, you are offered the flexibility to select if you prefer to work primarily remotely, hybrid, or onsite. We're open to considering candidates within 50 miles of our office location in Philadelphia, PA. Please note that onsite interviews and onboarding at our Philadelphia office may be required for this role. If applicable, advance notice will be provided to support planning. At NBME, we continue to innovate and improve how we fulfill the evolving needs of the health care community. This commitment starts and ends with the people at NBME. By recruiting and empowering talented individuals from various disciplines and backgrounds, which includes professionals with diverse life experiences, abilities, and perspectives, NBME can take a well-informed, robust approach to advancing medical education and assessment for years to come.

Requirements

Strong background in psychometric theory and application, including knowledge of IRT, CTT, statistics, test development procedures, and research design.
Advanced knowledge of applied measurement topics, such as equating, scaling, generalizability theory, Bayesian methodology, and/or standard setting.
Demonstrated strength in presenting assessment data graphically to facilitate user comprehension, support correct inferences, and guide subsequent actions.
Knowledge and/or experience in natural language processing, process data analytic methods, formative assessment approaches is desirable.
Excellent computer skills and a working knowledge of statistical software (e.g., Calibration software, SAS, Python, R).
Excellent oral and written communication skills.
Ability to identify errors and solve problems.
Excellent interpersonal skills, including the ability to work closely and successfully with individuals from a variety of professional and educational backgrounds.
A minimum of 5 years of experience in a Psychometrician/Measurement Scientist role is required for consideration.
Doctorate degree in Educational/Psychometric Measurement, Statistics, or a related field is required.

Nice To Haves

Knowledge and/or experience in natural language processing, process data analytic methods, formative assessment approaches is desirable.

Responsibilities

Applied research and development in support of modifications to examination designs, and analysis methods and procedures.
Planning and execution of psychometric analyses, scoring, and score reporting for operational examination programs.
Contribute to the educational measurement community via professional presentations and publications related to independent and organizational research agendas.
Evaluate existing procedures for improvement opportunities, identify new procedures to increase accuracy and efficiency of exam processing.
Designing and evaluating data visualization and graphical representation of data for stakeholder feedback, examinee feedback reports, and interactive data tools.
Supporting AI-related research initiatives.
Effectively collaborate with a variety of internal staff including data analysts, test developers, information technologists, and program managers.
External facing communication and engagement with clients and subject matter experts from a wide range of medical and health care fields is also necessary.