Senior Director, Assessment

Physician Assistant Education Association • US,

4d • $147,000 - $158,000 • Remote

About The Position

The Senior Director, Assessment provides strategic and operational leadership for PAEA’s assessment portfolio. The role translates organizational goals into assessment strategy and operational plans and ensures assessment programs and services are developed, delivered, evaluated, and continuously improved. Working closely with the Chief Data Officer and Principal Psychometrician, the Senior Director oversees assessment operations, psychometric and measurement activities, exam development, innovation initiatives, and external partnerships and vendors. The role ensures that assessments are valid, reliable, fair, defensible, and aligned with organizational strategy, supports future assessment products and services, and represents PAEA in assessment, measurement, and health professions education communities.

The position leads, develops, and grows a high-performing assessment team, fostering collaboration and continuous improvement. It oversees the full assessment lifecycle, measurement research, exam operations, security and compliance, and external partnerships. The Senior Director also strengthens members’ assessment literacy through educational programming, consultation, score interpretation resources, and dissemination of best practices.

This is a remote position with no in-office presence required. Periodic travel is required for all-staff meetings, PAEA events, professional development opportunities, and other necessary meetings. The position requires authorization to work in the United States and involves access to sensitive and confidential information, which must be protected in accordance with PAEA policies.

Requirements

• Doctoral degree (PhD, EdD, or equivalent) in Psychometrics, Educational Measurement, Quantitative Psychology, Educational Psychology, Statistics, Research Methodology, or a closely related field.
• At least 8 years of progressively responsible experience in assessment, psychometrics, educational measurement, testing, or related fields.
• At least 3 years of supervisory or leadership experience managing professional staff and complex projects.
• Deep knowledge of Classical Test Theory, Item Response Theory, Rasch measurement, standard setting, test development, validation, scoring, equating, and assessment reporting.
• Demonstrated experience leading assessment programs, initiatives, or portfolios.
• Experience managing external vendors, consultants, and stakeholder relationships.
• Strong strategic planning, organizational leadership, and project management skills.
• Excellent written and verbal communication skills, including the ability to explain technical concepts to non-technical audiences.
• Ability to manage multiple priorities and lead complex cross-functional initiatives.
• Strong analytical, problem-solving, and decision-making skills.
• Demonstrated commitment to assessment quality, fairness, accessibility, and continuous improvement.
• Must be authorized to work in the United States.
• Must be able to perform essential duties satisfactorily, with reasonable accommodation.

Nice To Haves

• Experience in health professions education and assessment.
• Experience developing innovative assessment products, technologies, or services.
• Record of professional presentations, publications, or contributions to the assessment and measurement field.
• Experience managing assessment security programs and policies.
• Experience with R, data visualization tools, assessment platforms, and modern analytical technologies.
• Active involvement in professional measurement or testing organizations.

Responsibilities

• Lead the strategy, planning, implementation, evaluation, and continuous improvement of current and future assessment products and services, aligning priorities with organizational goals and member needs.
• Monitor developments in educational measurement, psychometrics, assessment technology, artificial intelligence, and health professions education to identify opportunities to improve assessment design, delivery, reporting, and product development.
• Collaborate with organizational leaders on assessment initiatives, business planning, and product development opportunities.
• Oversee the assessment lifecycle, including blueprint development and evaluation, item banking, test construction, administration and delivery, scoring, scaling, equating, standard setting, reporting, quality assurance, and technical documentation.
• Ensure assessments are valid, reliable, fair, accessible, secure, legally defensible, and compliant with testing standards, industry best practices, organizational policies, and governance requirements; establish and maintain related policies and procedures.
• Lead psychometric analyses and measurement research, including item and test performance monitoring, scoring methodology, standard setting, applied research, corrective action, and continuous evaluation.
• Guide the development and use of data tools, analytical solutions, and measurement technologies, and communicate technical findings, score interpretation guidance, and assessment information to technical and nontechnical audiences.
• Supervise, coach, and mentor Assessment staff; manage performance; and foster alignment across psychometric, operational, and strategic functions.
• Support staff development and participation in professional organizations, conferences, publications, and research; strengthen organizational capacity through cross-training, knowledge sharing, and succession planning.
• Oversee exam development activities, production schedules, budgets, resource allocation, volunteer engagement, and vendor performance.
• Coordinate the work of assessment staff, subject matter experts, vendors, and external partners, and cultivate relationships that advance PAEA’s assessment strategy.
• Implement process improvements that enhance quality, efficiency, scalability, sustainability, and stakeholder experience; support assessment-related meetings, summits, workshops, and professional development activities.
• Represent PAEA through presentations, publications, workshops, consultation, and professional service within testing, psychometrics, educational measurement, credentialing, and health professions education communities.
• Strengthen members’ assessment literacy through educational programming, consultation, score interpretation resources, and dissemination of best practices.
• Protect the confidentiality, integrity, and security of assessment materials, data, and processes, and maintain related policies, procedures, and risk controls.
• Coordinate incident response, risk mitigation, and compliance activities with organizational leaders, legal counsel, and other relevant stakeholders.
• Perform other duties and responsibilities as assigned; duties and activities may change with or without notice.

Benefits

• Competitive salary and benefits that consider external market compensation and maintain internal equity.
• Health, life, and disability insurances.
• Paid time off and holidays.
• 403(b) retirement plan with an 8% employer contribution.
• Monthly wellness stipend.
• Professional development funding.
• Tuition assistance.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Ph.D. or professional degree