Evaluator

University of Colorado•Hybrid, CO

21h•$59,417 - $75,579•Hybrid

About The Position

The Evaluation Team within the School of Medicine’s Office of Assessment, Evaluation, and Outcomes (AEO) is responsible for the design, implementation, analysis, and reporting of all course, faculty, and program evaluation for Undergraduate Medical Education (UME). The Evaluator applies both project management and data-focused processes to evaluation activities, including longitudinal reporting and continuous quality improvement. This position assists with the design and distribution of survey instruments, data collection and analysis, and results reporting to support curricular improvement, LCME accreditation, faculty promotion, and faculty research, all while maintaining the confidentiality of sensitive data. This position serves and communicates with a diverse group of faculty, staff, and students, with different agendas and objectives, to ensure the successful evaluation of all UME courses, faculty, and programs.

Requirements

Bachelor’s degree in education, psychology, business, public administration, health care, educational technology, social science, liberal arts, communications, accounting, finance, library science, or a directly related field from an accredited institution and 2 years of professional experience serving as a project/change manager, evaluator, analyst, or in a directly related role.
A combination of education and related technical/military/paraprofessional experience may be substituted for a bachelor’s degree on a year for year basis.
Applicants must meet minimum qualifications at the time of hire.
Ability to communicate effectively, both in writing and orally.
Ability to establish and maintain effective working relationships with employees at all levels throughout the institution.
Outstanding customer service skills.
Outstanding oral and written communication skills
Outstanding organizational skills
Ability to prioritize tasks and work on multiple projects concurrently
Attention to detail and the ability to ensure accuracy in work
Ability to work with minimal supervision and show initiative
Ability to multi-task and work cooperatively with others
Applicants must be legally authorized to work in the United States without requiring sponsorship. We are unable to provide work visa sponsorship or employment authorization for this position now or in the future.

Nice To Haves

Master’s degree in education or related field,
Experience working with UME curriculum, faculty, and structure.
Experience working with data management programs/applications.
Experience working in an academic medical school environment.

Responsibilities

Communicates with faculty, staff, and IT professionals to answer questions about the evaluation process and resolve issues that arise during data collection.
Designs and implements surveys to adhere to best practices for data collection including random sampling to reduce survey burden.
Distributes evaluations to students and tracks progress of submissions to ensure sample sufficiency and representativeness.
Implements timelines for releasing, collecting, and managing course and faculty evaluations.
Reviews all open-ended survey feedback and determines whether any comments indicate potential issues to escalate to the AEO Manager.
Analyzes, summarizes, and reports evaluation data to match the data type and stakeholder needs, ensuring accurate results and the confidentiality of survey respondents.
Designs, builds, and maintains Tableau Dashboards to report evaluation results for course and program leadership, both longitudinally within and across courses.
Create and distribute summary reports for program and course directors, distribute individual reports to faculty, residents, and other educators, and create individual reports for upload to PRiSM accounts of faculty educators.
Collects, interprets, and synthesizes course and curricular content area feedback from various sources (e.g., students, staff, faculty, and historical documents) to create thematic analyses used to prompt and facilitate conversation during continuous quality improvement meetings for LCME accreditation.
Conducts thematic analyses of open-ended feedback from course evaluations.
Facilitates and summarizes results from focus groups.
Responds to requests for ad-hoc reports, assessing the feasibility from both logistical and student confidentiality standpoints, and providing alternate reporting possibilities when necessary.
Uses Microsoft SQL to generate reports based on stakeholder needs.