Qualitative Coding Assistant

Northeastern University • Boston, MA

7d

About The Position

You will serve as the ground truth for AI validation by manually coding student simulation transcripts using detailed rubrics. This role requires meticulous attention to detail, strong qualitative analysis skills, and the ability to apply complex coding schemes consistently. Your ratings establish the validity benchmark against which AI scoring will be evaluated.

Requirements

• Master’s degree or Doctor of Education (EdD) in Education, Economics, Public Policy, Psychology, Sociology, Communication, or related social science
• Demonstrated experience with qualitative coding using established frameworks such as thematic analysis, content analysis, or discourse analysis
• Familiarity with inter-rater reliability concepts and procedures
• Strong writing skills for documentation and analytical memos
• Ability to work independently while maintaining consistency across coding sessions
• Comfort with ambiguity during iterative rubric refinement

Nice To Haves

• Experience with education assessment or program evaluation
• Background in adolescent development or youth workforce development
• Training in qualitative analysis software such as MAXQDA, NVivo, or ATLAS.ti
• Understanding of psychometric concepts, including validity, reliability, and construct definition

Responsibilities

• Study and internalize 18-indicator rubrics across 3 skill domains
• Code 50–80 student simulation transcripts using detailed rubric criteria
• Participate in calibration sessions to establish inter-rater reliability
• Document edge cases and ambiguous scenarios requiring rubric clarification
• Provide systematic feedback on rubric usability and clarity
• Identify indicators with low inter-rater agreement requiring revision
• Calculate inter-rater reliability metrics, including Cohen's kappa and intraclass correlation coefficient (ICC)
• Compare human versus AI ratings to identify systematic differences
• Contribute to validation methodology documentation
• Support qualitative findings write-up for research reports

Benefits

• medical
• vision
• dental
• paid time off
• tuition assistance
• wellness & life
• retirement
• commuting & transportation

What This Job Offers

• Job Type: Part-time
• Career Level: Mid Level