Pearl Talent - Assessment Psychometrist

Pearl

2d•Remote

About The Position

Pearl Talent is seeking a Psychometrician to be the scientific backbone of their AI voice-model initiative. This role focuses on defining measurement strategies, designing and validating instruments to infer psychological constructs from voice, and converting these measurements into reliable ground-truth labels and evaluation criteria for AI model training. The Psychometrician will be responsible for determining the scientific limits of voice-based psychological inference and ensuring the quality and defensibility of the measurement system. This is not a machine-learning engineering role, but rather a role focused on the foundational measurement science.

Requirements

Advanced degree (MSc or PhD preferred) in Psychometrics, Quantitative Psychology, Psychological Measurement, I/O Psychology, Behavioral Science, or a closely related quantitative field.
Hands-on instrument experience: designed, validated, and refined psychometric instruments.
Strong statistical foundation in factor analysis, reliability, validity, and measurement theory.
Comfort with messy, real-world data and ability to make evidence-based recommendations.
Cross-functional fluency: comfortable collaborating with AI/ML engineers, data engineers, and product teams.
Python or R experience is highly desirable.
Scientific backbone: ability to challenge unsupported assumptions, define responsible limits for AI-based psychological inference, and hold the line when necessary.
IRT experience is a strong plus.

Responsibilities

Review literature and evaluate established psychometric frameworks (e.g., Big Five/OCEAN, HEXACO, DISC) for validity, usefulness, and fit for the use case.
Recommend constructs and dimensions to include, exclude, or treat cautiously, explicitly identifying traits that cannot be responsibly inferred from voice.
Build and maintain a clear construct map connecting constructs, dimensions, indicators, survey items, and resulting scores.
Design psychometric surveys with appropriate items, response scales, scoring rules, and attention checks, minimizing fatigue and response bias.
Decide whether to use validated existing scales, adapt them, or develop new measures, and pilot/iterate based on empirical performance.
Run the full validation battery, including reliability (McDonald's omega, Cronbach's alpha, item-total analysis), EFA and CFA, construct/convergent/discriminant/criterion validity, test-retest, and IRT where useful.
Recommend sample sizes, pilot methodology, and evidence thresholds for measures.
Transform psychometric responses into defensible training labels (continuous scores, categories, normalization, confidence/reliability information) in partnership with AI and Data teams.
Define rules for missing, inconsistent, or low-quality responses and criteria for reliable training labels.
Design the framework for comparing survey-based ground truth against voice-model predictions and define evaluation metrics.
Analyze model performance, identifying areas of success and failure, and differences in prediction quality across constructs or populations.
Evaluate measurement invariance and potential bias across languages, cultures, and populations.
Define the scientific limits of voice-based psychological inference and partner with Product and AI to prevent unsupported interpretations.
Translate complex statistical findings into practical decisions for technical and non-technical stakeholders, ensuring alignment with current peer-reviewed research.