Verification & Validation Data Scientist

SHRM•Alexandria, VA

6d•Hybrid

About The Position

This role exists to answer one question independently: do the synthetic-audience and automated systems actually perform as claimed? It is a verification-and-validation function drawn from data science, closer to model validation, ML evaluation, and the independent V&V discipline used in regulated and safety-critical software than to market research. The V&V Data Scientist will design and run the evaluation that benchmarks system output against ground-truth data, owns the accuracy metrics and error bounds, and certifies fitness for use, working independently of the engineer who builds the models. The separation of duties is the point: the builder does not grade their own work.

Requirements

Bachelor’s degree in computer science, mathematics, or engineering or relevant equivalent experience in lieu of degree.
Seven (7) or more years in data science, model validation, ML evaluation, or an independent V&V / QA function.
Demonstrated experience independently validating models or systems against ground-truth data, evaluation design, benchmarking, and error analysis.
Strong applied statistics and metrics design; fluency in accuracy, bias, calibration, and uncertainty quantification.
Working understanding of LLMs and synthetic-respondent systems and their failure modes — enough to test them rigorously.
Independence and rigor: the temperament to challenge results and withhold sign-off, including from colleagues’ work.
Clear communication: able to tell a non-technical executive, with evidence, why a result can or cannot be trusted.
Ability to effectively leverage artificial intelligence (AI) tools and technologies to streamline workflows, enhance productivity, and improve overall work quality.
Prolonged periods of sitting at a desk and working on a computer.
Frequent use of hands and fingers for typing, handling documents, and using office equipment.
Occasional standing, walking, bending, and reaching.
Ability to lift and carry up to 30 pounds as needed.
Clear verbal and written communication skills for effective interaction with colleagues and stakeholders.

Nice To Haves

Master’s degree preferred.
Familiarity with survey/behavioral data as a validation target preferred.

Responsibilities

Design and run the independent validation methodology: benchmark synthetic-audience and automated output against real panel holdout and other ground-truth data.
Own the evaluation metrics, test harness design, and reported accuracy bounds; quantify where each system is and is not reliable.
Certify fitness for use, translate results into clear “validated for this use / not validated for that use” determinations.
Operate independently of the AI/Synthetic Engineer: verify the models and pipelines AI/Synthetic Engineer builds without having built them.
Monitor for model and data drift over time; re-validate as models, prompts, and source data change.
Maintain the audit trail and evidence base that supports the function’s credibility with stakeholders and any external auditor.