Evaluation Quality Data Analyst
About The Position
This role supports the Globalization Evaluation Program by executing data analysis and quality measurement tasks within defined frameworks and methodologies. The role produces structured findings on evaluation trends, evaluator and vendor performance, dataset integrity, and annotation quality — translating evaluation data into clear, documented outputs that inform operational decisions. Strong analytical capabilities, proficiency with data tools, SQL or Python, and dashboard development are essential, as is the ability to work systematically within established standards and escalate findings clearly and promptly.
Requirements
- Proficiency in executing statistical analyses and inter-rater reliability metrics within defined frameworks, translating outputs into clear, structured findings
- Ability to operate within defined evaluation frameworks, QA methodologies, and quality metrics
- Experience with golden dataset curation, validation, and data quality control
- Experience working with annotation or human evaluation data, including annotator feedback analysis and vendor quality assessment
- Strong analytical thinking with the ability to identify patterns, inconsistencies, and root causes across large datasets
- Demonstrated expertise in data visualization and dashboard development using Tableau, Power BI, or similar tools
- Experience querying and analyzing large datasets using SQL or Python
- Experience working cross-functionally, with clear communication and escalation abilities
Responsibilities
- Proficiency in executing statistical analyses and inter-rater reliability metrics within defined frameworks, translating outputs into clear, structured findings
- Ability to operate within defined evaluation frameworks, QA methodologies, and quality metrics
- Experience with golden dataset curation, validation, and data quality control
- Experience working with annotation or human evaluation data, including annotator feedback analysis and vendor quality assessment
- Strong analytical thinking with the ability to identify patterns, inconsistencies, and root causes across large datasets
- Demonstrated expertise in data visualization and dashboard development using Tableau, Power BI, or similar tools
- Experience querying and analyzing large datasets using SQL or Python
- Experience working cross-functionally, with clear communication and escalation abilities
Benefits
- Medical, dental & vision
- Critical Illness, Accident, and Hospital
- 401(k) Retirement Plan – Pre-tax and Roth post-tax contributions available
- Life Insurance (Voluntary Life & AD&D for the employee and dependents)
- Short and long-term disability
- Health Spending Account (HSA)
- Transportation benefits
- Employee Assistance Program
- Time Off/Leave (PTO, Vacation or Sick Leave)
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Education Level
No Education Listed
