Senior UX Language Researcher - Variable Staffing Pool

Humana

1d•$104,000 - $143,000•Remote

About The Position

Humana's Experience Design team, within the Consumer Digital organization, strives to eliminate friction in today's healthcare ecosystem so consumers feel confident, respected, and heard in the decisions affecting their health. The Senior UX Language Researcher (VSP/PT) must have a passion for data, efficiency, accuracy, and should be capable of analyzing conversational transcripts for industry standard metrics like accuracy, fallback rate, hallucination rate etc. Familiarity with LLMs; how they work, how to prompt them effectively, and how to evaluate the quality of their outputs. Familiarity analyzing conversational chat and call transcripts with tools like Python, R, SQL. Experience managing transcription and annotation projects including for LLM-based metrics. Deep knowledge of language data labeling best practices, including transcription and annotation for conversational data. Providing data quality expertise cross-functionally and advocating for best practices. Delivering high-quality work in a fast-paced environment. Working autonomously with minimal direction. Contributing to process improvements to reduce time and improve output. Improving software tools by identifying bugs and suggesting enhancements. Diving deep into issues and implementing solutions independently. Proactively addressing issues and problems. Keeping up with changing project conventions and priorities. Additional Responsibilities include: Label, evaluate, and ensure the quality of datasets. Work closely with UX, Product and Engineering to understand data ambiguities and resolve issues in conversational data. Conduct in-depth qualitative error trend analysis, developing action plans to enhance data quality. Conduct in depth quantitative data analyses to develop new metrics for tracking conversational experience performance and drive enhancements to the user experience.

Requirements

5+ years of related experience
2+ years of computational linguistics, language data processing, semantics, or syntax experience
Bachelor's degree or equivalent
Experience in natural language data labeling, data annotation, linguistic annotation or other forms of data markup, as well as working cross-functionally with different stakeholders
Experience using Python, R, SQL for language data analysis
Experience using Microsoft Excel for data analysis, formulae, and data visualization
Familiarity with LLMs; how they work, how to prompt them effectively, and how to evaluate the quality of their outputs
Familiarity analyzing conversational chat and call transcripts with tools like Python, R, SQL
Deep knowledge of language data labeling best practices, including transcription and annotation for conversational data

Nice To Haves

Strong analytical skills, attention to detail, and effective communication abilities
Interest in pragmatics and conversational design
Ability to navigate a Unix terminal and use common command line tools

Responsibilities

Analyzing conversational transcripts for industry standard metrics like accuracy, fallback rate, hallucination rate etc.
Managing transcription and annotation projects including for LLM-based metrics
Providing data quality expertise cross-functionally and advocating for best practices
Delivering high-quality work in a fast-paced environment
Working autonomously with minimal direction
Contributing to process improvements to reduce time and improve output
Improving software tools by identifying bugs and suggesting enhancements
Diving deep into issues and implementing solutions independently
Proactively addressing issues and problems
Keeping up with changing project conventions and priorities
Label, evaluate, and ensure the quality of datasets.
Work closely with UX, Product and Engineering to understand data ambiguities and resolve issues in conversational data.
Conduct in-depth qualitative error trend analysis, developing action plans to enhance data quality.
Conduct in depth quantitative data analyses to develop new metrics for tracking conversational experience performance and drive enhancements to the user experience