Remote | Russian Audio AI Evaluator — Up to $45/hour

24-Mag • New York, NY

1d • $45 • Remote

About The Position

We are sharing a specialised part-time consulting opportunity for Russian-language professionals with strong transcription, editing, analytical, and critical-listening skills to support advanced audio AI research. This role focuses on transcription, annotation, and evaluation of Russian-language audio and video content for AI training and benchmarking. Selected professionals will assess linguistic accuracy, develop evaluation standards, review model-generated outputs, and help improve the quality of Russian audio AI systems.

Requirements

• Native or near-native fluency in Russian, both spoken and written
• Professional fluency in English
• Strong familiarity with contemporary spoken Russian, regional vocabulary, dialects, and informal language
• Ability to accurately transcribe and analyse Russian-language audio
• Strong writing, editing, analytical, and critical-thinking skills
• Excellent attention to linguistic detail and ability to follow detailed guidelines
• Must be based in the San Francisco Bay Area
• Available approximately 10–20 hours per week

Nice To Haves

• Prior experience with transcription, annotation, localisation, language evaluation, or research workflows is advantageous
• Background in linguistics, humanities, social sciences, journalism, translation/localisation, or a technical discipline is welcomed
• Familiarity with AI, language models, or applied research environments is advantageous

Responsibilities

• Listen to, analyse, and transcribe Russian-language audio and video content
• Produce accurate written outputs in Russian, with supporting work in English when required
• Follow detailed transcription, formatting, and annotation guidelines
• Capture tone, intent, register, regional expressions, dialectal variation, and contemporary language usage
• Maintain high accuracy and consistency across structured audio datasets
• Evaluate Russian-language content for linguistic accuracy, fluency, clarity, and naturalness
• Identify grammatical issues, colloquialisms, regional variations, and linguistic edge cases
• Assess spoken Russian across general consumer audio contexts
• Distinguish between formal and informal registers and evaluate context-appropriate language use
• Provide structured feedback on language and transcription quality
• Develop clear evaluation criteria for high-quality Russian-language audio responses
• Create and refine grading rubrics in Russian and English
• Document evaluation standards to support consistency across reviewers
• Identify challenging linguistic cases requiring additional guidance
• Contribute to the refinement of annotation and evaluation frameworks
• Test language-model outputs across Russian audio tasks
• Evaluate responses for accuracy, completeness, fluency, and instruction adherence
• Grade model-generated outputs against defined evaluation criteria
• Participate in quality-assurance and review cycles
• Identify recurring model errors and provide feedback to improve performance
• Collaborate with project teams to resolve ambiguities and improve task design

Benefits

• Part-time independent contractor engagement
• Short-term, structured project
• Fully remote, with occasional in-person collaboration in the San Francisco Bay Area if required
• Compensation: Up to $45/hour
• Approximately 10–20 hours per week
• Flexible scheduling based on project requirements
• Projects may be extended, shortened, or concluded based on project needs and performance
• Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party
• H1-B and STEM OPT support is unavailable for this engagement

What This Job Offers

• Job Type: Part-time
• Career Level: Mid Level
• Education Level: No Education Listed