AI Quality Analyst

Innodata Inc.•Oklahoma City, OK

8d•Remote

About The Position

Innodata is a global data engineering company focused on enabling the responsible advancement of artificial intelligence. They provide data, evaluation frameworks, and human expertise for building trustworthy AI systems. The company offers solutions, platforms, and services for Generative AI/AI builders and adopters, leveraging over 36 years of experience in delivering high-quality data and outcomes. This role is part of a global contributor community within the Subject Matter Expert (SME) on Demand program, partnering with leading technology companies to build the future of generative AI and large language models (LLMs). It is a part-time, remote, flexible, project-specific opportunity for individuals who want to make an impact on their own schedule. The role involves helping LLMs understand language and reasoning, contributing to cutting-edge AI development.

Requirements

A Bachelor’s degree or higher in a humanities specialization is required.
Professional or Expert level proficiency (C1/C2) in English

Nice To Haves

Advanced degrees are strongly preferred (Master’s or PhD)

Responsibilities

Rating/assessing the performance of AI models or algorithms based on their output or behavior through a set of evaluative questions.
Labeling elements of a piece of content rather than the content as a whole.
Assigning predefined categories or labels to items.
Evaluating the perceived quality and/or appropriateness of content.
Generating labels to advance understanding of a concept, trend etc.
Creation of additional training data for machine learning models by applying transformations to the original data, such as modifying images (rotation, flipping, cropping), generating new text (paraphrasing, summarization), or altering audio/video signals (speed modification, pitch shifting) to reduce overfitting and increase dataset diversity.
Reviewing data and identifying whether or not a product feature works as intended based on the project's guidelines.
Labeling model outputs to identify if a piece of content is or isn't something. Examples: identify clickbait; identifying gaming videos; identifying branded content.
Ordering or ranking items based on a set of preferences or criteria.
Creating prompts or questions that will be used to generate responses from a language model or other AI system.
Projects that evaluate the relevance of content based on a relevancy scale (1-3, 1-5, etc.).
Generating responses to prompts or questions using a language model or other AI system.
Rewriting existing text while preserving the original meaning, often to improve clarity or style and adherence to guidelines.
Producing concise summaries of longer pieces of text or data.
Converting spoken language or audio content into written text.
Converting text or spoken language from one language to another.
Gathering and compiling various forms of data to be used for training, evaluating, or fine-tuning the AI models. This may include text, images, videos, audio files, or other types of digital content.