AI Operations, Applied AI Research and Design Developer

Apple•Cupertino, CA

About The Position

The AI Operations, Applied AI Research and Design Developer serves as a technical engine for advancing the organization’s AI ambitions, leading the technical design and validation of human-centered AI solutions for content across Movies, TV, Live Sports, and Apple Originals. The role owns the technical validation lifecycle, from partnering with product management to assess feasibility through building rapid proofs of concept (POCs) and making evidence-based go/no-go recommendations. The position collaborates with the Operations Product Manager to shape proposed solutions while adhering to responsible AI principles. It curates golden datasets and evaluations for training, testing, and benchmarking; leads POC testing and performance measurement against baselines; and provides engineering teams with clear findings, demonstrations, and practical handoff guidance. The role also validates production performance against POC results and, where applicable, human-level parity. The developer researches emerging AI capabilities and turns findings into recommendations for product strategy and tooling. The work includes building POCs and automated workflows using Apple-approved LLMs and multimodal models, and designing prompts, structured outputs, retrieval, and failure handling. The role works with domain experts to create ground-truth datasets, select business-relevant metrics, and run controlled experiments across models, prompts, preprocessing, and scoring. Results are communicated through technical write-ups, demos, and engineering handoff guidance that clearly explain limitations and failure modes.

Requirements

Bachelor’s or master’s degree in Computer Science, AI/ML, Data Science, or a related field, completed or expected by the start date.
Strong Python skills, including data manipulation and analysis with pandas and SQL, REST APIs, and structured data formats such as JSON and XML.
Hands-on experience building AI applications on LLM APIs, including prompting, structured outputs, RAG, or tool-using workflows, through coursework, research, internships, or projects.
Judgment about when to use an LLM, a classical ML model, or a deterministic approach.
Working knowledge of ML evaluation, including precision, recall, F1, test-set construction, experiment design, and validation against baselines and benchmarks.
Understanding of AI guardrails, including safety, privacy, compliance, and responsible deployment.
Proficiency with LLM-assisted coding tools for designing, implementing, and iterating on software.
Clear written and verbal communication skills, including technical documentation and explaining complex results to non-technical stakeholders.

Nice To Haves

Internship or research experience building and benchmarking an AI system end to end, from data and model development through evaluation and recommendation.
Ability to work independently and quickly deliver working prototypes as requirements change, demonstrated through a thesis, publications, or public projects.
Experience translating business requirements into implementable solutions and writing technical specifications and engineering handoff materials.
Experience with multimodal models, video or audio processing, LLM-as-judge evaluation, or open-weight model serving.
Experience with agent frameworks, observability or monitoring for LLM systems, or cloud/ML infrastructure.
Front-end experience with React/TypeScript for building demos and internal tools.
Familiarity with content operations, such as sports data, video metadata, content quality, or similar data-heavy operational settings.

Responsibilities

Partner with the Operations Product Manager to shape proposed solutions and assess technical feasibility while adhering to responsible AI principles.
Curate and own golden datasets and evaluations used for training, testing, and benchmarking.
Lead rapid proofs of concept by creating testing plans and measuring performance against baselines.
Translate POC findings into reports, demos, and hands-on guidance for engineering teams.
Validate production-system performance against POC results and, where applicable, human-level parity.
Research emerging AI capabilities and translate findings into actionable recommendations for the product roadmap and tooling.
Build POCs and automated workflows combining Apple-approved LLMs and multimodal models with tools and data to perform multi-step tasks.
Design prompts, structured outputs, retrieval, and failure handling for AI workflows.
Work with domain experts to build ground-truth datasets, select relevant metrics, and run controlled experiments across models, prompts, preprocessing, and scoring.
Communicate recommendations, limitations, and failure modes through technical write-ups, demonstrations, and engineering handoff guidance.