AI Operations, Applied AI Research and Design Developer

Apple•Cupertino, CA

About The Position

Act as the technical engine accelerating the organization’s AI ambitions by leading the technical design and validation of new opportunities. Develop hyper-scalable, human-centered AI solutions that deliver accurate, timely metadata and high-quality video and audio for Movies, TV, Live Sports, and Apple Originals. Own the technical validation lifecycle, partnering with product management to translate proposals into feasible solutions and developing rapid proofs of concept (POCs) to establish viability. Research emerging AI technologies and use rigorous, data-driven evidence to support go/no-go recommendations. Collaborate closely with the Operations Product Manager to shape proposed solutions, assess technical feasibility, and adhere to responsible AI principles. Curate and maintain golden datasets and evaluations for training, testing, and benchmarking. Build POCs and automated workflows combining Apple-approved LLMs and multimodal models for text, image, video, and audio with tools and data to perform multi-step tasks. This includes prompt design, structured outputs, retrieval, and failure handling. Design evaluations with domain experts, establish ground-truth datasets and relevant metrics, and run controlled experiments across models, prompts, preprocessing, and scoring. Translate results into clear technical recommendations, reports, demonstrations, and practical engineering handoff guidance, including honest reporting of limitations and failure modes. Lead post-launch validation by comparing production performance with POC results and, where applicable, human-level parity. Contribute to long-term technical strategy by turning research into actionable product-roadmap and tooling recommendations.

Requirements

Bachelor’s or Master’s degree in Computer Science, AI/ML, Data Science, or a related field, completed or expected by the start date.
Strong Python skills, including data manipulation and analysis with pandas, SQL, REST APIs, and structured data such as JSON/XML.
Hands-on experience building AI applications with LLM APIs, including prompting, structured outputs, RAG, or tool-using workflows, through coursework, research, internships, or projects.
Sound judgment about when to use an LLM, a classical machine-learning model, or a deterministic approach.
Working knowledge of machine-learning evaluation, including precision, recall, F1, test-set construction, experiment design, and validation against baselines and benchmarks.
Understanding of AI guardrails, including safety, privacy, compliance, and responsible deployment.
Proficiency with LLM-assisted coding tools to design, implement, and iterate on software.
Clear written and verbal communication, including technical documentation and explaining complex results to non-technical stakeholders.
Internship or research experience building and benchmarking an AI system end to end, from data and model through evaluation and recommendation.
Demonstrated ability to work independently and quickly deliver working prototypes as requirements change, through a thesis, publications, or public projects.
Experience translating business requirements into implementable solutions and writing technical specifications and engineering handoff materials.

Nice To Haves

Experience with multimodal models, video or audio processing, LLM-as-judge evaluation, or open-weight model serving.
Experience with agent frameworks, observability or monitoring for LLM systems, or cloud/ML infrastructure.
Front-end experience with React/TypeScript for building demonstrations and internal tools.
Familiarity with content operations, such as sports data, video metadata, content quality, or similar data-heavy operational settings.

Responsibilities

Partner with product management and the Operations Product Manager to shape proposed AI solutions and assess their technical feasibility.
Lead the technical validation lifecycle from evaluating a proposal through building and validating a rapid POC.
Research emerging AI technologies and provide evidence-based go/no-go recommendations.
Curate and own representative golden datasets and evaluations for training, testing, and benchmarking.
Build POCs and automated workflows combining Apple-approved LLMs and multimodal models with tools and data to perform multi-step tasks.
Design prompts, structured outputs, retrieval, and failure-handling approaches for AI workflows.
Work with domain experts to build ground-truth datasets, select business-relevant metrics, and design evaluations.
Run controlled experiments across models, prompts, preprocessing, and scoring; measure results against baselines and benchmarks.
Present findings through technical write-ups, demonstrations, and engineering handoff guidance, clearly explaining limitations and failure modes.
Lead post-launch validation by benchmarking production performance against POC results and, where applicable, human-level parity.
Translate research insights into recommendations for the product roadmap and tooling.
Adhere to responsible AI principles when assessing and developing solutions.