Applied Science Manager, Sponsored Products and Brands

Amazon • Seattle, WA

3d • Onsite

About The Position

The Sponsored Products and Brands team at Amazon Ads is re-imagining the advertising landscape through industry leading generative AI technologies, revolutionizing how millions of customers discover products and engage with brands across Amazon.com and beyond. We are at the forefront of re-inventing advertising experiences, bridging human creativity with artificial intelligence to transform every aspect of the advertising lifecycle from ad creation and optimization to performance analysis and customer insights. We are a passionate group of innovators dedicated to developing responsible and intelligent AI technologies that balance the needs of advertisers, enhance the shopping experience, and strengthen the marketplace. If you're energized by solving complex challenges and pushing the boundaries of what's possible with AI, join us in shaping the future of advertising.

As the Applied Science Manager for the Continuous Model Evaluation and Learning workstream, you will own the quality backbone for this agentic brand-intelligence system. You will lead a mix of applied scientists and engineers who define what "good" looks like for each brand skill, instrument the system to measure it, diagnose why skills underperform, and close the loop by generating, validating, and deploying improvements. You will deliver the evaluation and remediation framework that attains accuracy targets, enables forward evaluation for skills as they develop, and establishes the autonomous detect-diagnose-remediate loop that lets us scale quality across all brand skills and multiple advertiser-facing surfaces.

This is a business-critical, greenfield initiative within SPB. You will set the scientific charter, grow the talent on your team, and ship the framework that every other brand-intelligence workstream depends on.

Requirements

• 4+ years of applied research experience
• 3+ years of scientists or machine learning engineers management experience
• 3+ years of building machine learning models for business application experience
• PhD, or Master's degree and 6+ years of applied research experience
• Knowledge of ML, NLP, Information Retrieval and Analytics
• Experience programming in Java, C++, Python or related language

Nice To Haves

• Experience working on recommender systems or personalization within search, e-commerce, shopping, advertising or other related fields
• Ph.D. in computer science, machine learning, engineering, or related fields, or Master's degree and 4+ years of a quantitative field such as statistics, mathematics, data science, business analytics, economics, finance, engineering, or computer science experience
• Have publications at top-tier peer-reviewed conferences or journals
• 5+ years of scientists or machine learning engineers management experience
• Hands-on experience designing, deploying, and evaluating LLM-based agentic systems, including tool use, multi-step planning and reasoning, retrieval-augmented generation (RAG), and multi-agent orchestration (orchestrator-worker, swarm, critic-actor patterns).
• Experience with large-scale LLM fine-tuning (SFT, RLHF, DPO), prompt engineering, and programmatic prompt optimization frameworks.

Responsibilities

• Lead, mentor, and grow the talent on a team composed of applied scientists and machine learning engineers, fostering a culture of scientific excellence, customer obsession, and ownership.
• Own the scientific vision and multi-quarter roadmap for continuous model evaluation and learning across the brand-intelligence system.
• Design and deliver evaluation frameworks for agentic brand-intelligence skills, including LLM-as-Judge rubrics, multi-model ensemble judging, gold-set construction, and calibration against human evaluators.
• Lead development of the optimization engine that programmatically refines prompts, generates synthetic training pairs, and composes agent decomposition strategies (orchestrator-worker patterns) when single-agent skills hit complexity limits.
• Establish rigorous offline-to-online consistency, A/B testing discipline, and drift monitoring so that quality improvements generalize to production traffic.
• Communicate scientific vision, research breakthroughs, and business outcomes to senior leadership, and drive alignment with broader Amazon Advertising objectives.

Benefits

• health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage)
• 401(k) matching
• paid time off
• parental leave
• sign-on payments
• restricted stock units (RSUs)

What This Job Offers

• Job Type: Full-time
• Career Level: Manager