Associate Product Manager, AI Products

Internet Brands•El Segundo, CA

63d•$80,000

About The Position

Internet Brands is seeking an Associate Product Manager, AI Products to help build, manage, evaluate, and optimize AI-powered product experiences across our content, discovery, search, personalization, and workflow products. In this role, you will focus heavily on the practical systems that make AI products useful, reliable, and measurable: prompt design, prompt management, prompt experimentation, AI response evaluation, quality measurement, feedback loops, and performance improvement. You will work closely with product, engineering, data science, UX, content, legal, and business stakeholders to help define how AI systems should behave, how quality should be measured, and how prompt changes should be tested before and after launch. This is a hands-on product role for someone early in their product management career who has practical experience working with LLM-powered products, conversational agents, prompt systems, structured AI workflows, analytics, and rapid experimentation. You should be comfortable reviewing AI outputs, identifying quality gaps, defining evaluation criteria, and helping cross-functional teams improve AI performance through disciplined iteration.

Requirements

1–3 years of product management, AI product, prompt engineering, product operations, research, analytics, or related experience.
Hands-on experience working with LLM-powered products, conversational agents, prompt systems, generative AI workflows, AI copilots, or intelligent automation tools.
Experience writing, testing, refining, or managing prompts for multi-turn AI experiences or structured AI workflows.
Strong ability to evaluate AI outputs and identify quality issues related to accuracy, relevance, completeness, tone, consistency, safety, hallucination risk, or user usefulness.
Familiarity with AI evaluation methods such as rubric-based scoring, human review, golden test sets, prompt comparison testing, grounding checks, safety review, and feedback loop design.
Strong analytical skills, including comfort using product data, dashboards, experiments, user feedback, or structured QA results to recommend improvements.
Ability to translate ambiguous user and business problems into clear product requirements, AI behavior expectations, prompt specifications, and measurable success criteria.
Experience collaborating cross-functionally with engineering, data science, UX, content, legal/compliance, customer-facing, or business teams.
Strong written communication skills, with the ability to document prompt behavior, evaluation criteria, product decisions, and experiment results clearly.
Curious, detail-oriented, and comfortable working in fast-moving AI product environments where product requirements and technical capabilities evolve quickly.

Nice To Haves

Experience with enterprise AI products, workflow automation, search/Q&A products, recommendation systems, self-serve platforms, or data-rich user experiences.
Experience supporting pilots, demos, user testing, customer feedback sessions, A/B tests, or product experiments.
Exposure to SQL, analytics dashboards, product instrumentation, structured QA workflows, or annotation tools.
Experience improving AI behavior through prompt refinement, grounding, model evaluation, safety testing, or structured feedback collection.
Familiarity with retrieval-augmented generation, prompt versioning, AI quality scorecards, human-in-the-loop review, or AI response monitoring.
Background or coursework in cognitive science, computer science, human-computer interaction, data analysis, linguistics, research methods, or a related field.

Responsibilities

Support the development and optimization of AI-powered product features, including conversational interfaces, search/Q&A experiences, content generation workflows, recommendation experiences, personalization, and intelligent workflow tools.
Create, manage, document, and iterate on prompts used across AI product experiences, including system prompts, user-facing prompt flows, structured task prompts, and workflow-specific instructions.
Help define prompt management practices, including versioning, documentation, use-case mapping, change logs, testing workflows, and approval processes for prompt updates.
Partner with senior product leaders, engineering, ML/data science, UX, and content teams to translate product goals into clear AI behavior requirements, prompt specifications, evaluation plans, and launch criteria.
Define evaluation criteria for prompt changes, including accuracy, relevance, completeness, tone, clarity, hallucination risk, groundedness, safety, compliance, consistency, latency, and user task success.
Build and maintain prompt evaluation frameworks that measure quality before and after changes, using human review, test sets, rubric-based scoring, structured QA, user feedback, and product analytics.
Help design experiments to compare prompt variants and measure their impact on AI system performance, user experience, engagement, task completion, and quality metrics.
Review AI outputs across common, edge-case, and high-risk scenarios to identify failure modes, quality gaps, confusing responses, unsafe outputs, or opportunities for improvement.
Support the creation of golden test sets, evaluation rubrics, sample queries, expected answer patterns, and benchmark scenarios for ongoing AI quality monitoring.
Analyze prompt performance using dashboards, SQL-based reporting, qualitative feedback, annotation results, and product usage data to identify where prompts and AI workflows should be improved.
Collaborate with content, legal, compliance, and editorial teams to ensure AI outputs are accurate, trustworthy, transparent, and aligned with company standards.
Document product requirements, AI behavior expectations, evaluation results, prompt decisions, experiment learnings, and recommendations so teams can make informed product and technical decisions.
Support rapid iteration cycles from prototype to pilot to production by helping teams test AI behavior, measure results, and prioritize improvements.