Product Manager II, AI Evaluations

Box•Redwood City, CA

5d•$160,500 - $200,500•Hybrid

About The Position

Box (NYSE:BOX) is the leader in Intelligent Content Management. Our platform enables organizations to fuel collaboration, manage the entire content lifecycle, secure critical content, and transform business workflows with enterprise AI. We help companies thrive in the new AI-first era of business. Founded in 2005, Box simplifies work for leading global organizations, including JLL, Morgan Stanley, and Nationwide. Box is headquartered in Redwood City, CA, with offices across the United States, Europe, and Asia. By joining Box, you will have the unique opportunity to continue driving our platform forward. Content powers how we work. It’s the billions of files and information flowing across teams, departments, and key business processes every single day: contracts, invoices, employee records, financials, product specs, marketing assets, and more. Our mission is to bring intelligence to the world of content management and empower our customers to completely transform workflows across their organizations. With the combination of AI and enterprise content, the opportunity has never been greater to transform how the world works together and at Box you will be on the front lines of this massive shift.

Requirements

We are an AI-first company. This means you approach your work with a growth mindset and find ways to leverage AI to help make faster, smarter decisions that will 10X your impact at Box.
You have a minimum of 2 years of product management experience.
You know the AI landscape deeply, follow frontier models closely, and can hold your own in a technical conversation with any model provider.
You're technical and hands-on: statistically literate and comfortable enough in code (Python, SQL, notebooks) to build your own evals, run your own analyses, and prototype your own data pipelines.
You've driven cross-functional launches across Product Marketing, Communications, Developer Relations, and Engineering.
You're an excellent written and verbal communicator who can represent Box externally, presenting complex, technical topics simply to senior stakeholders.
You operate with extremely high agency, taking ambiguous, undefined problems and driving them to resolution on tight timelines.
You're fluent across many stakeholders—from Product Marketing and Sales to Security, Legal, and Compliance—adapting your communication to each audience.
You're a natural disambiguator who makes messy, fast-moving AI problems simple and clear.

Nice To Haves

Direct experience with AI/ML products, model evaluation, or benchmarking.
Experience managing external partnerships with vendors or providers.

Responsibilities

Run rigorous evaluations across frontier models (OpenAI, Anthropic, Google/Gemini) and continuously mature the stack, methodology, and process behind them.
Define what makes a high-quality evaluation dataset and robust, cost-appropriate graders, and translate what end users care about into measurable criteria.
Drive synthetic data generation and data-generation pipelines to ensure they are high-quality, reliable, and scalable.
Own online evaluations, turning real production usage into rich insight on how our product is actually used.
Lead each model launch end to end, managing the process, the stakeholders, and the timeline.
Own the external narrative for each model launch — partnering with Product Marketing on blog, social, and PR content, and creating video demos yourself.
Build and manage deep relationships with model providers and AI companies — securing early access to frontier models, delivering technical deep-dives back to providers, and identifying collaboration opportunities on evaluations and benchmarking.
Identify, pitch, and structure opportunities to collaborate with providers and other AI companies specifically on evaluations and benchmarking.
Run evaluations to decide whether a model is production-ready, and own the decision of which models power agentic products at Box.
Assess the compliance posture of production models, partner on model pricing decisions, and define how and when we deprecate models.