Product Analyst

INSPYR Solutions • Austin, TX

1d • Onsite

About The Position

The Product Analyst will join the Channel Strategy & Operations team to support the development of a portfolio of strategic AI agents and an enterprise agent marketplace/platform. You own the evaluation, testing, measurement, and quality component of the products being built.

Requirements

• 2-5 years of experience in Testing, Measurement, Evaluation or Quality Assurance
• Experience designing and executing evaluations to measure accuracy, quality, reliability, or performance
• Experience developing test cases or evaluation frameworks, analyzing results, identifying failure patterns, and validating outputs
• Strong analytical and problem-solving skills with the ability to translate data and testing results into actionable insights
• Hands-on experience with Claude or other agentic AI/coding tools, including using AI to automate workflows, testing, analysis, or other tasks
• Strong cross-functional collaboration skills with experience partnering with Product, Engineering, Data/AI, and business stakeholders

Nice To Haves

• Understanding of sales organizations, sales processes, market expansion, or sales operations
• Experience working within a large, highly matrixed organization
• Portfolio or examples of AI products, prototypes, workflows, or applications personally developed or helped bring to market

Responsibilities

• Develop and execute evaluations to measure the accuracy, quality, reliability, and overall performance of AI agents and agentic workflows.
• Create test cases, evaluation criteria, guardrails, and measurement frameworks to validate agent outputs against expected outcomes.
• Analyze evaluation and testing results, identify failure patterns and edge cases, and translate findings into actionable product and engineering recommendations.
• Use Claude or similar agentic AI/coding tools to automate testing and evaluation workflows, analyze results, and improve the efficiency of quality processes.
• Partner closely with Product and Engineering teams throughout development to validate requirements, test new functionality, troubleshoot issues, and improve agent performance.
• Develop reporting and analytics that provide visibility into product quality, performance trends, and opportunities for improvement.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed