Manager, AI Analysis and Evaluation

Red River • Boston, MA

3d

About The Position

The Red Hat Platforms and Analytics team is looking for a Manager for our Analysis and Evaluation team. In this role, you will lead a diverse team of data scientists, analysts, and AI engineers to optimize performance across our entire portfolio. You will be a key partner for internal product teams and executive leadership delivering customer experiences, providing the insights needed to guide our technical strategy. This is an opportunity to help mature a high-impact function and set the standard for how we measure and improve AI performance at scale. At Red Hat, our commitment to open source innovation extends beyond our products - it’s embedded in how we work and grow. Red Hatters embrace change – especially in our fast-moving technological landscape – and have a strong growth mindset. That's why we encourage our teams to proactively, thoughtfully, and ethically use AI to simplify their workflows, cut complexity, and boost efficiency. This empowers our associates to focus on higher-impact work, creating smart, more innovative solutions that solve our customers' most pressing challenges.

Requirements

• 5+ years of experience managing agile technical teams creating measurable impact against outcomes
• Experience managing data science and analytics functions
• Proficiency in the concepts and vocabulary of data science, AI engineering, and data analysis
• Professional experience establishing and maturing new technical functions or teams
• Excellent communication skills, with the ability to translate complex data insights for both technical teams and executive audiences
• Solid understanding of data visualization tools like Tableau

Nice To Haves

• Background in reviewing code or data models is a plus, though not required for daily tasks
• Experience with AI/ML frameworks or AI evaluations is a plus.

Responsibilities

• Lead and grow a technical sub-group focused on data science, data analysis, and AI evaluation functions
• Mature and accelerate the AI evaluations function, expanding its reach across the full portfolio of experiences we deliver
• Establish and report on key metrics that demonstrate the tangible impact and accuracy of our AI initiatives to executive leadership
• Advise stakeholders on impact measurements and provide clear visibility through data products like Tableau dashboards
• Advise development teams on AI evaluation responsibilities, best practices, and provide assistance
• Collaborate with platform managers to ensure our foundational infrastructure supports advanced analytics and innovative AI performance goals
• Remove roadblocks for your team by managing high-level strategy and fostering a culture of innovation and optimization

Benefits

• Comprehensive medical, dental, and vision coverage
• Flexible Spending Account - healthcare and dependent care
• Health Savings Account - high deductible medical plan
• Retirement 401(k) with employer match
• Paid time off and holidays
• Paid parental leave plans for all new parents
• Leave benefits including disability, paid family medical leave, and paid military leave
• Additional benefits including employee stock purchase plan, family planning reimbursement, tuition reimbursement, transportation expense account, employee assistance program, and more!

What This Job Offers

• Job Type: Full-time
• Career Level: Manager
• Education Level: No Education Listed