Principal AI Quality Developer - REMOTE

Jobgether

6d • Remote

About The Position

This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Principal AI Quality Developer. In this role, you will design and implement testing and evaluation strategies for applied AI and ML systems across various products. Your work will directly affect the quality, reliability, and performance of AI features. By partnering closely with engineering teams, you will help establish a robust framework for evaluating AI capabilities, ensuring they meet customer expectations and industry standards. This is a high-impact position that offers the opportunity to lead quality initiatives in a dynamic environment.

Requirements

• 7+ years of experience in quality engineering or test automation.
• Strong programming skills in Python.
• Hands-on experience in testing AI or ML systems.
• Practical knowledge of LLM-based system validation.
• Experience defining acceptance criteria for engineering projects.
• Proven debugging skills across complex systems.
• Strong communication skills in a collaborative environment.
• Bachelor's degree in Computer Science or related field.

Responsibilities

• Define and drive the quality strategy for applied AI features.
• Conduct test planning and acceptance criteria for AI capabilities.
• Build automated evaluation frameworks for LLM and ML outputs.
• Establish evaluation metrics and scoring for AI behavior.
• Validate workflows and behaviors across various edge cases.
• Lead safety testing in collaboration with security teams.
• Implement post-deployment validation and monitoring.
• Perform issue triage for AI-related defects.
• Mentor engineering teams on AI testing practices.
• Collaborate to embed responsible AI practices in workflows.

Benefits

• Flexible working hours and remote work opportunities.
• Opportunity to work on high-impact AI technologies.
• Collaborative and inclusive company culture.
• Continuous learning and professional development opportunities.
• Possibility to influence product development processes.
• Access to advanced tools and technologies for AI testing.

What This Job Offers

• Job Type: Full-time
• Career Level: Principal