Senior Artificial Intelligence Evaluation Engineer

Ciklum • United States,

3d • Remote

About The Position

Ciklum is looking for a Senior AI Evaluation Engineer to join their team full-time in the US. As a Senior AI Evaluation Engineer, you will become part of a cross-functional development team engineering experiences of tomorrow. Ciklum is a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and product owners, they engineer technology that redefines industries and shapes the way people live.

Requirements

• 4+ years in ML/LLM evaluation, QA engineering for AI systems, or applied research engineering
• Hands-on with eval frameworks, LLM-as-judge patterns and their failure modes, statistical rigor on small samples
• Independent spine: the team shipping a thing does not set its own pass bar

Nice To Haves

• Healthcare or safety-critical evaluation experience
• Red-teaming background

Responsibilities

• Build the central eval harness and templates every pod uses; hub-and-spoke: central standards, pod-written tests
• Create golden datasets with business and clinical reviewers, including safety and medical-accuracy suites
• Set and defend pass thresholds; audit pod evals; report eval pass rates and incidents monthly
• Stand up production monitoring for drift and regression with the pipeline engineer

Benefits

• Healthcare
• Basic Life Insurance
• Short and Long-term disability insurance according to the Company’s Benefit Plans
• Tailored learning: Boost your skills with internal events (meetups, conferences, workshops), Udemy access, language courses, and company-paid certifications
• Endless opportunities: Explore diverse domains through internal mobility, finding the best fit to gain hands-on experience with cutting-edge technologies

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed