AI Operations Specialist

Bretton AI•San Francisco, CA

29d•$90,000 - $105,000

About The Position

We're looking for a detail-oriented, analytically-minded person to join our AI Operations team. You'll be on the front lines of ensuring our AI agents are performant for banks and fintechs fighting financial crime at scale — monitoring quality, testing new features, and working directly with engineering and customers to keep our platform running at its best. This is a rare opportunity to learn how cutting-edge AI systems work in production while making a real impact on global financial security.

Requirements

1–3 years of professional experience or equivalent internship experience
Strong analytical and critical thinking skills with exceptional attention to detail
Ability to review complex information and identify patterns or issues
Comfortable learning new technical systems independently
Strong written and verbal communication skills
Excited to work in a fast-paced startup environment

Nice To Haves

Exposure to financial services, fintech, or regulated industries
Experience at a fast-paced startup
Familiarity with coding/scripting — SQL, dashboards, quick scripts
Customer-facing experience in any capacity
Genuine interest in AI/ML systems and how they're deployed in the real world

Responsibilities

Conduct quality reviews of AI agents to ensure they meet the highest standards for compliance teams at banks and fintechs
Test new features and custom logic before they go live for customers
Analyze case data to identify patterns, issues, and opportunities for improvement
Work directly with engineering to understand and measure the impact of system changes
Learn state-of-the-art AI monitoring techniques including performance drift detection, human scoring pipelines, and dataset preparation
Support our largest customers by providing operational insights and recommendations
Partner with product and engineering to translate operational findings into platform improvements