AI Optimization Analyst (Non-Technical)

Truliant Federal Credit Union•Greenville, SC

About The Position

The AI Optimization Analyst is a critical specialist responsible for taking full ownership of the credit union’s AI-driven member and staff experiences, directly influencing millions of interactions each year. This role serves as the primary expert on AI agent performance, using strategic analysis of conversational data to enhance accuracy, efficiency, and brand alignment with a goal to continuously improve the member experience and frontline productivity. The candidate works closely with the AI Solutions Architect to bring new use cases to life relative to the bots they are responsible for supporting. Key responsibilities include implementing prompt-engineering best practices, managing tone and ethical guardrails, owning end-to-end user acceptance testing of the bots they are responsible for, and consistently bringing forward innovative, high-impact ideas to expand and strengthen Truliant’s AI self-service and internal support capabilities.

Requirements

Must have a strong understanding of conversational AI systems (e.g., AI, NLP, and intent-based bot frameworks).
Must have expertise in prompt engineering and the ability to craft clear, effective, and optimized AI instructions.
Must have analytical skills to interpret conversational data, identify patterns, and translate insights into actionable improvements.
Ability to manage tone, style, and brand voice across AI-generated interactions.
Must have the ability to understand all business processes within the credit union and how to integrate AI solutions that improve member experience and organizational efficiency.
Must have knowledge of ethical AI principles, including bias prevention, guardrail creation, and hallucination mitigation.
Must have experience with UAT processes, including test planning, execution, and validation.
Must have strong collaboration and communication skills to work effectively with technical teams, business partners, and frontline staff.
Must have the ability to translate business needs into functional AI requirements and conversational design elements.
Must have familiarity with analytics platforms (e.g., dashboards, reporting tools, or conversation intelligence systems).
Must have a problem-solving mindset, with the ability to diagnose issues and propose creative, scalable solutions.
Must have strategic thinking skills to identify new use cases and guide long-term AI evolution.
Must have attention to detail, especially in reviewing AI responses, workflows, and system changes.
Must have an understanding of user experience (UX) and conversational design principles.
Must have the ability to manage multiple priorities in a fast-moving, innovation-focused environment.
Must have a curiosity and continuous-learning mindset to stay current on emerging AI capabilities and best practices.
Must be able to work in a team environment, with strong collaborative skills.
Must have superior interpersonal skills; ability to get along with diverse personalities; tactful, mature, flexible, respectful.
Must be able to work in a general office environment.
Must be flexible and able to shift resources and priorities as required.
Must be able to complete all assignments with minimal supervision.
Should possess a strong commitment to providing excellent service to Truliant’s members.
Occasional standing, walking, bending, and stooping required.
Must be able to sit at a desk for long periods of time and use a computer.
Must be able to moderately lift or move up to 5 pounds and occasionally lift or move up to 10 pounds.
Bachelor’s degree required in a relevant field such as Business Administration or Information Systems.
2–5 years of experience working with conversational AI platforms, generative AI systems, NLP technologies, or intent-based bot frameworks (e.g., Kore.ai, Dialogflow, Genesys, or similar) required.
Demonstrated experience in prompt engineering, AI response tuning, or bot conversation strategy required.
Hands-on experience leading or contributing to user acceptance testing (UAT), including test planning, scenario design, execution, and validation required.
Experience partnering with technical roles, such as AI engineers, product owners, or solutions architects, to translate business needs into functional requirements required.

Nice To Haves

Background working in financial services, credit unions, banking, customer experience design, or regulated environments is a plus.
An equivalent combination of education and directly applicable work experience may be considered in place of a degree, particularly in AI operations, conversational design, or analytics-focused roles.

Responsibilities

Owns and manages the end-to-end performance of the credit union’s AI self-service and internal support agents with a primary focus on Kore.ai.
Develops and maintains documentation for prompts, workflows, standards, and performance metrics.
Analyzes conversational data and system analytics to identify trends, gaps, and improvement opportunities.
Implement sand refine prompt-engineering best practices to enhance clarity, accuracy, and efficiency in AI responses.
Ensures all AI-generated interactions reflect the credit union’s brand voice, service standards, and member-first philosophy.
Assists in the process of monitoring and maintaining ethical guardrails, preventing hallucinations, incorrect assumptions, and off-brand behavior.
Continuously tunes and optimizes AI agent workflows, response logic, and model-driven capabilities.
Partners closely with the AI Solutions Architect to define, prioritize, and bring new AI use cases to life.
Designs and executes comprehensive testing plans, including leading UAT efforts to validate and approve all new features, prompts, and conversational flows.
Collaborates with business units and frontline teams to understand needs and translate them into AI enhancements.
Tracks and reports on KPIs such as containment, accuracy, satisfaction, escalation drivers, and operational impact.
Stays informed on AI trends and emerging capabilities, proactively recommending strategic improvements to maximize member and staff value.
Assists with other tasks and projects as assigned.