Quality Assurance Analyst/Conversational AI Testing

BTI•Adelphi, MD

About The Position

Business Technology Integrators (BTI) is a Service-Disabled Veteran-Owned Small Business (SDVOSB) with more than 25 years of experience delivering innovative IT solutions to the Federal Government. We are committed to excellence, innovation, and supporting mission-critical programs that serve our nation. We are seeking a skilled Quality Assurance Analyst to join our team and support testing activities for Kore AI's conversational AI platform. The ideal candidate will have a strong background in software testing, with a focus on AI and natural language processing technologies.

Requirements

Bachelor’s degree in computer science, information technology, or related field.
At least 5 years of experience in software testing and quality assurance methodologies.
Knowledge of AI and machine learning concepts, particularly in natural language processing.
Expertise in testing conversational AI or chatbot systems.
Excellent analytical and problem-solving abilities.
Strong communication and collaboration skills.

Nice To Haves

Experience in testing Chatbots such as Microsoft Bot and Kore AI platform.
Certifications in software testing or AI (e.g., ISTQB, AI Testing certification).

Responsibilities

Define functional and non-functional test strategies for the conversational AI platform.
Collaborate with AI developers and product owner to design comprehensive test cases covering various user scenarios, edge cases, and potential failure points.
Conduct rigorous testing of conversational AI models, including: Functional testing to verify correct behavior, assess the accuracy, relevance, and coherence of AI-generated responses.
Performance testing to evaluate response times, system load, and resource utilization.
Usability testing to assess the user experience and identify areas for improvement.
Coordinate security testing with the Information Technology Security team to identify and mitigate potential vulnerabilities.
Integration testing to ensure the product is integrated with various platforms and that data integrity is maintained between various back-end systems.
Coordinate with cross-functional quality assurance resources to conduct end to end testing between other systems/platforms.
Generate detailed reports on test executions, including bug reports, performance metrics, and user experience insights.
Analyze test results to identify trends, patterns, and areas requiring improvement and work with the product owner for prioritization in the product backlog.
Track and manage bug resolution and feature enhancements.
Provide actionable feedback to AI developers to improve model performance, accuracy, and user satisfaction.
Collaborate with the team to refine training data, adjust model parameters, and optimize algorithms.
Own User Acceptance Testing (UAT) and facilitate by providing demos and test data of the product to the UAT testers.
Keep up to date with the latest developments in conversational AI, natural language processing, and machine learning.
Research and explore new testing methodologies and tools to enhance the QA process.