QA Engineer

DataServ Integrations•North Andover, MA

About The Position

DataServ/xFact is seeking a QA Engineer to serve as the subject matter expert for Quality Assurance. You will test automation for our enterprise web, mobile, and backend products, and Help evaluate and improve AI/LLM-powered features, including how they’re tested, measured, and released. You’ll ensure the quality and reliability of our products while helping define and execute the crucial testing and evaluation workflows for our AI-driven features. About xFact/DataServ: xFact and DataServ bring together nearly 40 years of experience delivering innovative technology solutions to public and commercial sectors. We combine a strong foundation in Technology with a fast growing, mission-driven team focused on building impactful solutions for our existing and growing customer base that spans the country. Our culture is collaborative, driven, and focused on doing meaningful work that creates real value.

Requirements

Bachelor’s degree in computer science, or related professional experience in Quality Assurance
2+ years in Software Quality Assurance or a similar role.
Experience designing and executing manual and automated tests
Proficiency with at least one scripting/programming language (e.g., Python, JavaScript/TypeScript, or Java)
Familiarity with test management tools (e.g., TestRail or similar)
Defect tracking systems (e.g., Jira)
Version control (e.g., Git)
Demonstrates excellent written and verbal communication skills, in both collaborative team and customer-facing environments
Candidates are required to complete a drug test and background check

Nice To Haves

team player
excellent written and verbal communicator
tech savvy
strategic thinker
service oriented
motivated
self-disciplined
professional
flexible in a fast-paced environment
works independently with minimal supervision
exhibits strong work ethics
detail oriented
handles demanding situations/timelines
organized
responsible/dependable
willing to receive/provide feedback to improve performance
supportive
strong desire to support and mentor

Responsibilities

Design, develop, and execute manual and automated test plans and test cases for enterprise features across web, mobile, and backend systems
Develop and maintain automated test scripts (e.g., Selenium, Playwright, Cypress) using a scripting/programming language such as Python or JavaScript to increase efficiency and coverage
Run regression suites before releases to ensure existing functionality remains stable as new features and models are deployed
Identify, document, prioritize, and track defects using tools like Jira, and collaborate with developers to drive timely resolution
Integrate tests into CI/CD pipelines (e.g. GitLab CI) and participate in continuous improvement of QA processes, tools, and methodologies
Partner with Product Managers and Data Scientists to define scope, metrics, and goals for evaluating AI and LLM-powered features (e.g., accuracy, robustness, fairness, hallucination rate, safety)
Coordinate collection, labeling, and preparation of datasets used for AI model testing and human-in-the-loop evaluations
Help organize evaluation sprints, red-teaming sessions, and feedback loops, acting as a point of contact between AI/ML, Product, QA, and external evaluators (if applicable)
Compile and summarize evaluation results, providing clear, actionable feedback on model performance and readiness for production deployment
Maintain detailed documentation of evaluation criteria, test data, prompts, and results, ensuring traceability and auditability for AI/LLM releases
Use LLMs and AI tools (e.g., ChatGPT, Copilot, AI testing platforms) to generate test ideas, draft test cases, and assist with log/defect analysis, then validate and refine these outputs
Help evaluate and adopt AI-powered QA tools (e.g., self-healing locators, intelligent test selection, test impact analysis)
Contribute to best practices and guardrails for using AI safely and effectively in QA and evaluation workflows