Data Production Analyst (AI & QA Analyst) (Information Technology Analyst 3)

New Jersey Courts • Trenton, NJ

3d • Hybrid

About The Position

The New Jersey Judiciary is seeking an Information Technology Analyst 3 focused on AI solution testing, quality assurance, and use case analysis. The position bridges business stakeholders and technical development by leading the discovery, elicitation, and documentation of enterprise AI and data use cases, and by developing testing frameworks for AI models, machine-learning pipelines, data warehouse models, and data architectures within AWS cloud infrastructure.

The role designs and executes test plans, automated scripts, and evaluation frameworks for AI solutions, large language models, and retrieval pipelines. It validates model outputs, retrieval relevance, data lineage, and traceability against authoritative source data before production deployment. The analyst also verifies data ingestion, transformation layers, and warehouse structures to support reliable AI models and reporting.

The position collaborates with business subject-matter experts, application teams, data scientists, and machine-learning engineers to turn business challenges into requirements and acceptance criteria, establish testing benchmarks, and address model drift and hallucination risks. It provides first-line production support by monitoring deployed solutions and resolving analytical and pipeline anomalies.

The Judiciary currently offers a hybrid work schedule; this position may qualify for up to three remote workdays outside Judiciary buildings. Participation requires high-speed home internet access and installation of a multifactor authentication app on a phone or tablet. Regular and predictable physical presence at the worksite is an essential function. Salary ranges are $94,668.29–$133,073.24 for Schedule A and $94,668.29–$126,419.57 for Schedule B; newly hired employees normally start at the range minimum.

Requirements

• Bachelor’s degree from an accredited college or university in a field related to computer science, information technology, or the area to which the applicant is applying, plus three years of qualifying professional experience; listed education and experience substitutions may apply.
• Qualifying experience may include analyzing, designing, programming, troubleshooting, resolving technical problems, testing, defining business requirements, training, or installing computer systems applications; network or telecommunications work; system software; business analysis; business analytics; or data analytics.
• Professional experience in AI/ML testing, QA methodologies, and software or data verification.
• Experience in business analysis, use case gathering, requirements engineering, and translating strategic business goals into technical specifications.
• Understanding of cloud-based data and AI architectures, AWS infrastructure, vector databases, or modern data lakehouses.
• Proficiency writing complex SQL queries and using scripting languages such as Python for data validation and test automation.
• Strong communication, project coordination, and multidisciplinary collaboration skills.
• Regular and predictable physical presence at the worksite.
• Selected candidate must be authorized to work in the United States and agree to a background check, including fingerprinting.
• Newly hired employees must establish and maintain principal residence in New Jersey within one year, subject to the stated exemption process.
• A New Jersey driver’s license is required only if operating a vehicle, rather than employee mobility, is necessary for the position’s essential duties.

Nice To Haves

• Successful completion of an internship within the Central Office Information Technology Office, ATCSU, Data Analytics, Research and Statistics Office, or Information Security Unit may be substituted for required professional experience at the stated rate.
• Master’s degree in a related field may be substituted for required education or one year of experience, as described in the posting.
• A bachelor’s degree in any field, or an associate degree with the specified additional qualifying experience, may be used under the stated education substitutions.

Responsibilities

• Design, develop, and execute test plans, automated scripts, and evaluation frameworks for AI solutions, large language models, and retrieval pipelines.
• Validate model-output accuracy, retrieval relevance, data lineage, and traceability against authoritative source data before production deployment.
• Lead discovery sessions with business subject-matter experts and application teams to identify, gather, and formalize AI and data use cases.
• Translate business challenges into functional requirement documents, user stories, and acceptance criteria for engineering teams.
• Verify data ingestion, transformation layers, and data warehouse structures, including AWS Redshift, to ensure reliable data feeds for AI models and reporting KPIs.
• Collaborate with data scientists, machine-learning engineers, and cross-functional units to define testing benchmarks, mitigate model drift and hallucination risks, and enforce quality standards throughout the AI lifecycle.
• Monitor and maintain deployed AI and data solutions and serve as a technical point of contact for analytical and pipeline anomalies.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level