Test and Assessment Specialist (Job ID: 4613)

Valkyrie•Virginia Beach, VA

15h•$75,000 - $90,000•Onsite

About The Position

Valkyrie Enterprises is seeking a qualified Test and Evaluation Specialist to support the Navy Afloat Maintenance Training Strategy (NAMTS) program. The preferred candidate will have experience developing, validating, and evaluating Navy technical training assessments, with emphasis on performance-based qualification in maintenance, repair, troubleshooting, and equipment operations. The specialist will develop and evaluate written examinations, practical assessments, and qualification evaluation instruments to measure technical knowledge and demonstrated job proficiency. The position requires the ability to distinguish between knowledge-based requirements and hands-on technical competencies, ensuring assessment methods accurately measure a candidate's ability to perform maintenance tasks safely and in accordance with established technical standards. Located in Virginia Beach, VA.

Requirements

Must have Bachelor's degree in Instructional Design, Education, Training and Development, Engineering, a technical discipline, or a related field and at least 4 years of directly relevant technical training, test development, or performance assessment experience; or Associate's degree and at least 8 years of directly relevant Navy technical training, assessment development, curriculum development, or qualification development experience; or High school diploma/equivalent and at least 10 years of directly relevant Navy technical training, maintenance, assessment development, or qualification development experience.
Must have demonstrated experience developing, reviewing, or validating technical examinations, practical skills assessments, qualification standards, JQRs, PQS, or comparable Navy technical training and assessment products.
Must have working knowledge of Instructional Systems Development (ISD), task analysis, learning-objective development, criterion-referenced testing, and performance-based assessment methodologies.
Must have the ability to translate technical maintenance tasks, procedures, and qualification requirements into measurable assessment criteria and observable performance standards.
Must have experience developing technically accurate examination questions, scoring rubrics, evaluation checklists, or other assessment instruments traceable to approved technical references.
Must have the understanding assessment validity, reliability, question construction, and evaluation methods used to measure technical knowledge and demonstrated proficiency.
Must have the ability to read and interpret U.S. Navy technical manuals, maintenance procedures, drawings, instructions, work-control documents, and other authoritative technical references.
Must have knowledge of Navy shipboard or Intermediate-Level maintenance practices, terminology, technical documentation, and qualification processes.
Must have the ability to evaluate assessment results, identify technical proficiency gaps, and recommend improvements to training or qualification products.
Must have experience collaborating with technical SMEs and facilitating structured reviews to validate examination content, assessment criteria, and qualification requirements.
Must have strong technical writing, editing, analytical, research, and communication skills with the ability to produce accurate, concise, and auditable assessment products.
Must have demonstrated ability to manage multiple assessment development actions, examination discrepancies, review comments, deadlines, and configuration-controlled products.
Must have be proficient in Microsoft Word, Excel, and PowerPoint, with the ability to work with electronic learning-management, examination-management, or document-control systems.
Must have the ability to work independently and collaboratively while maintaining effective communication with Navy stakeholders, technical SMEs, and contractor personnel.
Must be eligible to obtain and maintain a DoD Secret security clearance.

Nice To Haves

Five or more years of direct experience developing or evaluating U.S. Navy technical examinations, JQRs, PQS, practical assessments, or formal technical training and qualification products.
Experience supporting NAMTS, CNRMC, an RMC, SIMA, IMF, shipyard, Fleet training activity, or Non-Nuclear Afloat Maintenance Department.
Experience developing or evaluating performance-based technical training and assessments for Intermediate-Level shipboard maintenance and repair in engineering, deck, electrical, combat systems, C4I, or related maintenance disciplines.
Experience conducting Job Duty Task Analysis (JDTA), Training Needs Analysis (TNA), technical competency assessments, or performance gap analysis.
Experience developing Job Performance Measures (JPMs), practical evaluation checklists, standardized oral board questions, and performance-based qualification criteria.
Experience conducting examination item analysis, including item difficulty, question discrimination, distractor effectiveness, and assessment performance trends.
Experience reviewing and resolving examination discrepancies, question challenges, technical reference conflicts, and assessment validation issues.
Familiarity with applicable NAVEDTRA training development guidance, Navy qualification standards, curriculum validation, and performance-based assessment practices.
Experience with Moodle, Navy learning-management systems, CeTARS, or comparable examination-management and training platforms.
Experience analyzing examination results, qualification performance metrics, and assessment data to support evidence-based program improvements.
Familiarity with ADDIE, competency-based training, formative and summative evaluation, and the Kirkpatrick Evaluation Model.
Formal Navy instructor, curriculum developer, instructional designer, or Master Training Specialist qualification/certification.
Experience collaborating with Fleet maintenance SMEs to convert operational maintenance tasks into standardized technical assessments and qualification requirements.
Experience presenting technical assessment findings, performance trends, and recommendations to Navy leadership, government program managers, or technical authorities.

Responsibilities

Analyze technical tasks, qualification requirements, and learning objectives to identify appropriate assessment methods and measurable performance criteria.
Develop, review, and validate written examinations, technical skills assessments, practical performance evaluations, structured oral boards, and standardized evaluation checklists.
Design criterion-referenced assessments and scoring rubrics that measure achievement of established technical knowledge and performance standards.
Develop technically accurate multiple-choice questions, scenario-based assessments, and practical evaluation instruments aligned with approved technical references and qualification requirements.
Establish observable performance indicators, critical task steps, safety requirements, acceptable tolerances, and completion criteria for hands-on technical assessments.
Evaluate existing examination items for technical accuracy, clarity, answer-key validity, distractor effectiveness, and alignment with authoritative references.
Apply established test development principles to support assessment validity, reliability, fairness, and consistent application across participating maintenance activities.
Establish and maintain traceability between examination items, technical tasks, learning objectives, qualification requirements, and supporting references.
Collaborate with Navy subject matter experts (SMEs) and instructional designers to determine appropriate assessment methods based on task complexity, criticality, safety, equipment availability, and operational requirements.
Analyze examination results, performance assessment outcomes, item-analysis metrics, and qualification trends to identify deficiencies in technical knowledge, demonstrated skills, or assessment design.
Conduct performance gap and root-cause analyses to distinguish training deficiencies from other factors, including inadequate practical experience, unclear procedures, or ineffective assessment instruments.
Recommend corrective actions, including examination revisions, assessment improvements, curriculum modifications, and changes to qualification requirements.
Support periodic JQR and examination reviews to ensure assessment products remain technically accurate, current, and aligned with Fleet maintenance requirements.
Maintain examination documentation, revision histories, assessment records, and configuration-controlled products to support quality assurance and program integrity.