Manager, Listening Test Development

Center for Applied Linguistics•Washington, DC

46d•$75,000•Remote

About The Position

CAL has reopened the search for the Manager, Listening Test Development, who will be responsible for overseeing the development of Listening test items and ancillary materials for the WIDA suite of assessments, which are large-scale English language proficiency assessments for multilingual learners in grades 1-12. This position involves reviewing, revising, and approving test items (linguistic content, graphics, scripts, audio files) that align to the WIDA English Language Development Standards Framework, with a focus on the language domain of Listening. The Manager, Listening Test Development, oversees the test development process from start to finish. This position develops test refreshment plans, timelines, and procedures; provides comprehensive feedback to test developers; and signs off on test items at critical points in the development process.

Requirements

A Master’s degree from an accredited college or university, in a relevant field, with at least two years of relevant experience.
Expertise in the development of large-scale English oral language proficiency assessments for K-12 multilingual learners.
Apply their understanding of multilingual learners’ English oral language development and K-12 education standards and practices to the development of oral language proficiency test items.
Coordinate remote staff across multiple tasks to meet deadlines and quality control expectations.
Provide clear, constructive feedback that both improves test items and builds staff understanding and skills.
Understand complex test designs with multiple interconnected parts.
Work independently with minimal support/supervision.
Prioritize tasks appropriately, meet deadlines, and work well within a collaborative, fully remote team.
Attend closely to details without losing sight of the bigger picture.
Demonstrate flexibility in responding to feedback and evolving project priorities.
Show leadership to attain team goals.
Bring both analytical and creative approaches to problem solving.

Nice To Haves

Experience supervising the development of test items in a high-stakes context.
Experience managing complex assessment projects.
Deep familiarity with the oral language that K-12 multilingual learners must understand to be successful in U.S. school settings.
Experience teaching K-12 multilingual learners in U.S. school contexts.
Exemplary writing, editing, and proofreading skills in English.
Excellent organization, planning, and time management skills.
Strong oral and written interpersonal communication skills.
Proficiency in a language other than English.
At least intermediate proficiency with Microsoft Office and Smartsheet, or other work management platforms.

Responsibilities

Manage test development staff who write, review, revise, and select Listening test items based on item specifications, test maps, and blueprints
Evaluate test items for item difficulty, oral language features, linguistic complexity, grade-level appropriateness, content correctness, universal accessibility, appropriate scaffolding, fairness, sensitivity, and adherence to style guides
Give clear, constructive, and timely feedback to test developers on test item text and graphics using test item specifications
Sign off on test item content at the end of each phase of the item development cycle
Work in close collaboration with managers and directors across functional teams and partner organizations to coordinate efforts
Produce and execute project plans and timelines for test item development in coordination with client and other vendors
Oversee the development and revision of item specifications and sign off on revisions
Coordinate the recruitment and contracting of consultant item writers and reviewers
Steer the creation of training materials to prepare item writers and reviewers
Ensure test development staff provide content, administrative, and technical support for item writers and reviewers
Lead meetings during which stakeholders review the content, fairness, and grade-appropriateness of test items, and ensure stakeholder feedback on test items is resolved
Work with test development staff and audio recording studio to cast actors and produce recordings for Listening test items, overseeing recording sessions and quality monitoring
Write, review, and edit written reports, procedures, and foundational documents for internal and external stakeholders
Update and maintain style guides and item development checklists
Ensure test development staff follow quality control and test security procedures
Conduct pilot testing and cognitive lab interviews in support of test development
Coordinate, conduct, and sign off on quality checks of field test QTI packages and ancillary materials
Organize and execute independent answer key verifications
Review and update field test functional rules
Review and approve test maps, test forms, ancillary documents, test practice items, and sample items for the public
Write, review, and maintain test and item design plans
Write, review, and maintain the Listening Test Construct Framework document
Ensure accuracy of data in test item databases
Set functional team goals and expectations for Listening test development
Monitor progress and performance of staff completing Listening test development tasks
Mentor test development staff in their job duties and professional goals
Fulfill administrative responsibilities related to hiring/retention and performance reviews
Other duties as assigned