Program Evaluation & Impact Analyst

FIVE18 Family Services Operations Inc.•Lynchburg, VA

3d•$17 - $23•Hybrid

About The Position

The Program Evaluation and Impact Analyst ensures FIVE18 has reliable, meaningful, and accessible program data that equips the Chief Program Officer and program leaders to evaluate effectiveness, make evidence informed decisions, continuously improve services and strengthen a culture in which data and consumer voice are routinely used to understand and increase impact. The Analyst joins FIVE18 at an important stage in the development of its evaluation infrastructure, stewarding strong existing practices while expanding organization-wide measurement systems, dashboards, standardized outcomes, and reporting processes. This role is expected to build, maintain, and improved the scalable measurement infrastructure needed to support FIVE18's Evidence-Based Impact value and long term goals. This individual will communicate effectively with colleagues and consumers to create and cultivate a positive impression of FIVE18, striving to present a culture of hospitality and competence.

Requirements

Applied program evaluation, outcome measurement, data-quality practices, continuous quality improvement, and evidence review
Quantitative and qualitative analysis and appropriate research ethics
Dashboard, reporting, survey, database, and data visualization practices
Be committed to diversity
Demonstrate analytical rigor, curiosity, discernment, empathy, and relational humility
Work collaboratively and proactively with others
Be energized by an environment in which some systems are established, others are developing, and new needs will emerge
Be comfortable asking questions and receiving questions in a high-feedback culture
Develop a strong working understanding of FIVE18's programs, consumers, strategy, evaluation systems, and organizational goals
Computer
Telephone
Scanner
Copier
Printer
Vehicle
Bachelor's degree with a minimum of 3 years of applied analysis, program evaluation, performance measurement, or closely related experience preferred.
Associate's degree may be considered with more than 5 years of strong applied analysis and evaluation experience.
Applied experience conducting program evaluations, analyzing quantitative and qualitative data, assessing outcomes, maintaining data quality, designing or using surveys and feedback tools, developing reports or dashboards, communicating findings to non-technical audiences, and using evidence to support program improvement and decision-making.
Demonstrated proficiency with Google Workspace, particularly advanced spreadsheet analysis in Google Sheets.
Experience with databases, case-management or program data systems, dashboards, reporting or visualization platforms, survey tools, and related data technologies.
Ability and willingness to quickly learn unfamiliar systems and emerging technologies; prior experience with FIVE18-specific platforms is not required.
Authentic passion for working with a team to collectively effect change in families.
Ability to maintain appropriate confidentiality and handle consumer and organizational data responsibly.
Must have a valid driver's license and an acceptable driving record, with safe and reliable transportation for work-related purposes.

Nice To Haves

Human services and trauma-informed approaches to care (preferred upon hire)
Trauma-informed modalities of care (preferred upon hire)
Commit to the Mission, Vision, Values, and Evidence-Based Impact culture of FIVE18
Nonprofit or human services experience is strongly preferred; relevant experience in healthcare, education, public administration, research, or another mission-driven program environment may also be considered.
Providing direct trauma-informed care to children and/or families is a plus but not required.

Responsibilities

Build, maintain, and improve meaningful outputs, outcomes, indicators, evaluation tools, and recurring evaluation processes.
Maintain continuity of established evaluation systems while identifying thoughtful opportunities for refinement.
Document measures, definitions, methodology, data sources, reporting processes, and historical trends so evaluation knowledge is organizationally accessible.
Monitor the accuracy, completeness, consistency, and reliability of program data; identify and communicate data-quality concerns, gaps, anomalies, and barriers to useful analysis.
Analyze quantitative and qualitative data to identify meaningful trends, patterns, gaps, risks, and opportunities rather than simply reporting numbers.
Apply healthy skepticism, test assumptions, investigate inconsistencies, and distinguish what the evidence supports from what it does not.
Develop sufficient understanding of FIVE18's programs, consumers, strategy, goals, and evaluation systems to exercise sound judgment about what deserves further investigation, program-level attention, or CPO awareness.
Determine what to elevate, when to elevate it, and the appropriate level of analysis before doing so, without relying on the CPO to routinely direct what should be analyzed.
Facilitate recurring evaluation processes that connect evidence to program learning, improvement, and decision-making.
Assess the impact of program adaptations and improvement efforts over time.
Integrate participant feedback, satisfaction data, qualitative information, and consumer voice alongside quantitative measures.
Maintain a systems-level understanding of who FIVE18 serves, who may not be represented in the data, and what quantitative measures may not fully capture.
Build, maintain, and improve dashboards, Right Performance Metrics (RPMs), organizational impact measures, and routine reporting tools.
Support reliable reporting of reach, engagement, outcomes, satisfaction, consumer characteristics, and other meaningful indicators.
Increase the efficiency and reliability of data collection, aggregation, analysis, visualization, and reporting, including appropriate use of automation and emerging technology.
Help build toward standardized, scalable outcome measurement and the infrastructure required to assess progress toward annual and long-term organizational goals.
Work collaboratively and proactively with Focus Area Leaders to strengthen evaluation practices across programs and increase consistency where appropriate.
Help staff and leaders understand, interpret, and routinely use data for learning and improvement.
Communicate complex information clearly and respectfully to people with varying levels of comfort with data, welcoming questions and adapting explanations as needed.
Partner with Focus Area Leaders around data-quality needs while maintaining clear accountability boundaries: program staff own accurate and timely data entry, and Focus Area Leaders own program-level follow-through and performance.
Research relevant evidence and promising practices and help establish appropriate measures for new or evolving initiatives.
Identify gaps between what FIVE18 measures today and what it will need to measure as programs, reach, partnerships, and organizational complexity grow.
Monitor data and system health, maintain dashboards and evaluation systems, investigate meaningful questions, and proactively collaborate with program leaders.
Lead the Program Impact & Learning Review with the Chief Program Officer, setting the primary analytical agenda and bringing forward the reports, trends, concerns, opportunities, and evaluation questions most deserving of CPO attention.
Conduct deeper program and organizational analysis examining performance, outcomes, reach, consumer experience, trends, data quality, and continuous improvement efforts.
Provide a broader assessment of program and organizational impact, consumer profile, outcomes, satisfaction, multi-year trends, annual goals, and progress toward FIVE18's long-term measurement and impact objectives.
Conduct timely targeted analysis when significant programmatic or organizational questions arise.
Participate in meetings, training, and consultative supervision as appropriate.
Create and cultivate a positive impression of FIVE18 in all communication modalities and contexts, striving to present a culture of hospitality and competence.
Produce accurate, timely, consistent, thorough, and concise work.
Conduct detailed program evaluations and produce reports that synthesize quantitative and qualitative findings, identify meaningful trends and areas for improvement, and provide evidence-informed insights and actionable recommendations for program enhancement.
Produce timely analysis, reports, dashboards, and other reporting outputs that provide meaningful visibility into program performance, consumers served, outputs, outcomes, and emerging trends. Exercise professional judgment in determining which findings warrant routine reporting, deeper investigation, or leadership attention.
Lead the monthly review with the Chief Program Officer, bringing forward the reports, analysis, trends, data-quality concerns, emerging insights, and recommendations most deserving of leadership attention. Set the primary analytical agenda while incorporating specific areas of review requested by the CPO.
Produce quarterly reports and analysis examining program performance, outcomes, reach, consumer experience, data quality, trends, and continuous improvement efforts. Include relevant comparisons over time and identify findings or recommendations that warrant program or leadership attention.
Produce an annual synthesis of program and organizational impact, outcomes, consumer characteristics, satisfaction, multi-year trends, annual goal performance, and progress toward longer-term organizational impact objectives. Identify significant findings and evaluation priorities for the coming year.
Develop and maintain appropriate consumer and stakeholder feedback processes. Produce meaningful summaries and analyses that integrate qualitative feedback, satisfaction, and consumer voice with quantitative program data.
Build, maintain, and improve evaluation frameworks, outcome measures, dashboards, RPMs, and related measurement tools that enable ongoing assessment of program and organizational effectiveness.
Produce timely analyses, research summaries, or briefings in response to significant programmatic or organizational questions, providing evidence and context that support CPO and leadership decision-making.
Maintain accurate and timely documentation of evaluation activities, measures, definitions, methodologies, data sources, calculations, reporting processes, and findings. Maintain a living record of what FIVE18 measures, why it is measured, where the data resides, how it is calculated, how often it is reviewed, and who uses it.