How Artificial Intelligence can be used to analyse Qualitative Data for Impact Evaluation
In this article we describe what is mediation analysis and why is important to test for mechanisms and pathways to get more insights from an RCT.
Funded by the European Union, this project involved the evaluation of and educational program to support unprivileged student (more than 10 thousand students) in different areas of Italy.
It is often said that cycling is a zero-emission mode of transport and has many health benefits.
Monitoring crop yield is central to addressing decisions on the product’s inventory strategy, revenues, and pricing
BACKGROUND OF PROGRAM EVALUATION FOR NGOs AND GOs
Program evaluation is a valuable tool for program managers who are seeking to strengthen the quality of their programs and improve outcomes. Program evaluation answers fundamental questions about a program’s effectiveness, and evaluation data can be used to improve program services. In this article, we outline 3 major reasons why conducting a program evaluation is important and its advantages.
PROGRAM EVALUATION DEFINITIONs
Let’s start with some definitions. Program evaluation can be defined as “a systematic method for collecting, analysing, and using information to answer questions about a program”.
Evaluations are typically divided into two major categories:
Process evaluations assess whether an intervention or program model was implemented as planned, whether the intended target population was reached, and the major challenges and successful strategies associated with program implementation.
Outcome evaluations determine whether, and to what extent, the expected changes in the outcomes of interest occur and whether these changes can be attributed to the program or program activities.
THREE MAJOR REASONS TO CONDUCT A PROGRAM EVALUATION
A program evaluation can showcase the effectiveness of a program to the community and founders.
First, evaluation findings can demonstrate to a community that a program is worthwhile. Sharing findings within the community can serve as a good outreach tool for:
-attracting collaborative partners
-building trust with families and community members
Additionally, founders often require that a program evaluation to be conducted when they agree to fund a program, and some founders will not fund, or re-fund, a program until an evaluation has been conducted and outcomes have been demonstrated. For example, A study conducted in Minnesota highlighted that 83% of founders required program evaluations for grants awarded in the past two years.
A program evaluation can find out “what works” and “what does not work.”
A process or outcome evaluation enables program managers to answer questions about a program’s effectiveness, including (are participants benefiting from program services?) Knowing “what works” helps program managers to focus resources on the essential components of the program model that benefit participants and volunteers; knowing “what does not work” allows program managers to improve and strengthen their service delivery models. Not knowing what is working may waste valuable time and resources.
A program evaluation can help refining and improve the program itself.
Being able to answer fundamental questions about the effects of a program on a set of outcomes might be not enough. For example, if a program influences an outcome, then knowing something about the mechanisms and pathways by which these effects arise may allow for the discarding of components of the program which perhaps are not ultimately important for the outcome.
In the same way, it might also be of interest decomposing the effect of a program in settings in which the program has been found not to influence an outcome. By considering various possible mechanisms and intermediates, it is possible to refine the program to better targeting a mechanism or intermediate, or whether the mechanism or intermediate that was targeted was in fact the wrong one because it had little effect on the outcome.