An evaluation that analyzes the actual effect or impact of a program on its intended target, along with unintended consequences.
(Pal, 2006, p. 322)
Impact evaluations are used to measure the success of government programs and policies.
For example, the World Bank has a database of impact evaluations of its economic intervention projects. The following categories are used [by the World Bank] to classify evaluation methods. These categories are in practice often combined:
1) Randomization or Experimental Design. This method applies to interventions where participants are randomly assigned to the intervention. Participants and non-participants have the same ex-ante probability of participating in the intervention. Impact can be estimated by comparing the two groups.
2) Propensity Score Matching. This method calculates propensity scores (probability of participating in the intervention as a function of observed characteristics) for participants and non-participants. Participants are matched to non-participants on the basis of their scores.
3) Pipeline Comparison. This method uses those who have applied and are eligible to receive the intervention in the future, but have not yet received it, as a comparison group. Their only difference with the current recipients is that they haven’t yet received the intervention.
4) Simulated counterfactual. This method is used for interventions affecting the entire population, for which no comparison group can be identified. A counterfactual distribution of outcomes in the absence of the intervention is simulated on the basis of a theoretical model and information on the situation prior to the intervention.
5) Difference in means or Single Difference. This method estimates impacts by comparing the value of the indicator of interest for the recipients and the non-recipients.
6) Difference-in-difference or Double Difference. This method estimates impacts by comparing the value of the indicator of interest between the recipients and non-recipients (first difference) before and after an intervention (second difference).
7) Instrumental Variables. This method uses instrumental variables (that affect receipt of the intervention but not the outcomes of interest) to control for selection bias when intervention placement is not random.
From the World Bank website, Poverty Impact Evaluations Database (The World Bank Group 2010)