An introduction illustrates the two conceptually distinct sets of questions behind impact evaluation. The first set of questions is devoted to establishing whether a given intervention produces the desired effects on some dimension of interest. The key question here: “does it make a difference?” This is answered by identifying and estimating causal effects through counterfactual methods. The second set of questions is primarily devoted to understand why an intervention produces intended and unintended effects, for whom and in which context. The key question here: “why and when does it work?”. This is answered by identifying the theory of change behind the program and assessing its success by comparing theory with actual implementation.
The second chapter aims to explain the counterfactual logic and its limitations: it introduces randomized controlled trials, clearly stating their limited relevance within the evaluation of cohesion policy; finally, it explains the logic of non-experimental methods and how they deal with the problem of selection bias as well as natural dynamics bias.
The ensuing four chapters explain in accessible language the main counterfactual methods for estimating impacts, namely statistical matching based on the propensity score, difference-in-differences, discontinuity designs and instrumental variables. All the chapters contain a revised bibliography.
All chapters are organized according to the structure: description and purpose of the method; circumstances in which it is applied; main steps involved; strengths and limitations of the method, and contain both stylized and real life examples.
- Introduction to impact evaluation
- Introduction to CIE
- Difference-in-differences
- Propensity score matching
- Discontinuity design
- Instrumental variables