Evaluation Helpdesk
- Case studies
Handouts and presentations
Case Studies in Evaluation: Presentation at Evaluation Helpdesk Summer School 2023
Handout: Case Studies in Evaluation: A methodological perspective
Reading list
- Flyvbjerg, B. (2006). Five Misunderstandings About Case-Study Research. Qualitative Inquiry, 12(2), 219-245. https://doi.org/10.1177/1077800405284363
- Estelle Raimondo, 2023. The Rigor of Case-Based Causal Analysis: Busting Myths through a Demonstration. Capacity Development Working Paper Series. Independent Evaluation Group. Washington, DC: World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/Evaluation/files/methods_paper-case_based.pdf
- Valery Ridde, Abdourahmane Coulibaly, Lara Gautier. Case studies. LIEPP Methods Brief n°12, 2023. https://scienceetbiencommun.pressbooks.pub/pubpolevaluation/chapter/case-studies/
- Yazan, B. (2015). Three Approaches to Case Study Methods in Education: Yin, Merriam, and Stake. The Qualitative Report, 20(2), 134-152. https://doi.org/10.46743/2160-3715/2015.2102
- Robert Yin (2013) Validity and generalization in future case study evaluations. Evaluation 19(3) 321–332. Sage Publications. https://doi.org/10.1177/1356389013497081
- Implementation
Handouts and presentations
Integrating implementation into evaluation
Handout: Mapping the implementation space
Reading list
- Bullock et al. Implementation Science (2021) 16:18 https://doi.org/10.1186/s13012-021-01082-7
- Cerna, Lucie (2013) The Nature of Policy Change and Implementation: A Review of Different Theoretical Approaches. OECD, Paris https://www.luciecerna.com/uploads/5/1/2/5/5125288/the_nature_of_policy_change_and_implementation.pdf
- Ridde V, Pérez D, Robert E. (2005) Using implementation science theories and frameworks in global health. BMJ Global Health 2020;5:e002269. https://doi.org/10.1136/bmjgh-2019-002269
- Rogers, Patricia J. and Michael Woolcock. 2023. “Process and Implementation Evaluations: A Primer.” CID Faculty Working Paper No. 433, Harvard University, School of Government https://bsc.hks.harvard.edu/publications/process-and-implementation-evaluations-a-primer/
- Stame, N., (2010) What Doesn’t Work? Three Failures, Many Answers. Evaluation 16(4) 371–387. https://doi.org/10.1177/1356389010381914
- Evaluation use
Handouts and presentations
Making Evaluation Useful: Presentation
Handout: Evaluation use
Reading list
- Alkin, M. C., & King, J. A. (2017). Definitions of Evaluation Use and Misuse, Evaluation Influence, and Factors Affecting Use. American Journal of Evaluation, 38(3) https://doi.org/10.1177/1098214017717015
- Delahais, T. D., What uses of evaluation along the policy cycle?
- Højlund, S. (2014). Evaluation use in evaluation systems – the case of the European Commission. Evaluation, 20(4), 428-446. https://doi.org/10.1177/1356389014550562
- Kupiec, T., Celińska-Janowicz, D., & Pattyn, V. (2023). Understanding evaluation use from an organisational perspective: A review of the literature and a research agenda. Evaluation, 29(3), 338-355. https://doi.org/10.1177/13563890231185164
- Mark, M. M., & Henry, G. T. (2004). The Mechanisms and Outcomes of Evaluation Influence. Evaluation, 10(1), 35-57. https://doi.org/10.1177/1356389004042326
- Weiss, Carol (1998) Have we learned anything new about the use of evaluation?. American Journal of Evaluation Volume 19, Issue 1, Winter 1998, Pages 21-33 https://doi.org/10.1016/S1098-2140(99)80178-7
- Techniques
Daniele Bondonio - Cross-Regional Sequential Difference in Difference (CR-SEQDD): An Empirical Approach for Evaluating EU
Thematic-Objective Interventions with Regional Data Aggregated at the National Level, Technical Note
Summer School, Bologna, June 2024
Presentations
Organisation and rules of the game
Evaluation and the policy cycle – ensuring that evaluations are relevant and useful for the policymaker
Involving stakeholders – experience in Emilia Romagna, Estonia and Sweden
Making the Evaluation Plan operational – evaluability and design
Portugal: Evaluability studies in evaluation planning
How best to evaluate different types of interventions, policy instruments and programmes
Opportunities for the use of AI in evaluations
Use of macroeconomic models in evaluation
Summer School, Madrid, July 2023
Presentations
Organisation of training and ‘rules of the game’ of working groups
Survey of the participants
The evaluation strategy: the "soul" of the plan
Integrating implementation into evaluation
Quality assurance: tips for practitioners
Case studies in evaluation
Approach to the evaluation planning in Portugal
Experience from the preparation of the evaluation plan in Czechia
Communication of evaluation findings – Estonia
Evaluation plan of Region Sardinia
Evaluation: Swedish challenges and needs
Handouts
Evaluation design steps
Evaluation design building blocks
Quality templates: broadening methods
Quality checklists for impact evaluations
Summer School, Prague, June 2022
Presentations
Evaluation Planning: process and building blocks
Design and manage for use – tips for practitioners
Introduction to the master class: why territorial effects?
Why, when and how to estimate (causal) effects on territories, CS-SEQDD method and other options
What to do IF…? Quality edition
Evaluation Design Building Blocks/Evaluation Planning Building Blocks, 1
Evaluation Design Building Blocks/Evaluation Planning Building Blocks, 2
Writing better evaluation questions
Quality checklists for Impact evaluations
Summer School, Tallinn (EE), Hybrid event, October 2021
Presentations
Evaluation Design and Planning: Introductory remarks
Evaluation Helpdesk. Organisation of training and ‘rules of the game’ of working groups
Designing an ex-post evaluation for use. 12 tips for practitioners
Evaluation Planning
Evaluation Planning. Handout
Evaluation Planning: Indicative ‘Building Blocks’
Evaluation Design Steps
Summer School, Online event, October 2020
Presentations
Strengthening Structural Fund evaluation systems
Evaluation of research, technological development and innovation
Impact evaluations on Support to SMEs: a guide to practice
Support to SMEs (Thematic Objective 3): Review of evaluations undertaken in Member States
Support to energy efficiency and renewables (Thematic Objective 4): Review of evaluations undertaken in Member States
Summer School, Burgas (BG), July 2019
Presentations:
Evaluation questions – Stakeholder needs and evaluation design
Evaluating whole programmes? …. Or delimiting the scope and scale of evaluations
How to make use of evaluation results
Reflections of practitioners – case study of ERDF (DE)
Summer School, Balatonfüred (HU) May 2018
Presentations:
Evaluating results and impacts
Designing evaluations (1) – Frameworks and evaluation questions
Designing evaluations (2) – Methodological choice
A practitioner’s guide to counterfactual impact evaluation
Counterfactual impact evaluation – case study of 1994-1999 ERDF programme, IT
What is theory based impact evaluation?
Theory based impact evaluation – case study in the field of enterprise support
Managing evaluations – case study of ERDF (DE)
Summer School in Athens, June 2017
Preparatory Material:
Unpacking Black Boxes: Mechanisms and Theory Building in Evaluation
Are social mechanisms usable and useful in evaluation research?
EVALSED: The resource for the evaluation of Socio-Economic Development
Presentations :
Frans Leeuw: General Introduction and Background information
Frans Leeuw: Coleman boat model
Frans Leeuw: Wrap up and consequences for the ToRs
Oliver Schwab: An example for theory-based evaluation - ERDF Berlin
Tecla Livi: Confronting housing poverty: the role of Social Agencies for Housing