Impact evaluations assess the degree to which a particular policy, program or policy has affected the well-being of individuals. They can play a tremendous role in making effective policy and necessitate a larger discussion in the context of efforts to improve development effectiveness. On January 15 to 16 2008 the conference “Making Smart Policy: Using Impact Evaluation for Policy Making” was held at the World Bank Headquarters in Washington, D.C. The conference was organized by the Poverty Reduction and Economic Management (PREM), Independent Evaluation Group (IEG), and Development Economics (DEC) networks of the World Bank, with co-sponsorship by DFID and the Government of the Netherlands.
The session entitled “The Role of Impact Evaluation in Development Agencies” was held on January 15th, 2008 and was chaired by Nick York, Evaluation Department Head at the United Kingdom Department for International Development (DFID). The panel members included Phil Davis, Researcher at the American Institute for Research, Owen Barder Director of Global Development Effectiveness in the DFID, Martin Ravallion, Director in the Development Research Group (DECRG) at the World Bank, and Abhijit Sen, member of the Planning Commission in the Government of India.
Davis focused on the role of Impact Evaluation in relation to other types of evaluations. He asked whether Impact Evaluation is just a technical exercise performed due to interest in the results, or whether it sits within a wider set of systems and systemic requirements which allows work to be done more effectively. Davis offered a textbook definition of Impact Evaluations, describing it as “a family of research methods” and stressing that it is not simply a cost benefit analysis, random allocation trials, or regression continuity analysis. He stated that while these are important; Impact Evaluation also encompasses qualitative work, theories of change and understanding the dynamics and sizes of problems. He drew the distinction between two types of Impact Evaluation: summative and process evaluation. Davis underscored that it is necessary to perform both types of analysis. Davis finished his presentation by presenting five questions that he believes should be considered in the Impact Evaluation process.
Ravallion delivered the second presentation. He discussed ways to make Impact Evaluations more useful and more relevant to development policy. He noted that there is usually investment in evaluations when there is absence of strong institutional support. Ravallion presented ten steps that he believed could make Impact Evaluations more effective, the first of which is to begin with a policy relevant question. Ravallion also emphasized that ethical objections and political sensitivities must be taken seriously. He discussed what he called the deeper set of questions around the term “central heterogeneity.”
Barder spoke next, offering the policy maker‘s point of view. He specified what policy makers want and need from Impact Evaluations. Barder articulated four things for which Impact Evaluations should be used: 1) choosing which kind of intervention to recommend; 2) designing it in a way that provides the most impact; 3) managing the organization’s performance, and; 4) communicating to people what is being done and why they should continue funding it. Barder emphasized the importance of being able to show that the work is actually producing an impact. He also noted that policy makers are not simply interested in the narrow group of people policy is being designed for, but how the whole system is affected by it. Barder reiterated that in every spending or policy decision it is necessary to be as explicit as possible about the resulting impact. He commented that while policy makers want to know when policy will be effective, they also want to know how certain this efficacy is. Barder’s concluding comments recognized Impact Evaluation as being at the heart of unlocking the results agenda, which in itself is the key to unlocking the whole aid effectiveness agenda. For donors to behave more rationally, Barder stated, it is necessary to provide them with results that encourage them to invest in what really works.
In the question and answer session, audience members inquired about moving from learning pilots to funding of large projects, and why the Bank hasn’t made use of research and data previously done by other global corporations.
Home
> Learning
& Knowledge > BSPAN
> Presentation
High Speed
Low Speed
