Impact evaluations assess the degree to which a particular policy, program or policy has affected the well-being of individuals. They can play a tremendous role in making effective policy and necessitate a larger discussion in the context of efforts to improve development effectiveness. On January 15 to 16 2008 the conference “Making Smart Policy: Using Impact Evaluation for Policy Making” was held at the World Bank Headquarters in Washington, D.C. The conference was organized by the Poverty Reduction and Economic Management (PREM), Independent Evaluation Group (IEG), and Development Economics (DEC) networks of the World Bank, with co-sponsorship by DFID and the Government of the Netherlands.
The session entitled “The Role of Impact Evaluation in National Policy”, was held on January 16th and drew on three country perspectives from Colombia, China, and Mexico to discuss the role of impact evaluation in national policy. The session was chaired by Maureen Lewis, Acting Chief Economist of HDN at the World Bank.
The first panelist, Bertha Briceño, Head of the Directorate for Evaluation of Public Policies in the Colombian National Planning Department, presented Colombia’s experiences with impact evaluations. She outlined the trajectory that begun in the late 1990s in response to national and international demands for increased evaluations on aid effectiveness and accountability, which ultimately led in 2000 to the creation of SYNERGIA, Colombia’s national evaluation system. Briceño noted the fundamental role political support plays in creating successful evaluation structures like SYNERGIA, which has been highly supported by President Uribe, in contrast to earlier models whose monitoring indicators were easily bypassed by an unsupportive political environment. Amongst gains achieved as a result of impact evaluations, Briceño included increased funding for programs that exhibit positive results like Familias en Accion and the ability to pin point initiatives to be expanded or eliminated based on data. Challenges included budget limitations and quality constraints, which Briceño demonstrated continue to improve as lessons in programs through impact evaluations are shared within Colombia and across Latin America.
Xiulan Zhang, Director of the Institute of Social Development at Beijing Normal University, presented China’s experiences with impact evaluations, which have played a large role in understanding the recent slowdown in declining absolute poverty levels despite increased funding for poverty reduction policies and programs in China. Zhang identified developments such as the rapidly changing socio-political and economic context and chronic poverty as key causes of the post-2000 slowdown. She presented various recommendations and highlighted constraints faced by impact evaluations, such as limited government demand, funding and access to information. Despite the challenges, Zhang highlighted through various case studies how demand for impact evaluations have continued to increase, and offered recommendations to ensure future increased effectiveness and demand for impact evaluations.
Gonzalo Hernández Licona, Executive Secretary of CONEVAL, Mexico’s National Council for Evaluation of Social Policies, spoke about the significant improvements Mexico has made in the past 10 years regarding the evaluation of social programs through five case studies. He attributed the improvements to increased governmental transparency, the success of past evaluations, the supportive role of academia and other institutional changes. Through the case studies, Licona highlighted the fundamental role impact evaluations play in deciding whether to expand, terminate or adjust certain elements of social programs. He noted that challenges included ensuring financial and political support, building local technical evaluation capacities, moving towards evaluating policies and building results-based monitoring.
In the question and answer session, questions included the effects of negative impact evaluation results, the role of legal frameworks on successful impact evaluations, and the use of criteria to select programs for impact evaluation, among others.