CONTENTS -- Preface -- PART ONE. INTRODUCTION TO IMPACT EVALUATION -- Chapter 1. Why Evaluate? -- Evidence-Based Policy Making -- What Is Impact Evaluation? -- Impact Evaluation for Policy Decisions -- Deciding Whether to Evaluate -- Cost-Effectiveness Analysis -- Prospective versus Retrospective Evaluation -- Efficacy Studies and Effectiveness Studies -- Combining Sources of Information to Assess Both the "What" and the "Why" -- Notes -- References -- Chapter 2. Determining Evaluation Questions -- Types of Evaluation Questions -- Theories of Change -- The Results Chain -- Hypotheses for the Evaluation -- Selecting Performance Indicators -- Road Map to Parts 2 and 3 -- Note -- References -- PART TWO. HOW TO EVALUATE -- Chapter 3. Causal Inference and Counterfactuals -- Causal Inference -- Estimating the Counterfactual -- Two Counterfeit Estimates of the Counterfactual -- Notes -- Chapter 4. Randomized Selection Methods -- Randomized Assignment of the Treatment -- Two Variations on Randomized Assignment -- Estimating Impact under Randomized Offering -- Notes -- References -- Chapter 5. Regression Discontinuity Design -- Case 1: Subsidies for Fertilizer in Rice Production -- Case 2: Cash Transfers -- Using the Regression Discontinuity Design Method to Evaluate the Health Insurance Subsidy Program -- The RDD Method at Work -- Limitations and Interpretation of the Regression Discontinuity Design Method -- Note -- References -- Chapter 6. Difference-in-Differences -- How Is the Difference-in-Differences Method Helpful? -- Using Difference-in-Differences to Evaluate the Health Insurance Subsidy Program -- The Difference-in-Differences Method at Work -- Limitations of the Difference-in-Differences Method -- Notes -- References -- Chapter 7. Matching.

Using Matching Techniques to Select Participant and Nonparticipant Households in the Health Insurance Subsidy Program -- The Matching Method at Work -- Limitations of the Matching Method -- Notes -- References -- Chapter 8. Combining Methods -- Combining Methods -- Imperfect Compliance -- Spillovers -- Additional Considerations -- A Backup Plan for Your Evaluation -- Note -- References -- Chapter 9. Evaluating Multifaceted Programs -- Evaluating Programs with Different Treatment Levels -- Evaluating Multiple Treatments with Crossover Designs -- Note -- References -- PART THREE. HOW TO IMPLEMENT AN IMPACT EVALUATION -- Chapter 10. Operationalizing the Impact Evaluation Design -- Choosing an Impact Evaluation Method -- Is the Evaluation Ethical? -- How to Set Up an Evaluation Team? -- How to Time the Evaluation? -- How to Budget for an Evaluation? -- Notes -- References -- Chapter 11. Choosing the Sample -- What Kinds of Data Do I Need? -- Power Calculations: How Big a Sample Do I Need? -- Deciding on the Sampling Strategy -- Notes -- References -- Chapter 12. Collecting Data -- Hiring Help to Collect Data -- Developing the Questionnaire -- Testing the Questionnaire -- Conducting Fieldwork -- Processing and Validating the Data -- Note -- References -- Chapter 13. Producing and Disseminating Findings -- What Products Will the Evaluation Deliver? -- How to Disseminate Findings? -- Notes -- References -- Chapter 14. Conclusion -- Note -- References -- Glossary -- Index -- Boxes -- 1.1 Evaluations and Political Sustainability: The Progresa/Oportunidades Conditional Cash Transfer Program in Mexico -- 1.2 Evaluating to Improve Resource Allocations: Family Planning and Fertility in Indonesia -- 1.3 Evaluating to Improve Program Design: Malnourishment and Cognitive Development in Colombia.

1.4 Evaluating Cost-Effectiveness: Comparing Strategies to Increase School Attendance in Kenya -- 2.1 Theory of Change: From Cement Floors to Happiness in Mexico -- 3.1 Estimating the Counterfactual: Miss Unique and the Cash Transfer Program -- 4.1 Conditional Cash Transfers and Education in Mexico -- 4.2 Randomized Offering of School Vouchers in Colombia -- 4.3 Promoting Education Infrastructure Investments in Bolivia -- 5.1 Social Assistance and Labor Supply in Canada -- 5.2 School Fees and Enrollment Rates in Colombia -- 5.3 Social Safety Nets Based on a Poverty Index in Jamaica -- 6.1 Water Privatization and Infant Mortality in Argentina -- 7.1 Workfare Program and Incomes in Argentina -- 7.2 Piped Water and Child Health in India -- 8.1 Checklist of Verification and Falsification Tests -- 8.2 Matched Difference-in-Differences: Cement Floors, Child Health, and Maternal Happiness in Mexico -- 8.3 Working with Spillovers: Deworming, Externalities, and Education in Kenya -- 9.1 Testing Program Alternatives for HIV/AIDS Prevention in Kenya -- 9.2 Testing Program Alternatives for Monitoring Corruption in Indonesia -- 10.1 Cash Transfer Programs and the Minimum Scale of Intervention -- 12.1 Data Collection for the Evaluation of the Nicaraguan Atención a Crisis Pilots -- 13.1 Outline of an Impact Evaluation Plan -- 13.2 Outline of a Baseline Report -- 13.3 Outline of an Evaluation Report -- 13.4 Disseminating Evaluation Findings to Improve Policy -- Figures -- 2.1 What Is a Results Chain? -- 2.2 Results Chain for a High School Mathematics Program -- 3.1 The Perfect Clone -- 3.2 A Valid Comparison Group -- 3.3 Before and After Estimates of a Microfinance Program -- 4.1 Characteristics of Groups under Randomized Assignment of Treatment -- 4.2 Random Sampling and Randomized Assignment of Treatment -- 4.3 Steps in Randomized Assignment to Treatment.

4.4 Randomized Assignment to Treatment Using a Spreadsheet -- 4.5 Estimating Impact under Randomized Assignment -- 4.6 Randomized Offering of a Program -- 4.7 Estimating the Impact of Treatment on the Treated under Randomized Offering -- 4.8 Randomized Promotion -- 4.9 Estimating Impact under Randomized Promotion -- 5.1 Rice Yield -- 5.2 Household Expenditures in Relation to Poverty (Preintervention) -- 5.3 A Discontinuity in Eligibility for the Cash Transfer Program -- 5.4 Household Expenditures in Relation to Poverty (Postintervention) -- 5.5 Poverty Index and Health Expenditures at the Health Insurance Subsidy Program Baseline -- 5.6 Poverty Index and Health Expenditures-Health Insurance Subsidy Program Two Years Later -- 6.1 Difference-in-Differences -- 6.2 Difference-in-Differences when Outcome Trends Differ -- 7.1 Exact Matching on Four Characteristics -- 7.2 Propensity Score Matching and Common Support -- 8.1 Spillovers -- 9.1 Steps in Randomized Assignment of Two Levels of Treatment -- 9.2 Steps in Randomized Assignment of Two Interventions -- 9.3 Treatment and Comparison Groups for a Program with Two Interventions -- P3.1 Roadmap for Implementing an Impact Evaluation -- 11.1 A Large Sample Will Better Resemble the Population -- 11.2 A Valid Sampling Frame Covers the Entire Population of Interest -- 14.1 Number of Impact Evaluations at the World Bank by Region, 2004-10 -- Tables -- 2.1 Elements of a Monitoring and Evaluation Plan -- 3.1 Case 1-HISP Impact Using Before-After (Comparison of Means) -- 3.2 Case 1-HISP Impact Using Before-After (Regression Analysis) -- 3.3 Case 2-HISP Impact Using Enrolled-Nonenrolled (Comparison of Means) -- 3.4 Case 2-HISP Impact Using Enrolled-Nonenrolled (Regression Analysis) -- 4.1 Case 3-Balance between Treatment and Comparison Villages at Baseline.

4.2 Case 3-HISP Impact Using Randomized Assignment (Comparison of Means) -- 4.3 Case 3-HISP Impact Using Randomized Assignment (Regression Analysis) -- 4.4 Case 4-HISP Impact Using Randomized Promotion (Comparison of Means) -- 4.5 Case 4-HISP Impact Using Randomized Promotion (Regression Analysis) -- 5.1 Case 5-HISP Impact Using Regression Discontinuity Design (Regression Analysis) -- 6.1 The Difference-in-Differences Method -- 6.2 Case 6-HISP Impact Using Difference-in-Differences (Comparison of Means) -- 6.3 Case 6-HISP Impact Using Difference-in-Differences (Regression Analysis) -- 7.1 Estimating the Propensity Score Based on Observed Characteristics -- 7.2 Case 7-HISP Impact Using Matching (Comparison of Means) -- 7.3 Case 7-HISP Impact Using Matching (Regression Analysis) -- 10.1 Relationship between a Program's Operational Rules and Impact Evaluation Methods -- 10.2 Cost of Impact Evaluations of a Selection of World Bank-Supported Projects -- 10.3 Disaggregated Costs of a Selection of World Bank-Supported Projects -- 10.4 Work Sheet for Impact Evaluation Cost Estimation -- 10.5 Sample Impact Evaluation Budget -- 11.1 Examples of Clusters -- 11.2 Sample Size Required for Various Minimum Detectable Effects (Decrease in Household Health Expenditures), Power = 0.9, No Clustering -- 11.3 Sample Size Required for Various Minimum Detectable Effects (Decrease in Household Health Expenditures), Power = 0.8, No Clustering -- 11.4 Sample Size Required to Detect Various Minimum Desired Effects (Increase in Hospitalization Rate), Power = 0.9, No Clustering -- 11.5 Sample Size Required for Various Minimum Detectable Effects (Decrease in Household Health Expenditures), Power = 0.9, Maximum of 100 Clusters -- 11.6 Sample Size Required for Various Minimum Detectable Effects (Decrease in Household Health Expenditures), Power = 0.8, Maximum of 100 Clusters.

11.7 Sample Size Required to Detect a 2 Minimum Impact for Various Numbers of Clusters, Power = 0.9.