Evaluating AIDS Prevention Programs -- Copyright -- Preface -- Acknowledgments -- NOTE ON CONTRIBUTIONS -- Contents -- Summary -- BACKGROUND -- EVALUATION: NEEDS AND IMPLEMENTATION -- OUTCOMES -- THE MEDIA CAMPAIGN -- COMMUNITY-BASED ORGANIZATIONS -- HIV TESTING AND COUNSELING -- RANDOMIZED AND OBSERVATIONAL APPROACHES TO EVALUATION -- SPECIFIC RECOMMENDATIONS -- All AIDS Intervention Programs -- National AIDS Media Campaign -- Community-Based Organizations -- Testing and Counseling -- 1 Design and Implementation of Evaluation Research -- TYPES OF EVALUATION -- EVALUATION RESEARCH DESIGN -- Process Evaluation Designs -- Outcome Evaluation Designs -- Nonexperimental and Quasi-Experimental Designs -- Randomized Experiments -- Rationale -- Pitfalls -- Unit of Assignment -- Choice of Methods -- THE MANAGEMENT OF EVALUATION -- Project Selection -- Research Administration -- Conducting the Research -- Independent Oversight -- Agency In-House Team -- Interagency Collaboration -- Costs of Evaluation -- REFERENCES -- 2 Measurement of Outcomes -- PROGRAM OBJECTIVES -- OUTCOMES FOR EVALUATIONS OF HIV PREVENTION PROGRAMS -- Biological Outcomes -- Strengths and Weaknesses of Biological Outcomes -- Behavioral Outcomes -- Primary Prevention Behaviors -- Risk Reduction -- Protective Behaviors -- Complementary Prevention Behaviors -- Strengths and Weaknesses of Behavioral Outcomes -- Psychological Outcomes -- Strengths and Weaknesses of Psychological Outcomes -- EVALUATION MEASURES -- Timing of Measurement -- Quality of Measures -- REFERENCES -- 3 Evaluating Media Campaigns -- BACKGROUND AND OBJECTIVES -- FORMATIVE EVALUATION: WHAT WORKS BETTER? -- Step 1: Idea Generation -- Step 2: Concept Testing -- Step 3: The Positioning Statement -- Step 4: Copy Testing -- Methodological Issues -- Resources and Aspirations.

EFFICACY TRIALS: CAN THE CAMPAIGN MAKE A DIFFERENCE? -- Randomized Experiments -- Methodological Issues -- Resources and Aspirations -- PROCESS EVALUATION: WHAT IS ACTUALLY DELIVERED? -- Methodological Issues -- Resources and Aspirations -- OUTCOME EVALUATION: DOES THE CAMPAIGN MAKE A DIFFERENCE? -- Randomized Experiments -- The National Health Interview Survey -- Hotline Calls -- Other Archival Sources -- Methodological Issues -- Effectiveness Evaluation of Current Activities -- Effectiveness Evaluation of Future Activities -- Problems with Sources of Data -- The NHIS and Other Surveys -- Hotline Calls and Other Archival Data -- Resources and Aspirations -- REFERENCES -- 4 Evaluating Health Education and Risk Reduction Projects -- BACKGROUND AND OBJECTIVES -- WHAT SERVICES ARE DELIVERED? -- Case Studies of a Sample of Projects -- Sample -- Data Collection -- Analysis -- Standardized Administrative Reporting -- A Census or Sample Survey -- Recommended Combination of Strategies -- Methodological Issues -- Resources and Aspirations -- DO THE PROJECTS MAKE A DIFFERENCE? -- Before-and-After Evaluation Designs -- Randomized Field Studies -- Methodological Issues -- Resources and Aspirations -- WHAT WORKS BETTER? -- REFERENCES -- 5 Evaluating HIV Testing and Counseling Projects -- BACKGROUND AND OBJECTIVES -- HOW WELL ARE SERVICES DELIVERED? -- A Site Services Inventory -- Client Surveys -- Population Surveys -- Case Studies Using Direct Observation -- Methodological Issues -- Resources and Aspirations -- OPTIONS FOR EVALUATING WHETHER HIV TESTING AND COUNSELING SERVICES MAKE A DIFFERENCE -- WHAT WORKS BETTER? -- Randomized Experiments of Alternative Treatments -- Unit of Assignment -- Experimental Regimens -- Service Delivery Setting -- Content, Duration, and Intensity of Counseling -- Additional Services -- Methodological Issues.

Resources and Aspirations -- REFERENCES -- 6 Randomized and Observational Approaches to Evaluating the Effectiveness of AIDS Prevention Programs -- OVERVIEW -- Choosing Among Strategies -- RANDOMIZED EXPERIMENTATION -- The Power of Experiments: An Example -- Compromised Randomization -- Attrition -- Compliance -- Spillover -- Compensatory Behavior -- Salvaging Compromised Experiments -- When Should Randomized Experiments Be Considered? -- Is a Randomized Experiment Appropriate? -- What Is Being Asked? -- Timeliness -- Is It Feasible? -- Affordability -- Resources for Evaluation -- Acceptability -- Logistics of Randomized Assignment -- DESIGNING COMPARABILITY INTO NONRANDOMIZED STUDIES -- Quasi-Experiments -- Interrupted Time Series -- Assumptions -- Data Needs -- Inferences -- Regression Discontinuity or Regression Displacement -- Assumptions. -- Data Needs -- Inferences -- Existing Data Sources for Use in Quasi-Experimental Designs -- The Neonatal Screening Survey. -- The National Health Interview Survey -- Natural Experiments -- Identifying Natural Experiments -- Assumptions of Natural Experiments -- Data Needs of Natural Experiments -- Matching Without Randomization -- Prospective Nonrandomized Matching -- Retrospective Nonrandomized Matching -- Assumptions -- Data Needs -- Inferences -- Existing Data Sources for Matching Without Randomization -- Cohorts of Gay Men -- Cohorts of Intravenous Drug Users -- MODELING AND STATISTICAL ADJUSTMENTS FOR BIAS -- Analysis of Covariance -- Assumptions -- Data Needs -- Inferences -- Structural Equation and Selection Models -- Structural Equation Models -- Selection Models -- Selection Models and Natural Experiments -- Selection Modeling and Historical Controls -- Assumptions of Modeling -- Data Needs of Models -- Inferences from Modeling -- The Role of Models.

WHEN SHOULD NONRANDOMIZED APPROACHES BE CONSIDERED? -- INTERPRETING EVALUATION RESULTS -- Randomized Experiments -- Nonrandomized Methods -- Accessibility of Assumptions -- Interpretation -- REFERENCES -- Appendixes -- A Collaborative Contracting Strategy -- References -- B Oversight and Coordination Strategy -- The Project Review Team -- Operations -- Summary and Discussion -- References -- C Methodological Issues in AIDS Surveys -- Introduction -- Fallibility of Measurement in Other Sciences -- Recruitment of Respondents in Sex and Seroprevalence Surveys -- Scope of the Review -- Participation in Sex Surveys -- Data Collection Procedures and Response Rates -- Survey Configurations Associated with High Response Rates -- Use of Telephone Surveys -- Participation in Seroprevalence Surveys -- Nonresponse Bias in Sex and Seroprevalence Surveys -- Nonsampling Issues in Aids Surveys -- Terms and Concepts -- Survey Measurement of Sexual Behaviors -- Overview -- Inference in the Presence of Bias -- Assumption of Constant Bias in Measurements -- Approaches to Validation -- Validation Using STD Rates -- Psychometric Approaches to Validity -- Empirical Studies of Sexual Behaviors -- Validation -- Partner Reports -- Other Validation Techniques -- Replication of Surveys on Samples of the Same Population -- Proportion of Teenagers Who Are Sexually Active -- Number of Sexual Partners Reported by Adults -- Replication of Measurements Using Same Respondents -- Empirical Studies of Drug-Using Behaviors -- Accuracy of Self-Reports of Drug Use Behaviors -- Measurement Bias -- Summary of Findings -- Feasibility -- Replicability -- Validity -- Reliability -- Improving Validity and Reliability -- Literacy -- Alternatives to Self-Reports -- Physical Evidence -- Skills Demonstrations -- Other Safeguards for Surveys -- Randomized Response Techniques -- Pilot Studies.

Pretests -- Cognitive Research Strategies -- Ethnographic Studies -- Examples of Studies Related to HIV Transmission -- Male-Male Sexual Contacts -- Variation in Drug Use Patterns -- Ethnographic Methods -- Ethnographic Methods in AIDS Research -- Findings of Ethnographic Research on AIDS -- Gaps and Deficiencies in Current Ethnographic Research -- Recommendations -- References -- D Sampling and Randomization: Technical Questions about Evaluating CDC's Three Major AIDS Prevention Programs -- Sampling Issues -- Number of Case Studies -- Estimating Sample Sizes -- Controlling Attrition -- Confidentiality Guarantees -- Compensation -- Stabilization Funds -- Cultivating and Tracking Respondents -- Personnel for Tracking Respondents -- Modeling Attrition -- Convenience and Probability Sampling -- Sample Studies of Gay and Bisexual Men -- Sample Studies of Intravenous Drug Users -- Randomization -- Examples of Randomized Experiments -- The Ethics of No-treatment Controls -- References -- E Ancillary, Emerging, and Related Projects -- Evaluating Clearinghouse Materials -- Background and Objectives -- Does It Work? -- What Works Better? -- Evaluating Referral Services of the Counseling and Testing Program -- Monitoring Services at Non-CDC Testing Sites -- References -- F The Use of Selection Modeling to Evaluate AIDS Interventions with Observational Data -- I. Introduction -- II. Historical Development of Econometric Methods for Program Evaluation -- III. The Statistics of Program Evaluation with Observational Data -- The Problem -- Solutions -- Solution 1: Identifying Variables ("Z's") -- Solution 2: Parametric Distributional Assumptions on -- Solution 3: Availability of Cohort Data -- The Relationship between Data Availability and Testing of Assumptions -- IV. Application to Aids Interventions -- The Search for Z's -- Collection of Histories.

V. Summary and Conclusions.