1. INTRODUCTION TO THE TOOLKIT

   • About the Toolkit
   • Toolkit Contents
   • Using the Toolkit
   • Case Study Background

2. INTRODUCTION TO EVALUATION

   • Defining Evaluation
   • Purpose of Evaluation
   • Types of Evaluation
   • Challenges to Conducting an Evaluation

   • Who Should Conduct the Evaluation

     • Assessing Readiness
     • Using a Team Approach
     • Using External Evaluators

   • Considerations for Planning

     • Human Subjects Protections
     • Cultural Competency
     • Budgeting for Evaluation
     • Working with Funders

   • Framework for Evaluation

     • Framework Components
     • Evaluation Standards

3. PLAN YOUR EVALUATION

   • Step 1: Engage Stakeholders

     • Engage Stakeholders
     • Identifying Key Stakeholders
     • Engaging Stakeholders
     • Tools and Resources

   • Step 2: Develop a Logic Model

     • Develop a Logic Model
     • What is a Program Description?
     • Introducing Logic Models
     • Creating a Logic Model for Evaluation
     • Tools and Resources

   • Step 3: Focus the Evaluation

     • Focus the Evaluation
     • Decide What to Evaluate
     • Develop Evaluation Questions
     • What Are Indicators?
     • Start the Evaluation Plan
     • Tools and Resources

   • Step 4a: Plan Data Collection

     • Plan Data Collection
     • Identify Data Sources
     • Select Data Collection Methods
     • Tools and Resources

   • Finalize Your Evaluation Plan

     • Identify the Person Responsible and a Timeline
     • Review Your Evaluation Plan
     • Tools and Resources

4. IMPLEMENT YOUR EVALUATION

   • Introduction
   • Review Your Evaluation Plan

   • Step 4b: Collect Data

     • Design Data Collection Instruments
     • Additional Considerations in Data Collection
     • Tools and Resources

   • Step 5: Analyze and Interpret Data

     • Analyze and Interpret Data
     • Organize the Data
     • Analyze the Data and Synthesize Results
     • Interpret the Results
     • Identify Limitations
     • Tools and Resources

   • Step 6: Share Results

     • Share Results
     • Develop Recommendations
     • Communicate the Findings
     • Follow-up
     • Tools and Resources

Pilot Testing

Once you've developed your data collection tool(s), you'll want to pilot test your instrument(s) before you begin formal data collection activities. Pilot testing helps to:

• Gauge how long the interview, survey, or focus group will take.

• Identify issues that may need to be addressed when training data collectors, such as asking the questions in a standardized way.

• Ensure instructions and questions are clear to the respondents.

You do not need to pilot test your instruments with a large number of people, but your pilot testers should be representative of your sample (or at least familiar with the perspective of the respondents) so that you are getting an accurate sense of how someone will actually respond to your questions. Typically, you will ask pilot testers to respond to your questions and then provide you with feedback about their experience (e.g., clarity, appropriateness of length). You may obtain feedback in a variety of ways, such as by having respondents make marks on a written copy of a survey (e.g., indicating unclear questions, missing response options, etc.), or discussing the instrument after they have gone through it. If there are certain items that you are particularly concerned about, be sure to flag these ahead of time so you can ask your pilot testers about these specific questions, too.

Once you've completed pilot testing and have made any necessary adjustments to your instrument(s), you are ready to begin data collection. Do not use the data you collected in the pilot test unless the respondents are part of your sample and you have not significantly changed the data collection instrument.