1. INTRODUCTION TO THE TOOLKIT

   • About the Toolkit
   • Toolkit Contents
   • Using the Toolkit
   • Case Study Background

2. INTRODUCTION TO EVALUATION

   • Defining Evaluation
   • Purpose of Evaluation
   • Types of Evaluation
   • Challenges to Conducting an Evaluation

   • Who Should Conduct the Evaluation

     • Assessing Readiness
     • Using a Team Approach
     • Using External Evaluators

   • Considerations for Planning

     • Human Subjects Protections
     • Cultural Competency
     • Budgeting for Evaluation
     • Working with Funders

   • Framework for Evaluation

     • Framework Components
     • Evaluation Standards

3. PLAN YOUR EVALUATION

   • Step 1: Engage Stakeholders

     • Engage Stakeholders
     • Identifying Key Stakeholders
     • Engaging Stakeholders
     • Tools and Resources

   • Step 2: Develop a Logic Model

     • Develop a Logic Model
     • What is a Program Description?
     • Introducing Logic Models
     • Creating a Logic Model for Evaluation
     • Tools and Resources

   • Step 3: Focus the Evaluation

     • Focus the Evaluation
     • Decide What to Evaluate
     • Develop Evaluation Questions
     • What Are Indicators?
     • Start the Evaluation Plan
     • Tools and Resources

   • Step 4a: Plan Data Collection

     • Plan Data Collection
     • Identify Data Sources
     • Select Data Collection Methods
     • Tools and Resources

   • Finalize Your Evaluation Plan

     • Identify the Person Responsible and a Timeline
     • Review Your Evaluation Plan
     • Tools and Resources

4. IMPLEMENT YOUR EVALUATION

   • Introduction
   • Review Your Evaluation Plan

   • Step 4b: Collect Data

     • Design Data Collection Instruments
     • Additional Considerations in Data Collection
     • Tools and Resources

   • Step 5: Analyze and Interpret Data

     • Analyze and Interpret Data
     • Organize the Data
     • Analyze the Data and Synthesize Results
     • Interpret the Results
     • Identify Limitations
     • Tools and Resources

   • Step 6: Share Results

     • Share Results
     • Develop Recommendations
     • Communicate the Findings
     • Follow-up
     • Tools and Resources

Primary Versus Secondary Data

The difference between primary and secondary data is related to the purpose for which the data were collected. Data you collected yourself (or that was collected on your behalf) specifically for the purpose at hand are primary data. Data that already exist are considered secondary data; this includes data already published or collected in the past by other parties, as well as data previously collected by your organization for other purposes. One thing to keep in mind is that data originally collected for one purpose (primary data analysis) could be useful for another purpose down the road (a secondary analysis).

Depending on your evaluation questions and indicators, some of the data you need may be available from an existing source. Whether or not to use secondary data depends on how well they meet your organization's and the evaluation's needs. Using secondary data may be less costly and time-consuming than collecting your own data, but it might not have the same level of utility. Secondary data should only be used when appropriate to the intended purpose of your evaluation.

Many local, state and national organizations collect and publish data on health indicators. Examples include data from various CDC surveillance systems, such as the Behavioral Risk Factor Surveillance System (BRFSS) or the Youth Risk Behavior Survey (YRBS); US Census files; cancer registries; and state governmental statistics sources. (See sidebar for a link to the ERC's list of Georgia-specific secondary data sources.)

When considering using secondary data, ask yourself:

• Will data be specific enough to represent the geographic area or population we want to study? If you're only serving a small portion of the population, it's probably not realistic to expect broad-level population data to show changes in outcomes due to your program.

• Do the questions included specifically address what we want to measure? Are we trying to measure the same or similar things?

• Will data be available within the timeframe we need, and cover the appropriate time period?

ERC Worksheet 5: Evaluation Plan, continued
EXERCISE: Continuing to work on Worksheet 5, identify one or more data source(s) for each of the indicators in your evaluation plan. As you identify each source, ask yourself if it will provide information that you need and can trust, and how easy or difficult it will be to get data from that source.

Click on the PDF documents in the sidebar to see examples of how this step was completed for our case study sites.