Toolkit
  1. INTRODUCTION TO THE TOOLKIT

  2. INTRODUCTION TO EVALUATION

  3. PLAN YOUR EVALUATION

  4. IMPLEMENT YOUR EVALUATION

Brainstorm Indicators

While indicators themselves are specific and measurable, it's important to recognize that there are often many different potential indicators that could be used to answer your evaluation questions.

Looking back at one of the sample indicators above, the score that students receive on standardized tests was identified as an indicator of academic achievement. But is this the only indicator of academic achievement, or are there others that could be used? How else might "academic achievement" be defined? Depending upon the program's goals and target population, other possible indicators might include:

  • The percentage of 3rd graders reading at a 3rd grade level

  • The percentage of students who achieve at least 2.0 GPA by the end of the year

  • The percentage of students who graduate from high school

  • A reduction in the gap between test scores for different racial/ethnic groups

While these might all be indicators of academic achievement, brainstorming indicators requires you think about your unique program and how it relates to your evaluation questions.

To illustrate this process, let's say the program works with elementary school students who are not fluent in English. The program includes before-school language sessions, afterschool tutoring, and activities to engage non-English-speaking parents in supporting their children's academic progress. One of the evaluation questions identified was "Is the program increasing students' academic achievement?" To come up with a list of possible indicators, it's helpful to think about the program in terms of broad questions that relate to the evaluation question. The brainstorming process might look like this:

  • What academic skills do the children work on in these activities?
    [English reading fluency, English language conversation practice, one-on-one tutoring focused on schoolwork and academic language]

  • How much improvement is expected? What level of change would the program consider a success?
    [Increase of at least one grade level in reading; and increase of at least 10 points on standardized math and language tests]

  • If you asked parents, what would they tell you about the program that would make you think it was successful?
    [Parents would tell you they are more involved in helping their children with their homework and that they feel comfortable contacting their child's teacher if they have questions]

  • What would teachers say about the program's contribution to students' academic success?
    [Teachers would say children are finishing their homework assignments, children are more likely to speak up in class, children's grades and test scores are increasing, and parents are more likely to contact them with questions]

Based on this brainstorming process, the indicators could then be:

  • Students' increase of at least one grade level in reading

  • Students' increase in both math and language standardized test scores

  • Parents report increased involvement in helping children with their homework

  • Teachers report increased child participation in classroom and homework

ERC Worksheet 4: Developing Indicators
EXERCISE: In the first column of Worksheet 4, record the evaluation questions you identified as most important in Worksheet 3. Then, brainstorm possible indicators for each evaluation question. Try not to be overly critical as you brainstorm, as the next step reviews criteria you will use for selecting indicators.

Click on the PDF documents in the box to see examples of how this step was completed for our case study sites.