Toolkit
  1. INTRODUCTION TO THE TOOLKIT

  2. INTRODUCTION TO EVALUATION

  3. PLAN YOUR EVALUATION

  4. IMPLEMENT YOUR EVALUATION

Qualitative Analysis

Qualitative data analysis involves carefully reviewing responses to open-ended survey questions, interview or focus group transcripts, and observation notes to identify the key points as well as patterns or themes that emerge. There are a number of software programs to assist in this process, but they can be costly and require specific training. You can use much simpler methods to aggregate and analyze qualitative data, like creating tables in Word or even hand coding paper copies of responses.

Interviews, open-ended survey questions and observation notes
An easy method is to create a table that allows you to sort responses, either by individual interviewed or by individual question. Assign each person interviewed an identification number (ID #) and each question a question number (Q #). The following table could be used to record the responses to four questions the person with ID #01 was asked during an interview. Key words are used to represent each question that was asked.

ID # Q # Question (key words) Response
01 1 What you expected  
01 2 What you experienced  
01 3 Satisfaction  
01 4 How could be improved  

Following the interview, type the individual's responses into the cell in the last column that corresponds to the question. Tables from multiple interviews can then be copied and pasted together (aggregated) into a single table in one document.

The "sort" feature in Word allows you to sort data in the aggregate table by either:

  • Column 1 (ID #) to look at all responses given by a single person interviewed, or
  • Column 2 (Q #) to look at all responses to a single question.

Here is how the aggregate table would look with three interviews (ID #10, 11, 12) sorted by question.

ID # Q # Question (key words) Response
10 1 What you expected  
11 1 What you expected  
12 1 What you expected  
10 2 What you experienced  
11 2 What you experienced  
12 2 What you experienced  
10 3 Satisfaction  
11 3 Satisfaction  
12 3 Satisfaction  
10 4 How could be improved  
11 4 How could be improved  
12 4 How could be improved  

The color "highlight" feature in Word is useful for flagging key points or coding themes from responses across all interviews. The "find" feature (Edit—>Find) is also helpful, as it allows you to search for key words or phrases that may recur.

Focus groups
If the data come from focus groups, it can be overwhelming to analyze the large volume of words and thoughts gathered during the session. Let the words speak for themselves—that is, listen carefully and try to avoid hearing just what you want to hear as you review the discussion.

A typed transcript and/or typed notes from the focus group discussion will make getting results easier and faster. A common method of analysis is to first organize the transcript by making comments related to each focus group question. Next, review the transcript and code (highlight or mark with a symbol) individual comments according to key concepts or phrases. Then review the comments coded similarly and interpret their meaning by asking:

  • What themes emerged—how often something was mentioned and by how many people?

  • What was the intensity—how much feeling or emotion was expressed in the comments?

  • What were the "big ideas"—what suspicions/beliefs were confirmed and what did I learn that I didn't know before?

  • What results were unanticipated?

If multiple focus groups were conducted, data from each should be analyzed separately and then the results should be merged to draw final conclusions.

Other suggestions
For all kinds of qualitative data, it is best to have more than one person do it, to ensure that findings are reliable (i.e., another team member analyzing the same data would reach essentially the same conclusion) and valid (i.e., findings are reflective of what respondents actually meant).

Also, flag any quotes that illustrate themes and could be used (anonymously) in reports or presentations to stakeholders.

Reporting
Qualitative data are usually synthesized and displayed as a written summary. This may be a series of paragraphs or a bulleted list of key points or quotes organized around the major themes. As with quantitative data, organizing the results in a clear, understandable format makes them easier to review and interpret.

ERC Worksheet 6: Data Analysis and Interpretation
Once you have determined which data collections methods you will use and have developed protocols for collecting the information, you want to think about how you will organize, analyze, and interpret the data. For each of your data collection protocols/instruments, complete ERC Worksheet 6, which will help you stick to asking only the most important, useful types of questions.

Click on the PDF documents in the sidebar to see examples of how this step was completed for our case study sites.

Northwest Center for Public Health Practice (2011). Data collection for program evaluation. This online course provides more in-depth information about analyzing qualitative data obtained from focus groups and interviews.