Selecting and Training Data Collectors
This section addresses considerations for selecting and training data collectors.
Training data collectors
For any data collection methodology — from document review to focus groups — it is important to train the person(s) who will be doing the collection to ensure activities are being performed consistently. The depth of the training will depend on the collectors' level of experience. At a minimum, everyone must understand the data collection protocol (i.e., the introductory script, the questions being asked, what you need to learn, any follow-up that will be needed, and what will be done with the information) so that they can adequately collect data and answer any questions that respondents may have. You also need to make sure they know how to record the information and what to do with completed responses, notes/transcripts or checklists.
When more than one person is involved in data collection, training will need to ensure that they will carry out the data collection activities consistently — often this means conducting mock data collection exercises for training purposes, or initially doubling up on data collection initially to ensure consistency (e.g., having two people observe the same event, then comparing how they recorded things in their observation checklists). If data collection is happening multiple times or over an extended period of time, it is helpful to debrief after the first experience to resolve any issues that have come up with either the protocol itself or how it is administered. Ongoing monitoring and/or refresher trainings may be needed to ensure consistency if implementing long-term data collection efforts.
Selecting appropriate data collectors
Even more than training, it is important to select the appropriate people to do data collection. Different methodologies have different considerations, described below.
Document review and media tracking
For document review and media tracking, it is very helpful to have someone who is familiar with the material or content is detail oriented, and who will be able to accurately and completely capture the data you need. Depending on the type of documentation reviewed, you many need someone with specialized skills and experience (e.g., familiarity with medical terminology for chart review).
Survey
For surveys, who you choose will depend on how the survey is being administered. If the survey is sent by mail or email, you need to have a contact person who can answer technical questions about it. If the survey is to be conducted anonymously, having a known person or entity distribute or introduce it can help to increase response rate. For example, if you are doing a community survey, you may have a city or county leader send a letter with the survey to introduce it and ask for participation. If program participants are being surveyed, then it should be introduced or sent by the person they have worked with. However, if the survey includes information that could identify individual respondents, it may be important for it to come from an unknown entity so that people feel like they can be candid.
If a paper survey is conducted in person — for example, distributed at the end of an event — generally it will be distributed by the person who has been working with the respondents. If multiple people are to distribute the survey, you must ensure that the same introduction and directions will be provided by each. Training will be very important to make sure the survey is administered consistently.
If an oral or telephone survey is being conducted, it will ideally be done by a neutral person(s) — someone unknown to the respondents.
Observation
Observation, though guided by a checklist, can be the most complicated data collection method in which to train observers. This is because generally they are recording not only what people are saying, but other (potentially intangible) dynamics that may influence the discussion (e.g., the environment, emotions, level of engagement).
An observer can be either a participant or a neutral observer. While the objectivity of an outside observer can be valuable, a participant is familiar with the program and may be able to offer more insight about the group. It is difficult for any one person to capture everything going on in a room, so it is beneficial to have more than one observer working. This will help ensure the observation is complete and individual bias is minimal. Again, when more than one observer is involved, training will be important to make sure observers have a common understanding of the checklist items and are capturing the same data.
Interviews
As discussed earlier, interviews can be done in-person or by phone. When selecting interviewers, you typically want someone neutral so that participants feel comfortable giving candid responses. However, if the interview is not directly tied to your program or organization, you may be able to use program staff—for example, to conduct interviews with community leaders about local needs your program could address. The interviewer should be familiar with the subject matter, but should influence or drive respondents' comments — they must be able to listen without inserting their own reactions or opinions. A good interviewer will be able to facilitate the conversation and get through the important questions during the requested amount of time.
Focus GroupsFocus groups
Of all of the data collection methodologies, focus groups are the most difficult to pull off effectively. When choosing a person(s) to lead a focus group, it is best to have a trained facilitator – someone who understand the structure of a focus group and knows what you are hoping to learn (see Quick Tips for Focus Groups) and what you are hoping to learn. Also, typically you want to select a neutral person — someone unknown to the participants—but someone who can relate to them – who understands their backgrounds and experiences well enough to ask probing questions appropriately.
It is important to avoid any power differentials when assembling a focus group — both among participants and between the facilitator and the participants. The facilitator needs to be able to create a comfortable environment, which is why it is important to choose someone skilled at facilitating group process — like how to respectfully get back on topic or how to engage everyone in the discussion without putting people on the spot.
The facilitator recaps the key points at the end of the discussion to confirm what participants said. It is important to remember that the facilitator's job is not to provide information or opinions, but to get participants to talk. If you would like someone on your staff to be trained as a focus group facilitator, the best way is to shadow an experienced facilitator and to debrief after the session.
Pilot Testing
