Choosing Methods — Advantages and Disadvantages
Once you have explored the different options for collecting data, the next step is to choose which will be the most appropriate for your indicators and evaluation questions.
Consider the following as you make your decisions:
-
Advantages and disadvantages of each method. Every method has its advantages and disadvantages (many of which were discussed in the preceding section), whether it's the type of information you can collect or the time and cost of collecting it. The attached PDF summarizes the advantages and disadvantages of each of the methods described above.
-
Information best suited for the job. In selecting your method(s), it will be helpful to know what type of information will resonate with your stakeholders (e.g., numbers, percentages, comparisons, stories, examples, pictures), as well as what type of information will most effectively show the progress or impact of your program.
-
Available resources. Be sure to choose methods that are feasible to implement—consider the time, resources and expertise you have available to carry out data collection efforts. Consider what data you are already collecting or have access to, and if they could be used to answer your evaluation questions.
-
Level of intrusiveness. You'll want to minimize any disruption to participants from your data collection efforts, as well as protect participants in the evaluation. For example, will your data collection take a lot of participants' time away from the program itself? Will you need to ensure the confidentiality of participants' responses?
-
Cultural appropriateness. Keep your target audience in mind as you select collection methods—their age, culture, language, literacy level, phone/computer access, and so on. For example, in some communities, having people tell stories in person may be a more effective and respectful way to collect data than administering a questionnaire.
-
Use of mixed methods. Evaluations that use more than one method provide stronger evidence because they gather data from different sources. Consider the feasibility of using multiple methods in your evaluation and determine which ones would provide complementary information.
Referring back to the CDC's Framework for Program Evaluation described in the Introduction to Evaluation section can help you select appropriate methods, especially in terms of:
- Utility:
- Will the data sources and collection methods serve the information needs of your primary users/stakeholders?
- Feasibility:
- Are your sources and methods practical and efficient?
- Do you have the needed capacity, time and resources?
- Are your methods non-intrusive and non-disruptive?
- Propriety:
- Are your methods respectful, legal, ethical, and appropriate?
- Does your approach protect and respect the welfare of all involved or affected?
- Accuracy:
- Are your methods technically adequate to:
- Answer your questions?
- Measure what you intend to measure?
- Reveal credible and trustworthy information?
- Convey important information?
- Are your methods technically adequate to:
ERC Worksheet 5: Evaluation Plan, continued
EXERCISE: Go back to Worksheet 5 and determine which data collection methods will be most appropriate for collecting information from your data sources. You may want to refer to Data Collection Methods: Advantages and Disadvantages table for an overview of common methods.
Click on the PDF documents in the sidebar to see examples of how this step was completed for our case study sites.
Methods 
Case Study
