Qualitative and Quantitative
data collection methods in M&E
June 17, 2021
Data is the heart and soul of monitoring and evaluation (M&E). Valid, reliable and accurate data can reveal and improve the performance and impact of your intervention and support decision making and learning, while enhancing your credibility and accountability. Data is divided into different categories based on how you source them and the techniques you employ to gather and analyse them.
In this article, we will explore:
- Qualitative data collection approach
- Quantitative data collection approach
- Key differences between qualitative and quantitative data collection approach and how they are used in M&E.
- Some common qualitative data collection tools and methods and quantitative data collection tools and methods to help you choose the ones that best fit your project needs.
- Plus, why a mixed-method approach might be your best option.
Primary and secondary data
All data are categorized as either ‘primary data’ or ‘secondary data,’ based on how you source them. Primary data are those that you and your team collect directly from the main sources, whereas, secondary data are those that were collected by other organisations, government agencies, or independent research institutions and individuals and are available for use. Secondary data could be censuses, surveys, organizational records or other previous research, extracted from books, journals, reports, newspapers, magazines, data archives, databases etc.
Once you are ready to collect your data, you will have to decide upon the data collection tools and methods. This will depend on a number of things, including the purpose of the data, the local context, cost, timeline, availability of skills and resources, and most importantly, the indicators and key questions you have identified and how the collected data will be utilized.
All data are further divided into two broad categories based on the techniques employed in the field to gather and analyse them – ‘qualitative data’ and ‘quantitative data.’ Stay with us as we walk you through each approach and explain their key differences.
Qualitative data collection approach
Qualitative data collection plays an important role in monitoring and evaluation as it helps you delve deeper into a particular problem and gain a human perspective on it. It provides in depth information on some of the more intangible factors like experiences, opinions, motivations, behaviours or descriptions of a process, event or a particular context relevant to your project. So, in other words, a qualitative approach uses people’s stories, experiences and feelings to measure change.
Compared to a quantitative approach, a qualitative approach is more open, informal and unstructured or semi-structured, and it provides more flexibility in how data is collected. Qualitative research is investigative in nature and the data collected through this process answers the question ‘why’ or ‘how’ – how do people feel about a situation, or why are health care facilities underutilized? This approach relies more heavily on interactive interviews, discussions and deeper conversations. While using this approach, many researchers also use triangulation or mixed methods to increase the credibility and authenticity of their findings. Data is often recorded in the form of field notes, sketches, audiotapes, photographs and other suitable means.
Usually the findings drawn from qualitative research are not generalizable to any specific population, rather each case study produces a unique piece of evidence that can help identify patterns among different studies of the same issue. The results produced from this approach can be subjective and as such can be subject to bias in their interpretation. Analyzing such data can also be quite complex and time-consuming which can make it an expensive process.
Quantitative data collection approach
The quantitative approach uses numbers and statistics to quantify change and is often expressed in the form of digits, units, ratios, percentages, proportions, etc. Compared to the qualitative approach, the quantitative approach is more structured, straightforward and formal. Quantitative approach is utilized to derive answers to the questions ‘how much’ or ‘how many’ – how many people attended the workshop or how often do people visit the health center.
Quantitative research is useful for multi-site and cluster evaluations that involve a large group of respondents or sample population. This approach relies heavily on random sampling and structured data collection instruments that fit diverse experiences into predetermined response categories. Typical quantitative data gathering strategies include, experiments or clinical trials, gathering relevant data from management information systems, administering surveys with closed-ended questions or observing and recording well-defined events.
Because quantitative methods are not about gaining an in-depth understanding but rather grasping a general understanding of a particular context with precise results, quantitative data is easier to collect and analyse and there are less chances of bias in the result interpretation. Results are numerical, objective, conclusive and to the point, so the results are easier to summarize and generalize and are useful for making comparisons across different sites or interventions.
Differences between qualitative and quantitative data collection approach
Qualitative and quantitative data collection methods
Below, we have summarized key data collection methods and tools used in monitoring and evaluation (M&E). Most methods and tools can be used in combination with other tools and methods and are applicable in both qualitative and quantitative research. However, this list is not complete, as tools and techniques continually evolve and new tools and techniques keep emerging in M&E.
The list is adapted from ‘Project/Programme Monitoring and Evaluation (M&E) Guide from the International Federation of Red Cross and Red Crescent Societies (IFRC), 2011.’
The most common qualitative data collection methods and tools
- Open-Ended Surveys: allow for a systematic collection of information from a defined population, usually by means of interviews or questionnaires administered to a sample of units in the population. Qualitative surveys include a set of open-ended questions that aim to gather information from people about their characteristics, knowledge, attitudes, values, behaviours, experiences and opinions on relevant topics. Surveys can be collected via pen/paper forms or digitally via online/offline data collection apps.
- Open ended interviews: are useful when you want an in-depth understanding of experiences, opinions or individual descriptions of a process. Can be done individually or in groups. In groups, you will ask fewer questions than in an individual interview since everyone has to have the opportunity to answer and there are limits to how long people are willing to sit still. In-person interviews can be longer and more in-depth.
- Community interviews/meeting: is a form of public meeting open to all community members. Interaction is between the participants and the interviewer, who moderates the meeting and asks questions following a prepared interview guide. This is ideal for interacting with and gathering insights from a big group of people.
- Focus group discussions (FGDs): is ideal when you want to interview a small group of people (6-12 individuals) to informally discuss specific topics relevant to the issues being examined. A moderator introduces the topic and uses a prepared interview guide to lead the discussion and extract insights, opinions and reactions but s/he can improvise with probes or additional questions as warranted by the situation. The composition of people in an FGD depends upon the purpose of the research, some are homogenous, others diverse. FGDs tend to elicit more information than individual interviews because people express different views, beliefs and opinions and engage in a dialogue with one another.
- Case study: is an in-depth analysis of individuals, organisations, events, projects, communities, time periods or a story. As it involves data collection from multiple sources, a case study is particularly useful in evaluating complex situations and exploring qualitative impact. A case study can also be combined with other case studies or methods to illustrate findings and comparisons. They are usually presented in written forms, but can also be presented as photographs, films or videos.
- Observation: It is a good technique for collecting data on behavioural patterns, physical surroundings, activities and processes as it entails recording what observers see and hear at a specified site. An observation guide is often used to look for consistent criteria, behaviours, or patterns. Observations can be obtrusive or unobtrusive. It is ‘obtrusive’ when observations are made with the participant’s knowledge and ‘unobtrusive’ when observations are done without the knowledge of the participant.
- Ethnography: Ethnographic research involves observing and studying research topics in a specific geographic location to understand cultures, behaviors, trends, patterns and problems in a natural setting. Geographic location can range from a small entity to a big country. Researchers must spend a considerable amount of time, usually several weeks or months, with the group being studied to interact with them as a participant in their community. This makes it a time-consuming and challenging research method and cannot be limited to a specific period.
- Visual techniques: in this method, participants are prompted to construct visual responses to questions posed by the interviewers, the visual content can be maps, diagrams, calendars, timelines and other visual displays to examine the study topics. This technique is especially effective where verbal methods can be problematic due to low-literate or mixed-language target populations, or in situations where the desired information is not easily expressed in either words or numbers.
- Literature review and document review: is a review of secondary data which can be either qualitative or quantitative in nature e.g. project records and reports, administrative databases, training materials, correspondence, legislation and policy documents, as well as videos, electronic data or photos that are relevant to your project. This technique can provide cost-effective and timely baseline information and a historical perspective of the project or intervention.
- Oral histories: it’s the process of establishing historical information by interviewing a select group of informants and drawing on their memories of the past. Oral history strives to obtain interesting and provoking historic information from different perspectives, most of which cannot be found in written sources. The insights from oral history can be discussed, debated, and utilized in numerous capacities.
The most common quantitative data collection methods and tools
- A structured closed-ended interview: this type of interview systematically follows carefully organised questions that only allow a limited range of answers, such as “yes/no” or expressed by a rating/number on a scale. For quantitative interviews to be effective, each question must be asked the same way to each respondent, with little to no input from the interviewer.
- Closed ended surveys and questionnaires: is an ideal choice when you want simple, quick feedback which can easily translate into statistics for analysis. In quantitative research, surveys are structured questionnaires with a limited number of closed-ended questions and rating scales used to generate numerical data or data that can be separated under ‘yes’ or ‘no’ categories. These can be collected and analysed quickly using statistics such as percentages.
- Experimental research: is guided by hypotheses that state an expected relationship between two or more variables, so an experiment is conducted to support or disconfirm this experimental hypothesis. Usually, one set of variables is manipulated (treatment group) and applied to the other set of dependent variables (control group) to measure their effect on the latter. The effect of the independent variables on the dependent variables is observed and recorded to draw a reasonable conclusion regarding the relationship between the two groups. This research is mainly used in natural sciences.
- Correlational research: is a non-experimental research that studies the relationship between two or more variables that are similar and interdependent and assesses their statistical relationship – how one variable affects the other and vice versa but with no influence from any extraneous variable. It uses mathematical analysis to analyse collected data and the results are presented in a diagram or generated in statistics.
- Causal-comparative: also known as quasi-experimental research, compares two variables that are not related. Variables are not manipulated. One variable is dependent and the other independent. Variables not randomly assigned.
- Statistical data review: entails a review of population censuses, research studies and other sources of statistical data.
- Laboratory testing: are precise measurement of a specific objective phenomenon, e.g. infant weight or water quality test.
Why a mixed method approach might be your best option for data collection
Each data collection tool and method has its own advantages but development projects are complex and their intricate dynamics cannot be disentangled through one method or one data collection tool alone. Therefore, mixing qualitative and quantitative methods and using different data collection techniques is recommended as it could add value to the monitoring and evaluation of your development projects.
Using a combination of quantitative and qualitative methods, which is often called a mixed-method approach, enables researchers to gain a more holistic understanding of the intervention and why it is or it isn’t manifesting the expected outcomes. It also addresses the shortcomings and limitations of each method to provide more coherent, reliable and useful conclusions and increases the overall confidence in the validity of the evaluation results.
Using mixed methods helps to capture a wider range of perspectives and in some cases, one method can be used to help guide the use of another method, or to explain the findings from that method. You can measure what happened with quantitative data and examine how and why it happened with qualitative data. Qualitative methods also help to uncover issues during the early stages of an intervention that can then be further investigated using quantitative methods, or quantitative methods can highlight particular issues that can be examined in-depth with qualitative methods. Some may point out that mixed methods could be costly and time-consuming but research shows that the benefits almost always far exceed the costs.
Want to learn more about mixed-method approaches? Check out this World Bank’s document on “Combining Quantitative and Qualitative Methods for Program Monitoring and Evaluation: Why Are Mixed Method Designs Best?”
We hope you found our article on qualitative and quantitative data collection methods helpful. As you see, both methods have their own advantages and disadvantages but when used in a balanced combination, they can really provide reliable evidence to unfold the progress and shortcomings of your development projects and help you make data-driven decisions for timely improvement and enhancement. However, the choices of methods depend on the nature of your project. But one thing to keep in mind, no matter which approach you may choose to utilize, it is important to observe the ethical principles of research for all data collection methods and tools.
For more on the ethics of data collection, check out this short report by INTRAC – Principles of Data Collection.
By Chandani Lopez Peralta, Content Marketing Manager at TolaData.
This is very important to me. I will be starting with my research project soon. It will be more helpful indeed.
Happy to hear that you found our article helpful, Makgabo!
useful document thanks
You are most welcome, Leila!
Great article. Thanks so much for sharing this !!
You are most welcome, Chris!
Excellent insight. I found your article quiet helpful even as a Seminarian.
Great Article on Monitoring and Evaluation, this is really helpful when you conducting surveys/research
I got something to your article God bless you