PROJECT RESULTS MATTER
And those results - the outputs, outcomes and impact - need to be monitored and ultimately measured.
This Isn't Just To Keep Donors Happy - Although We'd All Like That.
And those results - the outputs, outcomes and impact - need to be monitored and ultimately measured.
This Isn't Just To Keep Donors Happy - Although We'd All Like That.
But project monitoring:
And evaluation allows us to:
- Lets management and important stakeholders understand how much progress we are making
- Means we can make decisions in good time to keep the project on track
- Gives us a basis for evaluation and learning
And evaluation allows us to:
- Compare actual results with those planned or expected
- Understand whether the projects objectives were achieved and relevant
- Measure the project's efficiency, effectiveness, impact and sustainability
SO WHAT ARE INDICATORS? AND WHICH TYPE SHOULD WE USE?
According to OECD, an indicator is:
According to OECD, an indicator is:
"Quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor."
In short, they tell us how well we are doing. They describe progress towards and achievement of project events at output, outcome and impact levels.
TYPES OF INDICATOR
Let's start with the two main types - quantitative and qualitative. What's the difference? Which is better? I'm asked that a lot.
QUANTITATIVE INDICATORS
Now, quantitative indicators are factual - they give us a number, ratio, percentage or similar. How many girls enrolled in school? What percentage of the population has access to family planning services? What is the ratio of health workers per 100,000 people?
And we like those. They can be counted, the source of data is often easy to verify, they can be compared over time and compared in different places, too. They should also be free from bias - whoever collects the data, the result should be the same.
If you can use a quantitative indicator, go ahead. So they're usually easier.
And even when the source of data isn't directly available - or the cost of data collection is too high - we can often use a proxy. For example, an increase in mobile phone sales can indirectly tell us that incomes are increasing. (On its own that wouldn't be enough, though - you might need others just in case it's phones that are getting cheaper!)
Or if you wanted to measure the quality of improvements in the justice system - observing court cases is both time consuming and subjective - you could never measure it economically or reliably. But you could track the number of cases being challenged. If that falls, it indicates more people are satisfied with court decisions.
So when do we use quantitative indicators? When we have a quantifiable result we want to measure.
Let's start with the two main types - quantitative and qualitative. What's the difference? Which is better? I'm asked that a lot.
QUANTITATIVE INDICATORS
Now, quantitative indicators are factual - they give us a number, ratio, percentage or similar. How many girls enrolled in school? What percentage of the population has access to family planning services? What is the ratio of health workers per 100,000 people?
And we like those. They can be counted, the source of data is often easy to verify, they can be compared over time and compared in different places, too. They should also be free from bias - whoever collects the data, the result should be the same.
If you can use a quantitative indicator, go ahead. So they're usually easier.
And even when the source of data isn't directly available - or the cost of data collection is too high - we can often use a proxy. For example, an increase in mobile phone sales can indirectly tell us that incomes are increasing. (On its own that wouldn't be enough, though - you might need others just in case it's phones that are getting cheaper!)
Or if you wanted to measure the quality of improvements in the justice system - observing court cases is both time consuming and subjective - you could never measure it economically or reliably. But you could track the number of cases being challenged. If that falls, it indicates more people are satisfied with court decisions.
So when do we use quantitative indicators? When we have a quantifiable result we want to measure.
Keep Your Projects On Track
Monitor & Evaluate Your Project Results With The RIGHT Methods
|
Measure Progress Reliably, Connect With Your Stakeholders And
Save Time Lost through Wasted Efforts
Enter Your Name & Email Address To Get Your Free Guide Now
|
FREE 'HOW TO' GUIDE
|
QUALITATIVE INDICATORS
However, not everything we want to measure is so straightforward - and some things are better measured using qualitative indicators.
Let's take, for example, a project that wants to increase girls' completion of secondary education in Afghanistan. Among its aims - alongside establishing girl-friendly schools and ensuring safety - it needs to address parents' attitudes towards the importance of educating girls beyond primary level. Now that's qualitative - it looks at awareness, beliefs and attitudes.
Now, the project's outcome is a behaviour change, so we can't use that as the indicator. So what will tell us their attitude has changed?
In any social change, we are trying to apply scientific tools to non-scientific work - and when we explore any qualitative aspect we are trying to make it measurable and, to an extent, make it quantifiable.
In this particular case, we realised that while we were bound to use community meetings to communicate with stakeholders, deliver awareness-related services and gather information, results might not be too reliable. (Community pressure was a big factor to consider.)
Because of their private nature, we opted to use a survey/questionnaire at intervals to gauge changes in attitude using agree-type questions. A range of statements were presented to participants such as 'There is no point sending girls to secondary school' or 'I would be proud if my daughter became a doctor' and respondents placed a mark along the scale to indicate how much they agreed with the statement.
Over time we managed to:
SO WHICH ONE SHOULD I USE?
Neither is better than the other - but quantitative indicators are easier to measure. However, qualitative results needs a qualitative indicator, and quantitative results needs a quantitative indicator. And at higher levels, we will often need both.
So look at what you are planning to measure. Can you count it and ascribe it a number? Then plan to measure it quantitatively, and, where possible, directly.
Are you looking at things like attitudes, perceptions, reliance or resilience? Then you need a qualitative indicator and method to collect the data.
However, not everything we want to measure is so straightforward - and some things are better measured using qualitative indicators.
Let's take, for example, a project that wants to increase girls' completion of secondary education in Afghanistan. Among its aims - alongside establishing girl-friendly schools and ensuring safety - it needs to address parents' attitudes towards the importance of educating girls beyond primary level. Now that's qualitative - it looks at awareness, beliefs and attitudes.
Now, the project's outcome is a behaviour change, so we can't use that as the indicator. So what will tell us their attitude has changed?
In any social change, we are trying to apply scientific tools to non-scientific work - and when we explore any qualitative aspect we are trying to make it measurable and, to an extent, make it quantifiable.
In this particular case, we realised that while we were bound to use community meetings to communicate with stakeholders, deliver awareness-related services and gather information, results might not be too reliable. (Community pressure was a big factor to consider.)
Because of their private nature, we opted to use a survey/questionnaire at intervals to gauge changes in attitude using agree-type questions. A range of statements were presented to participants such as 'There is no point sending girls to secondary school' or 'I would be proud if my daughter became a doctor' and respondents placed a mark along the scale to indicate how much they agreed with the statement.
Over time we managed to:
- Track particular respondents changes in attitude over time
- Aggregate results to measure the overall community shifts in attitude
SO WHICH ONE SHOULD I USE?
Neither is better than the other - but quantitative indicators are easier to measure. However, qualitative results needs a qualitative indicator, and quantitative results needs a quantitative indicator. And at higher levels, we will often need both.
So look at what you are planning to measure. Can you count it and ascribe it a number? Then plan to measure it quantitatively, and, where possible, directly.
Are you looking at things like attitudes, perceptions, reliance or resilience? Then you need a qualitative indicator and method to collect the data.
