Indicators: What are they and how do you recognise a good one?

Data, Monitoring & Evaluation

“In 2020, Arqaam pored through thousands of indicators in its efforts to develop the Logframe Lab tool, a web application that helps organisations build their project’s logframe. Logframe Lab’s algorithms match a project idea with the most relevant metrics, drawing from an extensive database of standardised indicators.”

 

In the development sphere, donors, Non-Governmental Organisations (NGOs), and beneficiaries need to communicate the value and benefits of a project. When the two talk with each other about these benefits, it’s helpful to have a standardised set of terminology, so that each one understands exactly what the other means. In addition, having this set of understood terms helps to compare approaches across countries and organisations.

 

In the business world, companies follow Key Performance Indicators to track their progress towards goals on a regular basis. In development work, organisations and institutions track indicators. Indicators are measurable factors you can use to objectively track your progress and impact for a project and its ultimate success.

 

Small community organisations who are new to the development arena may find the subject of indicators to be overwhelming – after all, there are thousands of indicators to choose from, depending on the topic of your work, the type of action you are taking, and which funders you are working with. As you might imagine, choosing the appropriate indicators from among thousands can be a daunting task.

 

In 2020, Arqaam pored through thousands of indicators in its efforts to develop the Logframe Lab tool, a web application that helps organisations build their project’s logframe. Logframe Lab’s algorithms match a project idea with the most relevant metrics, drawing from an extensive database of standardised indicators.

 

In this article, we will explain why development actors need to pay close attention to the indicators they select for their projects and how they can learn to recognise the difference between a good indicator and a bad one. We will also offer a short overview of the main organisations that have defined their own lists of indicators and what differentiates them.

 

The Importance of Indicators for Your Development Project

 

NGOs and community-based organisations will need to become both familiar and comfortable with indicators, specifically when it comes to using logical frameworks (also referred to as logframes or result frameworks) for monitoring and evaluating a project. The indicators you select will be used to determine your project’s impact.

 

When using logframes for monitoring, in a best-case scenario, project planners will have defined an impact that they want to achieve – such as ‘Have drinkable water in X country’. They then go down a logical results chain, defining outcomes that would support this impact. One outcome could be a certain number of water filtration sites, while another could be working with the places that are polluters in order to decrease the amount of pollutants going into the water. From these outcomes, the project planners then develop outputs, which are basically ‘sub-goals’ that help to achieve the outcomes. Then, from the outputs, the activities are created. The main difference between outputs and outcomes is that outputs are controllable by the project, they are in charge of it, while outcomes are the changes to be expected from these controllable actions.

 

Once this entire results chain is done – and only at this point – do the planners start to develop indicators that are adequate for measuring what the project aims to accomplish. If the project is related to the environment, indicators might include the level of pollutants in the water supply of a vulnerable village. If the project is health related, indicators might include the percentage of children who have been immunised against measles.

 

Finally, the project planners will set the goals for the project, which should be decided upon based on their budget. For the environmental project, the goal could be a reduction of pollutants in the water by 80%. For the health project, the goal could be an increase in the percentage of immunised children to 90%. In addition, it is essential to always have a baseline. The baseline is the status quo before a project starts. It is important to have a baseline to understand if, in the end, the project made the situation better, even when it does not reach its target.

 

Types of Indicators

Indicators can be either quantitative or qualitative. As you might guess, quantitative indicators are ones that you can count. For example, ‘The proportion of children who are malnourished’, ‘The total level of nitrogen in well water’, or ‘The number of journalists jailed per year’. Qualitative indicators are less common and would include statements such as ‘Stable border relations’ or ‘Empowerment of land owners’.

 

Indicators are categorised into three types, based on the levels in a logical framework (aka, logframe): Outputs, Outcomes, and Impact.

 

Output indicators measure project outputs. For example, for an educational project, an output indicator might be ‘The number of adult literacy programs set up in rural towns in the country’. Outcome indicators measure the medium-term impacts of a project – meaning its outcomes. For the educational project, this might be ‘The percentage of working-age adults in rural towns who can read and write at a functional level’.

 

Impact indicators look at the more long-term and broader effects of the project. An example of an impact indicator for the educational project might be ‘The level of unemployment in the country’. The data for indicators on the impact level should (in their best form) not be collected by the project, but come from national statistics or surveys conducted by others.

 

How to Recognise a Good Indicator

 

What is the difference between a good and bad indicator? How can you tell whether an indicator is good or not? The quality of the indicators you select will be crucial to your ability to monitor and evaluate the success of your project. At Arqaam, we recommend you think RACER when choosing indicators: Relevant, Accepted, Credible, Easy, and Robust.

 

Relevant – Your indicator should be closely linked with the results that you want to reach.

 

Accepted – It should be accepted by your staff, stakeholders, and other users.

 

Credible – The indicator should be accessible to non-experts, unambiguous, and easy to interpret.

 

Easy – It should be feasible to monitor and collect data for this indicator at a reasonable cost.

 

Robust – The indicator should not be easily to manipulate.

 

Here are some other good rules of thumb to keep in mind when choosing indicators for your development project:

 

  • Indicators should not include targets. These should be in a different column of your logframe.
  • Indicators should only measure one value. You should hardly ever use ‘and/or’ in them.
  • Indicators should be disaggregated where possible, into groups that are mutually exclusive.
  • Indicators should be reliable, in that the indicator can be measured the same way each time, regardless of who is doing the measurement.
  • Indicators should always be neutral. They should never use words such as ‘increase’ or ‘decrease’.

 

It is important to always respect local laws and customs – as well as data privacy and informed consent – when you are thinking of indicators and how you will collect data and from whom.

 

A Look at the Main Indicator Lists

 

One of the most well-known lists of indicators is from the United Nations (UN), and it’s related to the institution’s 17 Sustainable Development Goals (SDG), which were adopted by all UN Member States in 2015. Any development project related to these goals can select from a broad range of indicators. The UN’s SDG indicator database is categorised by goal, and then further broken down by targets within each goal. For example, UN SDG Goal #3 is ‘Ensure healthy lives and promote well-being for all at all ages’ and includes 13 targets. For target 3.1, ‘By 2030, reduce the global maternal mortality ratio to less than 70 per 100,000 live births’, there are two different indicators listed. One is ‘Maternal mortality ratio’, and the other is ‘Proportion of births attended by skilled health personnel’.

 

The World Development Indicators, maintained by the World Bank, doesn’t just list important indicators, it also provides data for the indicators, with some data going back as far as 50 years. The indicators are separated into six ‘Data Themes’, such as ‘Poverty and Inequality’, ‘Environment’, ‘Economy’, and ‘States and Markets’.

 

In addition to the above, many funding institutions have lists of indicators they focus on, so that development actors seeking project funding can better tailor their proposals. For instance, the European Commission maintains lists of indicators specific to its many funds, which include the European Regional Development Fund, the European Agricultural Fund for Rural Development, and the European Maritime and Fisheries Fund, among others. Another example is the Council of Europe, which maintains an ‘Indicator Framework on Culture and Democracy’, which assesses how culture contributes to democracy and ‘the economic efficiency of financing culture’.

 

Logframe Lab: Bringing the Indicators Together in One Place

The benefit of these publicly available indicator lists is that there can be some standardisation among countries, regions, and projects. There are already so many indicators among these lists, that no development actor ever needs to spend time and effort to create new ones from scratch – there is very likely an indicator that already exists for their purpose.

 

Arqaam’s Logframe Lab brings the indicators into one handy tool. When you upload your project proposal, it helps you find the most adequate indicators based on the words you have been using to describe your project idea. The indicators in our database include the ones currently used by official organizations such as the UN, the EU, the World Bank, and other sources who have shared them online. We also continue to update and expand our database with indicators of supporters.

 

If you want to share indicators that your organisation is using, or that you feel are missing from our database, head over to the website. There you can download an Excel form where you can enter your indicators, and then submit them for review.

Read more from the Arqaam blog HERE.