Transforming Monitoring, Evaluation, and Learning in the Water Sector: Lessons from the MEL Roundtable

Donors and practitioners in India’s water sector participated in a roundtable discussion on systematising monitoring, evaluation and learning. Photo credit: Shawaz Malik.

In 2023-24, about ₹1,200 crore rupees ($150 million) was spent on water security and conservation initiatives by corporate social responsibility (CSR) programmes. Such interventions play an integral role in creating a water-secure India. Not all of them are effective and scalable though, due to the absence of robust evidence.

This is where monitoring, evaluation and learning (MEL) exercises come in. Monitoring involves collecting data and information before, during and after interventions. Evaluation is the examination of an intervention’s impact based on an analysis of the monitoring data. Learning is about continuously understanding and improving how to meet desired outcomes more effectively. Such an exercise can guide corrective action for the betterment of solutions.

A shared MEL framework in the water sector has the potential to provide common standards and a language for communicating impact. Initial analysis by WELL Labs and the Environmental Defense Fund (EDF) tells us that MEL practices are currently not uniform, and are seen as a donor requirement. They are focussed more on planning and ex-post evaluations can be rudimentary. Inputs and outputs are measured more often instead of outcomes. In some instances, multiple organisations may have multiple concurrent initiatives in the same watershed, but their impact is rarely analysed together.

WELL Labs and the Environmental Defense Fund (EDF) convened a roundtable on Systematising Monitoring, Learning, and Evaluation (MEL) in the Water Sector in New Delhi on September 16, 2024. It brought together 40 experts from nonprofits, academia, consultancies, and donor organisations (see list of participating organisations below). The aim of the roundtable was to:

1. Identify gaps and challenges in current MEL practices.
2. Learn from existing impact evaluation approaches and tools.
3. Encourage collaboration to drive effective solutions.

Experts from the water sector at a roundtable discussion on the future of monitoring, evaluation and learning (MEL) in New Delhi. Photo credit: Shawaz Malik.

Four key insights emerged from discussions at the MEL roundtable.

1. Communities and people need to be central to evaluating water security interventions.

People are deeply impacted due to changes in water resources, and need to be equipped with the tools and knowledge to manage their water. To ensure that evaluations keep communities at the forefront, we should:

Create ethical frameworks: Monitoring and evaluation processes involve the collection of significant amounts of data, information and knowledge from beneficiaries, which is why we need an empathetic and ethical framework. The collection of private data should involve consent and respect for privacy, and not be onerous. The on-ground support that local civil society organisations provide to these assessments should not go unrewarded. The process should involve a two-way feedback and knowledge transfer mechanism with the beneficiaries.

Account for socio-economic changes: In water security interventions, common positive impacts often look like an increase in farmer incomes, an improvement in water productivity, and a reduction in loss due to droughts and floods. However, there can be more nuanced and far-reaching impacts, like families reuniting because of reduced distress migration, decreased societal violence, or improving the number of years of school education for girls. Participants concluded that common frameworks of MEL need to capture the full impact on lives and livelihoods.

2. Impact evaluation can be technically complex and expensive; we need to simplify it.

Impact evaluation requires in-depth knowledge of hydrology, hydrogeology, systems transformation, economics and more. However, data collection is often conducted by field staff who may not be equipped to do this accurately or swiftly. This technical gap is also a reason why many organisations don’t work with large hydrological datasets. Thus, organisations need to develop easy-to-use tools and solutions to improve MEL.

Another challenge is that interventions may last for a short duration (less than a year), and the cost of a high-quality MEL exercise could be more than that of the intervention itself. MEL exercises, therefore, must be proportional to the scale and nature of interventions.

In addition, organisations should consider the trade-offs between accuracy and effort. Take the example of building check dams to improve groundwater recharge. The easiest metric to measure would be the storage potential of the check dam. However, the effective recharge is harder to measure as it involves factors like rate of water infiltration, shape of the structure, extent of siltation, and amount of evaporation. Measuring and estimating all parameters can be expensive and beyond the scope of some interventions.

3. The learning component of MEL needs to be a priority rather than an afterthought.

Impact evaluation is often seen as a regulatory requirement that needs to be fulfilled or a fault-finding exercise. MEL’s objective should be to improve and learn from interventions rather than act as a score-keeping exercise.

4. Impact evaluation should be flexible and responsive.

Factors like a sudden drought, a new government policy, or adverse market conditions may push organisations to pivot and adapt their interventions. In such cases, the metrics and indicators used for MEL also need to change. A rigid set of pre-defined indicators will not adequately capture impacts in such cases.

To chart a way forward for comprehensive impact evaluations, WELL Labs is conducting three MEL studies in collaboration with partners.

These studies represent the multi-dimensional nature of water conservation programmes in India — one attempts to manage the demand for water, another aims to improve the availability, and the third aims to improve the planning processes around water management.

1. Supply side: Assessing the effectiveness of JalTara (infiltration pits) by Save Groundwater Foundation in Jalna, Maharashtra
2. Demand side: Assessing the groundwater collectivization programme by WASSAN in Anantapur, Andhra Pradesh
3. Planning and process: Evaluating tools and trainings by the Foundation for Ecological Security (FES) for MGNREGA planning in Karnataka

WELL Labs and EDF are also developing a toolbox to address the above-mentioned challenges and systematise the evaluation of water interventions in India.

It is akin to a directory of existing tools and methodologies and offers a generalised protocol to approach MEL in the water sector in the Indian context. Through the toolbox, we seek to make MEL assessments more robust, affordable, and accessible for different kinds of organisations.

This article by EDF looks at the role of MEL in creating data-driven solutions for India’s groundwater crisis.

List of Roundtable Participants

• Donors: ATE Chandra Foundation, Arghyam, Bill & Melinda Gates Foundation (BMGF), Hindustan Unilever Foundation (HUF), International Development Research Centre (IDRC), Tata Consumer Products
• Civil Society Organisations: Foundation for Ecological Security (FES), Tarun Bharat Sangh (TBS), Watershed Support Services and Activities Network (WASSAN)
• Academia: ICT for Social Development, and School of Public Policy, both at the Indian Institute of Technology – Delhi (IIT-Delhi)
• Not-for-profit Organisations: International Water Management Institute (IWMI), Jameel-Abdul Poverty Lab (JPAL), World Wildlife Fund (WWF)
• Consulting and Start-ups: GeoVale Services, Impact Dash, Sattva Consulting, CultYvate

Related Posts