In contrast to commercial banks, whose primary focus lies in financial services for profitability, development banks finance projects that contribute to societal progress. In this article, Dr Nicola Francesconi, a senior economist at KIT Institute, makes the case for adopting more rigorous impact evaluation techniques within development banks. These techniques would allow us to truly realise the economic, social, and environmental impacts of the projects they fund.

Impact evaluation, an applied research technique, assesses the changes attributable to a specific project. It’s gained considerable popularity over the past two decades. Duflo, Banerjee, and Kremer were also awarded the Nobel Prize for Economics for championing and optimising this data-intensive technique. 

This evaluation technique has emerged as the preferred statistical/econometric means to evaluate the effectiveness of development projects. It’s hard to find a development project that does not factor in and budget for impact evaluation activities. For example, every development project funded by the Dutch Netherlands is bound to allocate at least 10% of its budget for Monitoring Evaluation and Learning (MEL) activities. 

Most bilateral and multilateral development organizations have established internal MEL teams or divisions and extensively use external/independent impact evaluation consultants. Even if the rigour and insightfulness of these MEL activities are not always optimal, due to inadequate financial or human resources, the development sector has set the example and provided the necessary evidence and tools for other sectors of the global economy to embrace impact evaluation techniques.    

What sets it apart?

The importance of impact evaluation has been growing in tandem with global concerns about the apparent ineffectiveness of international development projects. Even when development projects were delivering what they were intended to deliver, their social, economic, and environmental impacts remained rather controversial and were often contested. 

As development projects faced growing criticism for their inability to create measurable changes, researchers started to come up with increasingly rigorous and insightful methods and techniques to evaluate impact. Today, impact evaluation brings together development research and practice, to deliver evidence about what works and what does not, and improve policymaking processes. 

Compared to frequently used results frameworks and outcome monitoring, which examine whether targets have been achieved, impact evaluation is intended to answer unconventional questions, such as: would the beneficiaries/stakeholders be better or worse off today, had a given project not been implemented? 

Through sampling, statistical, and econometric techniques, impact evaluations seek to identify cause-effect relations or changes in outcome that are attributable to a certain project. The methodology compares what happened as a result of a given project, and what would have happened had that project not been implemented.

So, what’s the problem?

Despite its growing popularity, impact evaluation is still strictly confined to development projects funded through grants, as opposed to development projects financed based on investments or loans. Grants are not only disbursed by governments but also and increasingly so by private sector foundations, they represent a small share of the total amount of funds that are disbursed for development purposes worldwide. However, the vast majority of development funds are disbursed by development banks in the form of investments and loans for both governments and enterprises. 

Development banks include international (World Bank, IFC, IFAD, etc.), regional (Africa, Europe, Asia, Latin America Development Banks, etc.) and national (FMO in the Netherlands, etc.) institutions, as well as (multi)national companies (Rabobank in the Netherlands, etc.).

All these banks have developed and systematically applied state-of-the-art techniques to perform or conduct due diligence and assess the economic, social, and environmental risks associated with development projects, to decide which one to invest in, and which ones to avoid. Most (but not all) of these banks also claim to evaluate the impact generated by their projects. However, these investment-based projects have only sporadically been the subject of rigorous and insightful impact evaluations.   

Why? Unlike grant-based projects, investment-based projects strive to maximize economic returns. And since impact evaluation is costly—as explained above the Dutch government typically estimates the cost of impact evaluation at about 10% off the budget provided to grant-based projects—but generates no tangible economic returns, development banks are reluctant to embrace it systematically.

Arguably, the few rigorous and insightful impact evaluations that were produced  on investment-based projects were not financed through these projects’ budgets but through complementary and grant-based funding. 

While impact evaluation may be costly, is it costlier than the environmental and social consequences we are facing, and will face, if we keep making investments that are not thoroughly evaluated?