At the Arusha Savings Group Summit, Bram Thuysbaert (Yale) and Beniamino Savonitto from Innovations for Poverty Action presented early evidence from a 2-year study of the impact of CARE’s savings associations (VSLAs) in Uganda, using randomized control trials. The results were unexpected.
The study compared ‘treatment’ villages where savings associations had been formed with ‘control’ villages where they had not. Conference delegates were told that members’ investments did not increase in the treatment villages in key areas like schooling for children, agricultural equipment and inputs (like seeds and fertilizer) and household assets.
Given the high density of informal savings associations (like ASCAs and ROSCAs) in these villages at the outset, this study casts doubt on the assumption that VSLAs formed by NGOs must always be better than those formed without them.
I asked the authors how they had collected their data? They explained that it was collected through participant surveys. They acknowledge that if people were not telling the truth, it would impact the accuracy of their results.
“The main difference between the treatment and control villages that I see” I told them, “is that in the treatment villages, CARE has been working for 2 years forming associations and priming villagers in how to answer questions. Couldn’t this have an impact on villagers’ testimony?”
“Yes” they said, “it could.”
“How do you control for this variable?” I asked.
“We can’t” they said.
Is this really the ‘gold standard’ of research methodologies?
villagefinance 
As always, attempting to find a gold standard for assessing the direct effectiveness of almost any poverty alleviation measure is proving elusive. Apart from such variables as simply the exposure to NGO interventions, the fungibility of money makes it so very difficult to demonstrate casuality.
I’m curious why the study focused only on whether or not investments had been made in the designated activities. If the core question relates to whether or not participation in VSLAs positively contributes to a family/household/community’s reduction of poverty, then why not make use of recognised poverty measurement tools in making the comparison between the two groups?
Surely the question of the actual poverty experience (the result) of VSLA participants is more relevant than whatever they choose to do with any increased cash flows (the means)?
Greg
LikeLike
I don’t fully understand your concerns. Wouldn’t the CARE program prime people to over report investments, not under report? This would likely bias the results downwards for the treatment group, not upward. If anything, this presents stronger evidence that the CARE program did nothing for people.
LikeLike
Nathan — I am concerned about the quality of the research methodology. Why would we assume that savings associations in Uganda must be achieving high impact? That is an empirical question, and there are very good reasons not to prejudge the answer.
LikeLike
[…] Link: Measuring the Impact of Savings Associations […]
LikeLike