In yesterday’s post, I suggested that measuring the success of FTUSA’s FT4All coffee innovation pilots could be a complicated affair. The best imaginable scenario, in my mind, is an impact assessment process that is transparent, system-wide, longitudinal and conducted by an independent third party with no skin in the game. Hitting all four of those targets may be a tall order.
- Transparent. From design to implementation to reporting on results, the best impact assessment will be public and participatory, responding to the specific data needs identified by coffee chain actors from origin to the market.
- System-wide. All three coffee production models eligible for certification in the U.S. marketplace — cooperatives, independent smallholder farmers and estates — should be included in any FT4All impact assessment. It is not enough to know that new entrants are benefiting from their participation in the model. We also need to know that coops haven’t been hurt commercially in the process. FT4All can’t rob Peter to pay Paul. If the innovations in the Fair Trade system just shift Fair Trade supply from coops to other, non-cooperative production models, their net development impacts will likely be negative.
- Independent. An independent third party whose own fortunes are not directly tied up with the success or failure of the pilots may be better positioned for credibly assessing the impacts of the pilots than FTUSA.
- Longitudinal. While the best point of comparison for an independent smallholder farmer in a Fair Trade for All pilot may be her neighbor, the ideal assessment would compare her with her neighbor over a period of years on a small number of key livelihood indicators. This may be the most difficult of these four characteristics to achieve. Longitudinal studies, by definition, take time — likely more than coffee brands can afford given the market pressure for them to decide which approach to Fair Trade Certification suits them best.
7 Comments
Michael, Fair Trade USA’s primary sales pitch/goal for FT4All has been that it will double impact by 2015.
Are you saying they haven’t worked out how to measure impact yet?
Michael:
What I am saying is that I haven’t seen any M&E plan. That I think it will be complicated in this environment for FTUSA to be the official source of information on the impacts of its innovations. And that I think these principles should guide any credible effort to assess the impact of the pilots.
Michael
The principles you’ve laid out are good indeed, but it opens up the broader question of whether or not FTUSA has developed their own measurement framework yet.
It would also be helpful to know which indicators they’ll use to use to measure impact. Presumably they have these, as the pilots have already been launched and the goal to double impact has already been set.
Perhaps someone from FTUSA can provide more details.
Its just my personal opinion but I had guessed that by ‘twice the impact’ FTUSA had meant merely “twice the pounds” of certified product.
But, as Michael (Sheridan) makes clear, and as we at Equal Exchange have stated in different ways and in different places, not all pounds of exported product have the same social/economic/commercial/political impact.
So this is an interesting line of thought and suggests a host of Q’s. Including the pts raised above we have:
– How is FTUSA defining ‘impact’ (& not just for un-organized farmers, but across all the producer communities)?
– How will it measure that impact?
– How will FTUSA (& the rest of us) compare apples & oranges? e.g. how do you translate the benefits delivered thanks to the sale of one container of plantation coffee (maybe exported at an unknown price*?) with the benefits delivered by the sale of a comparable container by a co-op or a group of un-organized farmers working through a ‘Market Access Partner’?
– And what is the mechanism, or formula, by which observers will know that any future plantation plantation exports did _not_ represent lost potential sales by the co-op sector?
(*some representatives of the co-op sector have expressed concern that Fair Trade certified plantations have a variety of structural advantages [certainly in the banana sector] and therefore will undercut the co-ops on price and sell their FT certified coffee for less – which I imagine has to somehow represent a net loss to the coffee growing countries since that would mean fewer dollars coming in for every FT pound shipped out.)
“Its just my personal opinion but I had guessed that by ‘twice the impact’ FTUSA had meant merely “twice the pounds” of certified product. ”
That would be a terrible measure, so let’s hope not. Simply by expanding what can be labeled as Fair Trade Certified, you could effectively double the amount sold as FTC without having any producer-side impact at all.
This really needs some clarity from FTUSA.
Like Rodney, I wonder “- …what is the mechanism, or formula, by which observers will know that any future plantation plantation exports did _not_ represent lost potential sales by the co-op sector?”
In the absence of the counterfactual–how the co-ops would have done in the absence of FT4All’s initiatives–it is very difficult to measure “system-wide” impacts, both positive and negative.
Matt:
There are plenty of challenges on the measurement side, but this is the biggest from my perspective. A counter-factual may be an impossibility. But I think longitudinal data from coffee cooperatives beginning with pre-FT4All trading data could be useful in testing the hypothesis advanced by coops and their advocates: the entry of new production models will have adverse effects on the commercial standing of coops. Over time, as new entrants push more volume through the system, is there evidence of downward pressure on price (as measured by differentials with the “C” market)? Are coops selling steadily less of their volume on Fair Trade terms? Are there consistent changes in trading relationships from certain types of trading partners to others? Etc.
This might not tell you whether coffee purchased from an estate would have been purchased from a coop in the absence of FT4All, but it could help show whether the commercial well-being of coops erodes over time as a result of FT4All.
Michael
1 Trackback