thoughts

Stats of Excellence

shot

Lies, damned lies, and statistics. The Cup of Excellence is now in its third decade of existence. That, while technically true, is also quite misleading, given that last summer was its tenth anniversary. Nonetheless, that is a striking amount of time for a competition that kicked-off humbly in Brazil in 1999, with the number one coffee receiving what would now seem a very modest $2.60 high bid, while other coffees received only a fraction above the commodity price of the time. The competition has matured enormously, spread to 9 countries (including one-night stand Rwanda), and record prices for those countries have been set and broken in the process.

The transparency of the system, and its archival data allows us to ask some questions, see if trends emerge, and maybe if any new insight can be gained.

Have prices changed over time?

This seems obvious, prices of speciality coffee across the board are increasing (or so we are led to believe). While there are year to year variances, average Cup of Excellence prices* are trending upwards in every country, apart from one, Bolivia, where (inflation adjusted) prices appear to be trending downwards.

Bolivia Downward

The inflation rates in Bolivia weren’t out of line with any other country, yet even with inflation adjustment prices in every other country maintained an upward trend. Is inflation adjustment just a trick? Is it important/valid? For me it gives a better idea of the worth of the price given to the farmer (changes in dollar value would also be of interest here, but perhaps that is an exercise for another day).

Has the recession influenced prices paid?

I’ve heard it bandied about that speciality coffee is recession-proof. Average prices paid in 2009 were down from 2008 in Guatemala, Colombia, El Salvador, Nicaragua and Honduras. Bolivia rebounded from an (inflation adjusted) all time low in 2008, the Brazilian auction is yet to happen, while prices for latecomer Costa Rica continue to increase.

Data 2

This suggests at least some correlation between the recent economic downturn, which many would claim peaked in September/October 2008 (the first auction after that was Bolivia 08). Having said that prices also fell in Bolivia, Nicaragua, Brazil and Honduras in 2005 (in every country bar El Salvador that there was also a 2004 auction), so we don’t have a strong argument here. So what other factors might be influencing prices?

What relationship exists between quality and price?

Another metric we can examine is the cupping scores. Although there is a bottom cutoff of 84 points, the distribution of scores above that could give us an indication of changes in annual quality. For instance, lets take our example of Bolivia once more.

Bolivia Scores

Both mean and median scores have trended downward in Bolivia, with the mean giving an overall picture, while the median, which has decreased more rapidly suggesting, that of the coffees at auction in Bolivia, year on year a greater proportion are receiving relatively lower scores. Do mean and median cupping scores dictate the price paid? If this is to be valid for the apparent decline in Bolivian prices then all other countries should exhibit increasing mean and median cupping scores. They do not. The scores in Nicaragua, El Salvador and Colombia appear to be trending strongly upwards. In Brazil, Honduras and Guatemala a specific trend is unclear, while in Costa Rica the scores appear to be declining.

Ok, a bit of a bust there. What if we look broader? In the graph above showing the yearly average prices for different countries you can see that some countries do better than others consistently. For instance Guatemala has received the highest average price every year since 2006, while El Salvador has received the lowest or second lowest amount since 2007. Maybe a comparison of the cupping scores between countries will tell the tale here.

Data 8

Note here, that in 2003 in El Salvador coffees above 80 and below 84 were allowed to go to auction, hence the outlying number. Guatemala, while commanding higher prices between 2006 to 2008 appears to be distincly middle pack in terms of cupping scores (while just being pipped to the top place in 2009 by Bolivia). In fact in both 2006 and 2007 El Salvador scored more highly on average. The graph also suggests that 2007 and 2008 were red-letter years for Colombia, similarly 2006 for El Salvador, while scores in 2009  across all countries are closer than ever. Does this suggest that countries are reaching a unity in terms of quality? Perhaps just that teams of judges across different competitions are better calibrated? It certainly doesn’t tell us why average prices seem to be so high in Guatemala.

So do any of these statistics make a stronger argument?

Yes. I’m no economist, but this economic model is one I’m relatively aware of.

inverse_bags

Supply and demand. This plots shows the number of bags up for auction for each country each year. This much more closely reflects the average price graph from above. El Salvador had the biggest number of bags in auction for the last two years – it had the lowest price. Honduras had the second biggest auction this year, it had the second lowest price. While we don’t know the Brazil prices yet, Nicaragua had the next biggest auction and the next lowest price. Guatemala has consistently had the smallest or second smallest auction since 2005.

Look how nicely Honduras’ prices correlate to auction size.

honduras

So while we talk and talk about the Cup of Excellence being about quality, it is also very much about prestige and branding. While the quality in El Salvador may be higher on average than Costa Rica (at least going by average scores of which Costa Rica’s were the lowest in 09), in 2009 Costa Rican coffees received on average $3 more than those from El Salvador. The crux is of course that a lot speciality coffee people want to carry a coffee from Costa Rica, and may be willing to pay a bit extra for a coffee that carries the brands CoE and Costa Rica, regardless if there is better value elsewhere. I don’t think these kind of observations will come as any great shock, the Esmeralda brand has been built on limited supply driving up big prices. It would however be interesting to see what trends might emerge if auctions became more uniform in terms of size. I suspect based on my observations that prices would become increasingly dictated by quality (of course I know that prices within any one auction are generally score driven, but I speak broadly here).

What about the record breakers?

Ok, so we’ve talked about the average prices, but as much if not more attention is given to the high priced coffees, the coffees that break records. Do these coffees also correlate inversely to lot size or positively to cupping score, or even to average price? Overall, no there was no consistent correlation between the price of the highest priced coffee. Obviously, to a certain extent these number one coffees are all scored relatively highly, but you cannot say that a certain score guarantees a particular price. For instance why does a 93.60 score in Guatemala in 2007 gain $19.50, while 93.68 in 2008 gains $80.20. Bidding wars obviously come into the astronomical prices, but I wanted to see if there was something more imagination-capturing about these coffees.

For this I chose all coffees that have broken the $20 mark (14 coffees) and I’ve compiled their cupping notes. For those years I’ve also chosen a middle of the pack coffee and compiled their cupping notes. The idea would be to see if certain words are more frequently associated (or not associated) with the high priced coffees.

+$20 coffees

Screen shot 2010-01-14 at 20.45.48

Mid-priced coffees

Screen shot 2010-01-14 at 20.48.35

First of all, there were a lot more words used to describe the high priced coffees. Both groups associate with chocolate, cherry, orange, clean, creamy, caramel  and sweet more or less equally. Apple is a term used to frequently describe the mid prices coffees, but rarely the high priced coffees. Conversely complex and floral are frequent descriptors of the high priced coffees but rarely the mid priced ones. Indeed “complex” was a term that seemed to jump out from the high priced coffees just by cursory reading.

To complete this I chose a third group, first placed coffees under $20, in countries that have gotten +$20, either directly preceding or following a +$20 year.

Screen shot 2010-01-14 at 21.39.12

Floral and complex also appear in these scores, but to a lesser extent, while long  and honey appear more frequently. Apple once more makes little impact on these coffees, suggesting again that it is a descriptor more common among mid-scoring coffees, though perhaps that it doesn’t affect final price. I think the subjective nature of these descriptor makes using them for this kind of analysis quite difficult. Perhaps comparing two distinct groups, like first placed and mid placed coffees it is somewhat easier to make distinctions, but separating first placed coffees delimited by a somewhat arbitrary figure… shaky ground methinks.

prediction time

Ok, so the auction for the 2009 Brazil coffees is next Tuesday (Jan 19th). I’m going to stick my neck out and make some predictions based on my observations.

The average price will be greater than Honduras ($4.66) but less than Nicaragua ($6.20), based on the size of the auction.

My prediction = $5.43

The top price, and this is where I feel much less certain, will not break $20. While complex and floral are used to describe the number 1 coffee, so are apple and long.

My prediction = $12-$14

Wish me luck.

*My average price calculations were obtained by generating the sum of the products of coffee price by lot size divided by total auction size. Put simply - the average is weighted depending on the size of the lot.
Word clouds generated with Wordle.

Advertisements
Standard

12 thoughts on “Stats of Excellence

  1. I think there are a lot of things that effect the prices attained in COE lots that you need to scratch below the surface for to really find out. Not my place to point them out but if you look at some past data compared to new data its very interesting.

    I also think we are at a crucial point of the programs development that is very very interesting (and worrying). I find it telling some of the big names in the industry used to buy from the program (that got the headline prices) no longer do. There are reasons for this. I think the days of the high prices for a number one maybe gone while these buyers invest there dollars else where. Some could (and do) say that its because of the success of the program to create the relationships (and the real reason for the program) but I’m not so sure. Panama seems to be still building on those foundations (although a much younger program).

    I find the descriptor part very interesting and very dependent though on the jury and its abilities. Might be interesting to compare some experienced jury’s alongside some not so. Anecdotal experience leans me towards experienced jury’s being much harsher than heavily loaded jury’s with new cuppers to the program.

    The strangest competition for me last year was the El Salvador one which was way of target price wise compared to the wonderful coffees that were put forwards to auction.

    I think some other interesting data is the spread of buyers with Japan leading the way as the main buyer in the group with Europe a way behind and the US no where to be seen.

    Lots of fuss about Target buying coffee from the competition last year, but this worry’s me very much that a HUGE brand like that try’s to piggyback on the success of a program for a cheap marketing opportunity.

    Which leads me onto the reasons for buying COE. for many its a marketing opportunity and exactly that. for some its a way of the market buyer meeting the seller. I would be very interested to see how many people follow up there prize purchase with some kind of relationship. Unfortunately data that is not really collected.

    Nice piece my friend

    • Some really great points there Steve. Absolutely I agree, there are a lot of factors, many of which I have not touched on. Not my intention to conclude anything definitive, rather to present some observations based on available evidence.

  2. A fascinating read, thanks David.

    The descriptor clouds make for interesting reading too.

    Could a reason that average prices have fallen in recent years be that buyers have formed direct trade relationships from previous years’ entrants and therefore no longer need to compete in an auction scenario?

    If the prices continue to fall, could we see a return to more European and US buyer dominance as displayed in early COE auctions?

  3. I think CoE’s strength is the focus on discovery of the special. In that CoE looks through all these coffees and allows people to cup a highly select representative of a countries coffee. To me it encourages people to form relationships and shows origin to roaster isn’t as hard as some would like us to believe. I think the pricing and quality data collection is an area that is influenced by alot of other factors. A look at direct trade and diversity of coffee points to the benefits.

  4. Statistics. A very tricky thing. You can read volumes into this, but your assumptions still might be wrong. The numbers can be put into various different models and produce different results. I’m not saying that what you’re pointing at is wrong, but having a wife with a PhD in this has made me aware of these dangers.

    Yes, thats right wifey; I actually did pick up a word or two of what you were saying even though I was watching The Simpsons.

    As for the lot sizes; the increasingly smaller lot sizes has made it less interesting for some larger buyers to buy from the program so they more and more often buy them outside the program directly. This may account for the lower overall prices.

    As Steve pointed out there probably is more behind the numbers than what has been pointed out.

    Still a very interesting post with many great points and graphs! Thanks for doing such a thorough job of putting the numbers into a system. Great post!

    • Ola – you are quite right. While I may have observed some correlations, we cannot assume causation. There may be other explanations. That said I think the supply and demand argument fits nicely. Also from looking at the data on the size of the auctions, lot size plays some role in contributing to the overall size, but the number of coffees plays a much bigger role eg there were 36 coffees on auction in El Salvador 08 CoE, compared to 25 in Guatemala the same year, and only 18 in Colombia.

  5. I share Ola’s skeptiscism for the causation, but I love how your word clouds make the descriptions look incredibly silly. There shouldn’t be a correlation between the simple aroma descriptor “apple” and mid-priced coffees – the fact that it is certainly makes it reasonable to look again at how those descriptors are gathered.

    Is “Notes” in the second cloud a mistake?

  6. Pingback: Red Links 18/01/10 | Alexia Golez

  7. Pingback: Twitted by uberproject

  8. Pingback: Blogger apathy, the death of Bloglines and missing links « Sam Kinsley

  9. Pingback: Enter Dave stage left… « Dublin Barista

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s