Did These Elite Economists Make A Basic Error?

February 10, 2013 1:17 pm

When we first spotted this, it was because we went looking for some statistics on USPS – yes, the US postal service which is sinking in losses, losing to the tune of $14.4 Million every day. We then spotted a claim that the USPS is the most efficient mailing service in the world – well, if that one was based on some very measurable parameters on delayed delivery numbers etc, it would have been a great stat to spot. However, this claim is based on a 2012 study by some economists from the University of Ottawa, Tuck Business School at Dartmouth College, EDHEC Business School, and Harvard University. The Forbes article, where we spotted this whole thing, says this:

“What they did was mail letters to 10 fake addresses in 159 countries and see how long it took them to be returned, and what percentage of them are returned.

The U.S. Post Office returned all ten letters and they did it the fastest, which has caused some to claim that they are the most efficient.

There is one problem with this interpretation of their results: the letters in the study were sent from Cambridge, Massachusetts. Is it really surprising that letters sent from one U.S. address to another U.S. address and back were returned faster than letters sent to from a U.S. address to New Zealand and back? ”

Source

We couldn’t believe it, we thought that this might be some interpretation issue, so we looked at the full paper of the study. Table 1 – Measures of Mail Efficiency – has United States listed at the top, with ‘Avg number of days to get the letter back’ listed as 16.20 – opening up a huge set of possibilities to quote this data in multiple ways. Which is precisely what seems to have happened, with this number being used to claim that USPS is the most efficient mail delivery system, internationally.

We do not know enough of the domain to claim that there was an issue with the way the study was conducted. But we do think that the table that published the results must have clearly called out the fact that the letters were sent from the U.S.

[We might be wrong with some points here, and this is based on a quick read of the paper, so we apologize for any misinterpretation from our end, if we have missed something here]

For Interesting Statistics Everyday, Find Statspotting on Facebook and Follow Statspotting on Twitter

Leave a Reply