Gates Foundation statistical error repeated in the New York Times?
“In College Endowment Returns, Davids Beat the Goliaths” starts off by pointing out that small college endowments and large ones had roughly the same return:
the smallest endowments— those under $25 million — edged out the biggest endowments, averaging a five-year annualized return of 10.6 percent to the $1 billion-plus category’s 10.4 percent.
[Don’t expect these returns going forward for your own portfolio! We’re still getting a dead cat bounce from the Collapse of 2008.]
Curiously, the Times considers 10.6 versus 10.4 to be a significant difference in favor of the smaller endowments, but the real story is in the next paragraphs:
Even more surprising, the top-performing endowments over 10 years among all schools reporting data weren’t giants like Harvard and Stanford or even Yale … the top-performing colleges are two Virginia universities whose financial resources amount to a negligible fraction of the typical Ivy League endowment. … Radford University, which ranked first, has an endowment of $55.5 million, and Southern Virginia University, which was second, has an endowment of just $1.1 million. Radford’s annualized 10-year return is 12.4 percent, and Southern Virginia’s is 11.2 percent. For the most recent fiscal year, 2015, Radford earned over 13 percent and Virginia Southern’s return was 10.5 percent. The average return last year for all endowments was just 2.4 percent.
Should we be surprised that a particular portfolio of $1.1 million outperformed a portfolio of $37.6 billion? Perhaps, but not if there is a bell curve of performance and there are a lot more $1.1 million portfolios than there are $37.6 billion ones.
The Gates Foundation went down this road after finding that a lot of the best high schools in the U.S. were small. The conclusion was that smallness leads to better academic performance. It later transpired that small high school performance fell on a bell curve, just the same as large high schools, but with a somewhat higher standard deviation (and therefore it was easier to find good-performing small schools). See this analysis in Marginal Revolution. (And, of course, don’t forget that a high school falling on the center of the U.S. bell curve would be considered an emergency situation in Finland, Shanghai, Singapore, or South Korea! See “Smartest Kids in the World Review”.)
Readers: Did the New York Times fall into the same statistical trap that led the Gates Foundation to squander hundreds of millions of dollars? If so, why are we so prone to this one? Can we blame the lack of emphasis on null hypothesis testing in AP Statistics?
Full post, including comments