And Caught By A Student No Less…

Posted on May 1, 2013by 

By Ruth Alexander
BBC News
Thanks to L.

Thomas Herndon

This week, economists have been astonished to find that a famous academic paper often used to make the case for austerity cuts contains major errors. Another surprise is that the mistakes, by two eminent Harvard professors, were spotted by a student doing his homework.

It’s 4 January 2010, the Marriott Hotel in Atlanta. At the annual meeting of the American Economic Association, Professor Carmen Reinhart and the former chief economist of the International Monetary Fund, Ken Rogoff, are presenting a research paper called Growth in a Time of Debt.

At a time of economic crisis, their finding resonates – economic growth slows dramatically when the size of a country’s debt rises above 90% of Gross Domestic Product, the overall size of the economy.

Word about this paper spread. Policymakers wanted to know more.

And so did student Thomas Herndon. His professors at the University of Massachusetts Amherst had set his graduate class an assignment – pick an economics paper and see if you can replicate the results. It’s a good exercise for aspiring researchers.

Thomas chose Growth in a Time of Debt. It was getting a lot of attention, but intuitively, he says, he was dubious about its findings.

Some key figures tackling the global recession found this paper a useful addition to the debate at the heart of which is this key question: is it best to let debt increase in the hope of stimulating economic growth to get out of the slump, or is it better to cut spending and raise taxes aggressively to get public debt under control?

EU commissioner Olli Rehn and influential US Republican politician Paul Ryan have both quoted a 90% debt-to-GDP limit to support their austerity strategies.

But while US politicians were arguing over whether to inject more stimulus into the economy, the euro was creaking under the strain of forced austerity, and a new coalition government in the UK was promising to raise taxes and cut spending to get the economy under control – Thomas Herndon’s homework assignment wasn’t going well.

No matter how he tried, he just couldn’t replicate Reinhart and Rogoff’s results.

“My heart sank,” he says. “I thought I had likely made a gross error. Because I’m a student the odds were I’d made the mistake, not the well-known Harvard professors.”

His professors were also sure he must be doing something wrong.

“I remember I had a meeting with my professor, Michael Ash, where he basically said, ‘Come on, Tom, this isn’t too hard – you just gotta go sort this out.’”

So Herndon checked his work, and checked again.

By the end of the semester, when he still hadn’t cracked the puzzle, his supervisors realised something was up.

“We had this puzzle that we were unable to replicate the results that Reinhart-Rogoff published,” Prof Ash, says. “And that really got under our skin. That was really a mystery for us.”

So Ash and his colleague Prof Robert Pollin encouraged Herndon to continue the project and to write to the Harvard professors. After some correspondence, Reinhart and Rogoff provided Thomas with the actual working spreadsheet they’d used to obtain their results.

“Everyone says seeing is believing, but I almost didn’t believe my eyes,” he says.

Thomas called his girlfriend over to check his eyes weren’t deceiving him.

But no, he was correct – he’d spotted a basic error in the spreadsheet. The Harvard professors had accidentally only included 15 of the 20 countries under analysis in their key calculation (of average GDP growth in countries with high public debt).

Australia, Austria, Belgium, Canada and Denmark were missing.


Herndon and his professors found other issues with Growth in a Time of Debt, which had an even bigger impact on the famous result. The first was the fact that for some countries, some data was missing altogether.

Reinhart and Rogoff say that they were assembling the data series bit by bit, and at the time they presented the paper for the American Economic Association conference, good quality data on post-war Canada, Australia and New Zealand simply weren’t available. Nevertheless, the omission made a substantial difference.

Click Here to continue reading.