Professors in statistical bikinis

Via Tony Woodlief, Ross McKitrick and Bruce D. McCullough uncover fraudulent data manipulation on the part of academics in driving policies. Cases include:

Federal Reserve Bank of Boston on mortgage lending

A 1992 study by economists at the Federal Reserve Bank of Boston purported to show a widespread discrimination against minorities in the Boston mortgage market. This study quickly became the basis for government mandates to relax lending rules, allowing people who did not meet traditional lending requirements to obtain mortgages. This ultimately contributed to the current U.S. financial crisis.

When independent researchers attempted to replicate the study, the underlying data were inaccessible. Key information was eventually obtained using the Freedom of Information Act, which revealed coding errors in the original data that invalidated the results. But the replication process took six years, by which time the new lending rules had long been enacted.

U.S. Centers for Disease Control and Prevention on obesity

A 2004 study published in the Journal of the American Medical Association from the U.S. Centers for Disease Control and Prevention claimed obesity kills 400,000 Americans annually. The study attracted significant media attention and resulted in the U.S. government immediately allocating $60 million for obesity-related programs.

But other researchers soon discovered that the study’s data were unreliable and that a proper peer review had not been conducted. The next year CDC scientists estimated the number of deaths attributed to obesity might only be 26,000, and the CDC began downplaying any numerical estimate of deaths related to obesity.

The “hockey stick” graph and climate change

A 1998 study into the climate history of the northern hemisphere, led by Michael Mann, resulted in a graph implying the Earth’s climate cooled slightly for 900 years and then warmed rapidly in the 20th century. The graph was used extensively by the United Nation’s Intergovernmental Panel on Climate and played an influential role in convincing governments around the world to ratify the Kyoto Protocol in 2002.

But when independent researchers McKitrick and Stephen McIntyre tried to replicate Mann’s results, they were stymied by his refusal to identify the data he used and to clarify key steps in his calculations. Using what little data were available, they found errors in Mann’s work that invalidated his conclusions.

In 2006, the U.S. National Research Council investigated the issue and concluded that Mann’s study failed key tests of statistical validity. Mann’s conclusions were also deemed insupportable by an expert panel led by Edward Wegman, professor of statistics at George Mason University and chairman of the National Academy of Sciences Committee on Theoretical and Applied Statistics, which was convened at the request of the U.S. Congress.

“Publicly-traded companies are required by law to ensure transparency and veracity in all their financial reports. Yet the same stipulations don’t apply to academic research or academic journals, even when large amounts of public money are at stake,” McKitrick said.

This is why I believe prediction markets are important--they inform policy and can offset institutional bias of the greatest order.

Statistics are like a bikini. What they present is suggestive, but what they conceal is vital--Aaron Levenstein

Photo link here. Previous B.S. installment here.