kyk

Why Most Published Research Findings are False!

2007-09-24T20:33:00Z

Here's a very interesting paper from a guy named Ioannadis, which was recently written up in the WSJ: http://medicine.plosjournals.org/perlserv?request=get-document&doi=10.1371/journal.pmed.0020124

The controversial title of his paper is "Why Most Published Research Findings are False." It's published in journal meant more for a medical audience, but I think statistically speaking, the findings apply to all sorts of research.

The argument sort of rests on the Bayesian interpretation of what probability is. I don't want to get into the gory mathematical details because that would be uninteresting but let the following stylized example illustrate:

Suppose you have a medical test that is 99% accurate. That is, if a person has a certain ailment (call it grooteditis), the test will come back positive 99% of the time. If the person on the other hand, does NOT have grooteditis, the test will show negative 99% of the time. So it is 99% correct.Now suppose you are a doctor and a person comes into your office and tests positive for grooteditis. Can you say with 99% certainty that this person has grooteditis?

Well, not really. The certainty of whether or not this person has grooteditis depends on what the prevalence of grooteditis is in the population. For example, what if there are SO few people with the disease in the population, that even when the test shows positive, among those times, it is often wrong? So for example, if only 1% of the population had grooteditis in reality, there's a good chance it's a false positive.

If you do the calculation it turns out that you can only say with 50% certainty that this person has the disease.

So what I think Ionnadis is mostly saying is an extended, detailed and sophisticated version of the above. Statistics obviously don't lie. But even if the data looks great, most theories are reductionist by necessity. In fact, one might say that the probability that a theory is true is probably very low. So the implication of this would be that we're probably, on average, too trusting of research findings.

I think this is good reason to pause before being sold on a "statistically significant" study done by this or that scholar or interest group. The probability that their research findings are true is probably even lower, as they generally have a more worldly vested interest in their subject matter.

Easy to read

2007-09-23T05:45:00Z

Everyone knows that elections are all about political posturing, but this is one of the most blatant and un-nuanced forms of political posturing that I've seen in a while. Got it off The Politicker, link to the article: http://polhudson.lohudblogs.com/2007/09/21/giuliani-at-the-nra/

Giuliani at the NRA

Former New York Mayor Rudy Giuliani told members of the National Rifle Association earlier today that the lawsuit he filed against gun manufacturers in 2000 “is not necessarily what is needed now.’’

The Republican presidential hopeful told NRA members to “not feel singled out.’’

“I was excessive in every way that I could think of in order to reduce crime,’’ Giuliani said. “I enforced the guns laws that existed strongly.”

Giuliani added, “That is not necessarily what is needed now. It certainly isn’t the interpretation that I think is the correct interpretation of the Second Amendment. So I would say that I didn’t anticipate the lawsuit would go in some of the directions it is going.’’

However, the general counsel for Beretta USA Corp., which is one of the defendants in the lawsuit, disagreed with the former mayor’s characterization of the lawsuit.

“In fact the lawsuit is exactly the same one he started at the beginning of his administration,’’ said Jeff Reh, Beretta’s general counsel. The case is scheduled to be argued today before the Second Circuit Court of Appeals, Reh said.

Compelling column by Starquest

2007-09-07T03:08:00Z

Once we get past the ironic hilarity of the bathroom stall incident:

Former Parks Commissioner and former CM, Henry Stern, has what I think is a very well-reasoned stance on the Larry Craig incident. He has, from the start, supported letting the Idahoans decide... although, it's clear for most people why the quick and dirty political crucifixion is more appealing to his erstwhile bedfellows.

Here's an excerpt that I received in my inbox recently. You can probably find the whole thing at www.nycivic.org.

"We ask whether a Senator elected by the people of any state should be forced from office for tapping his foot and swiping his hand in a toilet stall? Is intent sufficient for a conviction for lewd behavior, resulting in the loss of a job one has held for 17 years? What if, when they met face to face, one man had found the other to be a troll, or an undesirable sex partner for any other reason? What if the older man had been solicited but refused to pay? (Senatorial privilege, they are used to freebies). We usually punish crimes, not intentions, except for cases of terrorism or where injury to others is foreseen. Who would be injured in this case, whether they had sex or not? What about the defense of entrapment? If the officer had not responded to the tapping by moving his own shoe, that could have ended the matter."

Should there be a market for organs?

2007-09-06T00:34:00Z

Ignoring for a second that it's politically unpalatable, should we be able to buy and sell our organs?

There was an interesting article by Dubner and Levitt, both of Freakonomics fame in the NYT.

http://www.nytimes.com/2006/07/09/magazine/09wwln_freak.html?ex=1310097600en=a9dde90f918e52cbei=5088partner=rssnytemc=rss

A very complex issue, with a number of facets to consider. There are probably also hundreds of non-negligible scenarios as well. But the important question is whether it would do more good than harm or more harm than good.

I feel like it has a significant upside that people dismiss without considering fully. Perhaps individuals in our society have some sort of training to be averse to implementing extremely free-market principles to such grave matters.