Still no evidence prayer works

You may have read recently about a new meta analysis that purportedly shows that intercessory prayer works. If you had read my posts on intercessory prayer you might have been surprised by this since other good studies have shown that prayer doesn’t work, and prayer still doesn’t work.

Steven Novella at NeuroLogica Blog explains why this prayer meta analysis is bogus:

But meta-analysis is tricky business. First, it should be pointed out that it does not represent new data – it is just taking a fresh look at old data. It can be useful but only when it is very carefully applied. For example, the studies that are lumped together should have very similar design, they should be looking at the same type of subjects and should use similar outcome measures. The results of a meta-analysis are only meaningful if data from the different studies can be reasonably combined.

Also, a meta-analysis does nothing to address the quality of the studies being looked at. The old adage of “garbage in-garbage out” still applies. If you lump together 10 bad studies, you don’t get one good study, you get a useless meta-analysis. For these reasons meta-analyses have a poor track record of predicting the ultimate outcome of a question, once definitive studies are done, failing over a third of the time.

A meta-analysis is not always the best method for coming to an overall conclusion about an area of research. There are aspects to the pattern of results that are important to consider, and are white-washed in a meta-analysis. For example, what is the trend between study quality and size of the effect? If we see a real tendency for the better studies to have a smaller effect (which we do, in my opinion, in the intercessory prayer literature), that strongly suggests that the effect is not real. By combining these studies, however, these differences are erased. In effect, the good data is diluted in bad data.

More at the link.

Unfortunately, as Novella laments, this study has already produced headlines of the “new study finds prayer works” kind, while the later retractions, as the data is analyzed properly, will get little airtime.

Ironically, Paul Kurtz of the skeptical Center for Inquiry will be undergoing cardiac surgery today. I wish him well, and I won’t be praying for him.

Comments

[dikkii]

I really don't know why people continue to look into this sort of thing.

Prayer can't work, almost by definition.

I blogged about this, once.

[Louie]

So what you're saying is, asking a supernatural being to help me, isn't going to work...?

[Eric]

I will pray for Paul.

[Marcus Ranum]

I loved Christopher Hitchens' observation in the "blasphemy debate" about the efficacy of prayer. After all, he points out, millions of English people pray "God Save The Queen" every DAY. With such a massive blast of prayer you'd expect the royal family would be doing better than they are. :)

[dikkii]

After all, he points out, millions of English people pray "God Save The Queen" every DAY.

Cue drunken river cruise, riots and headlines in papers the next day.

Is it just me, or is the first tune that anyone else hears in their head when they hear this phrase is the Sex Pistols' tune of the same name?

[Marcus Ranum]

It's not you. The Pistols completely re-purposed that song eternally. More power to 'em!

[KenJ]

So scared of death arent ya!

[Tom Foss]

What does that even mean?