Can you test astrology to see if it works? If you can’t, how do you know if it does work? And how did anyone work out all the detailed rules of astrology if they couldn’t test it to see if the rules were right?
In December 1985, Shawn Carlson published “A double-blind test of astrology” in the journal Nature. The purpose was to test the fundamental thesis of astrology, as agreed by the astrologers involved, which was the proposition that:
the position of the “planets” (all planets, the Sun and Moon, plus other objects defined by astrologers) at the moment of birth can be used to determine the subject’s general personality traits and tendencies in temperament and behaviour, and to indicate the major issues which the subject is likely to encounter.
There were two tests. In the first test, subjects were asked to pick their own horoscope out of three (their own and two controls). In the second test, astrologers were asked which one of three California Personality Index (CPI) results belonged to the subject whose natal data they had been given. Astrology failed both tests – the results were no better than chance. You can read more in my write-up from February 2005 - What do you mean, “test” astrology?
Recently there has been some discussion of these tests in the comments. In response to a question from me, one commenter Apollia argues that the CPI might not be that accurate, or that the astrologers might not understand it. Her comment is here. My reply to her is here.
Testing the people
Apollia’s comment raised for me an interesting point – how do you determine the actual personality traits of a person, in a way that they can be compared with what astrology predicts they will be?
Off hand I can think of a few ways:
- Ask the subject
- Ask friends of the subject
- Ask an some other third party expert to assess the subject
- Ask the astrologer to assess the subject
- Use a standardized test such as the CPI.
There may be others, although I can’t think of any that are significantly different from the above. There are problems with each.
Ask the subject
This is test # 1 from the Nature article. A criticism of this is that the subject may not be sufficiently self-aware to select an accurate description of himself from a group of three, and/or might be unwilling to “admit” to certain negative characteristics that he might in fact have. The Nature article agreed that this would be a problem, and concluded that because of this, test # 1 was a poor test of astrology.
Ask friends of the subject
This would get around the problem of the subject not wanting to admit to an undesirable characteristic. However, this method assumes that other people know the subject better than he or she does. While this might be true in some cases, it is likely to be less true in others. As with “ask the subject”, you would need first to determine scientifically that friends could select accurate descriptions of the subject.
Ask an some other third party expert to assess the subject
Again, you would need scientific evidence that these experts could select accurate descriptions of strangers, based on interview or other methods. In addition, you would need to demonstrate that the expert could accurately match this information to a horoscope, or that the descriptions could be accurately conveyed to the astrologers who would match them to a horoscope.
Ask the astrologer to assess the subject
This is basically Apollia’s idea. But here we would be assuming that an astrologer could select an accurate personality description more often than the subject, the subject’s friends or an expert psychologist. I don’t think astrologers necessarily have training in this area, so it’s hard to see how this would be true. But there is a much bigger problem with this method. There would be a huge possibility of sensory leakage – other ways the astrologer would have of “finding out” details of the subjects’ natal charts. Any valid test has to be securely controlled to prevent all other ways (other than astrology) for the astrologers to pick the correct chart. Any test that allowed the astrologers to spend time with the subjects would be deeply flawed and therefore invalid.
Use a standardized test such as the CPI
This is the Nature test #2. It is true that the CPI may not be 100% accurate or 100% understandable. But the CPI was chosen for the Nature test by the astrologers “because the advising astrologers judged the CPI attributes to be closest to those discernable by astrology.” (Page 420.) Also, the Nature test was devised and run by astrologers recommended by the National Council for Geocosmic Research for their expertise in astrology and in their ability to use the CPI. Even if the CPI is not perfect, you would still expect results better than chance. But chance results were exactly what they got.
It seems to me the CPI method is probably one of the best on offer, although I would be open to the friends or experts evaluations (# 2 and 3 above), if it could be demonstrated that these people could select the accurate description of the subject from two controls. But the burden of proof is clearly with astrologers to demonstrate that astrology can pass a test to demonstrate that it works, not with skeptics to show it doesn’t.
There is one thing I want to be clear about here. If none of these methods are acceptable, and if astrology can’t be tested, then it means that astrology is almost certainly bogus. For one, astrology’s doubtful provenance (no known method by which it is supposed to work, no known way its rules were derived, its absurd premises), mean we need extraordinary evidence that it works. By this I mean better evidence that we demand for many other things. But we are only offered poor evidence – anecdotes that are biased by the Forer Effect and confirmation bias. And for two, if astrology can’t be tested, then clearly no one would ever have been able to work out all the detailed rules astrologers use in the first place. How would they have been able to work out the rules if there is no way of ever testing them to see if they were right?
So either astrology can be tested – so astrologers, please tell us how. (And so far it has failed all well designed tests.) Or it can’t be tested – so astrologers please tell us how its rules were derived and how you know it works.
Which is it? Because it can’t be both.