Can you test astrology to see if it works? If you can’t, how do you know if it does
work? And how did anyone work out all
the detailed rules of astrology if they couldn’t test it to see if the rules were
right?
In December 1985, Shawn Carlson published “A
double-blind test of astrology” in the journal Nature. The purpose was to test the fundamental
thesis of astrology, as agreed by the astrologers involved, which was the
proposition that:
the
position of the “planets” (all planets, the Sun and Moon, plus other objects
defined by astrologers) at the moment of birth can be used to determine the
subject’s general personality traits and tendencies in temperament and
behaviour, and to indicate the major issues which the subject is likely to
encounter.
There were two tests. In the first test, subjects were asked to
pick their own horoscope out of three (their own and two controls). In the second test, astrologers were asked which
one of three California Personality Index (CPI) results belonged to the subject
whose natal data they had been given. Astrology
failed both tests – the results were no better than chance. You can read more in my write-up from
February 2005 - What do
you mean, “test” astrology?
Recently there has been some discussion of these
tests in the comments. In response to a
question from me, one commenter Apollia argues that the CPI might not be that
accurate, or that the astrologers might not understand it. Her comment is here. My reply to her is here.
Testing the people
Apollia’s comment raised for me an interesting
point – how do you determine the actual personality traits of a person, in a
way that they can be compared with what astrology predicts they will be?
Off hand I can think of a few ways:
- Ask the subject
- Ask friends of the subject
- Ask an some other third party expert to assess the
subject
- Ask the astrologer to assess the subject
- Use a standardized test such as the CPI.
There may be others, although I can’t think of any
that are significantly different from the above. There
are problems with each.
Ask the
subject
This is test # 1 from the Nature article. A criticism of this is that the subject may
not be sufficiently self-aware to select an accurate description of himself
from a group of three, and/or might be unwilling to “admit” to certain negative
characteristics that he might in fact have. The Nature article agreed that this would be a problem, and concluded
that because of this, test # 1 was a poor test of astrology.
Ask friends
of the subject
This would get around the problem of the subject
not wanting to admit to an undesirable characteristic. However, this method assumes that other
people know the subject better than he or she does. While this might be true in some cases, it is
likely to be less true in others. As
with “ask the subject”, you would need first to determine scientifically that
friends could select accurate descriptions of the subject.
Ask an some
other third party expert to assess the subject
Again, you would need scientific evidence that
these experts could select accurate descriptions of strangers, based on
interview or other methods. In addition,
you would need to demonstrate that the expert could accurately match this
information to a horoscope, or that the descriptions could be accurately
conveyed to the astrologers who would match them to a horoscope.
Ask the
astrologer to assess the subject
This is basically Apollia’s idea. But here we would be assuming that an
astrologer could select an accurate personality description more often than the
subject, the subject’s friends or an expert psychologist. I don’t think astrologers necessarily have
training in this area, so it’s hard to see how this would be true. But there is a much bigger problem with this
method. There would be a huge possibility
of sensory leakage – other ways the astrologer would have of “finding out”
details of the subjects’ natal charts. Any
valid test has to be securely controlled to prevent all other ways (other than
astrology) for the astrologers to pick the correct chart. Any test that allowed the astrologers to
spend time with the subjects would be deeply flawed and therefore invalid.
Use a
standardized test such as the CPI
This
is the Nature test #2. It is true that
the CPI may not be 100% accurate or 100% understandable. But the CPI was chosen for the Nature test by
the astrologers “because the advising astrologers judged the CPI attributes to
be closest to those discernable by astrology.” (Page 420.) Also, the Nature test was devised and run by
astrologers recommended by the National Council for Geocosmic Research for their expertise in astrology and in
their ability to use the CPI. Even
if the CPI is not perfect, you would still expect results better than
chance. But chance results were exactly
what they got.
It
seems to me the CPI method is probably one of the best on offer, although I
would be open to the friends or experts evaluations (# 2 and 3 above), if it
could be demonstrated that these people could select the accurate description
of the subject from two controls. But
the burden of proof is clearly with astrologers to demonstrate that astrology
can pass a test to demonstrate that it works, not with skeptics to show it
doesn’t.
Conclusion
There
is one thing I want to be clear about here. If none of these methods are acceptable, and if astrology can’t be
tested, then it means that astrology is almost certainly bogus. For one, astrology’s doubtful provenance (no
known method by which it is supposed to work, no known way its rules were
derived, its absurd premises), mean we need extraordinary evidence that it
works. By this I mean better evidence
that we demand for many other things. But we are only offered poor evidence – anecdotes that are biased by the
Forer Effect and confirmation bias. And
for two, if astrology can’t be tested, then clearly no one would ever have been
able to work out all the detailed rules astrologers use in the first place. How would they have been able to work out the
rules if there is no way of ever testing them to see if they were right?
So
either astrology can be tested – so astrologers, please tell us how. (And so far it has failed all well designed
tests.) Or it can’t be tested – so
astrologers please tell us how its rules were derived and how you know it
works.
Which
is it? Because it can’t be both.
Recent Comments