« The thing that made the things for which there is no known maker | Main | Lame Pet Psychic Tricks »

August 03, 2009

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

AvalonXQ, if true, that's a reasonable argument. However, proving the study is flawed is no evidence that astrology isn't.

Avalon,
In the first part of the test, subjects failed to identify their own astrological readings. So, using your quite reasonable line of argumentation, that says more about the accuracy of astrology than it does about the participants self awareness.

And if people can't even tell their own reading from someone else's, then on what grounds do people like Ken claim that it has any substance at all? If the person getting a reading is in no position to judge its accuracy, then who is?

Avalon, In the first part of the test, subjects failed to identify their own astrological readings. So, using your quite reasonable line of argumentation, that says more about the accuracy of astrology than it does about the participants self awareness.
yakaru, I agree. However, Carlson himself indicated that the Part I results were not a meaningful test of astrology, especially considering the "statistical fluctuation" in this section (although the test group performed according to chance, the control group actually performs 2.3 standard deviations above chance). Certainly these tests are not positive evidence for astrology -- but they're also pretty clearly not good negative evidence either.

AvalonXQ wrote:

The Carlson study was fatally flawed because of its reliance on the CPI. What is not usually reported is that the subjects themselves were not able to identify their own CPI profiles, either.

Avalon, I don’t think you understand the tests that were done. The reason the subjects were asked to select their own CPI profile had nothing to do with the CPI portion of the test (part 2); it had to do with the selection if natal charts portion (part 1). The purpose if doing this was to see if the subjects could select their own profile from two control profiles (profiles of other people). If they couldn’t do that, then the logic was that they wouldn’t be able to select a correct natal interpretation either. As they couldn’t, test part 1 was ruled “a poor test of astrology.”

This had no bearing on the CPI portion of the test (part 2), where the astrologers (not the subjects) were asked to select the correct CPI.

This is all explained on the last page of the article – last paragraph, left hand column.

This is evidence that, even if a non-psychologist has a working knowledge of a personality, he still can't successfully match it to a CPI.

Nonsense - it’s nothing of the kind. It’s evidence that people don’t know their own personality well enough to be able to select it from two controls.

For that reason alone, the Carlson study says much more about the CPI than it does about astrology.

Really? What does it say about the CPI, exactly? Please show your work.

Carlson himself indicated that the Part I results were not a meaningful test of astrology, especially considering the "statistical fluctuation" in this section (although the test group performed according to chance, the control group actually performs 2.3 standard deviations above chance).

No, not ‘especially considering the "statistical fluctuation”’ as you put it. The control group performing better than chance had nothing to do with why part 1 was disregarded. The control group didn’t even have their own chart to choose (they had three controls – ie none of the three natal interpretations were from their own actual chart). Rather, part 1 was disregarded for the reason I gave above.

Certainly these tests are not positive evidence for astrology -- but they're also pretty clearly not good negative evidence either.

Of course it is good evidence against astrology. The null hypothesis is that astrology doesn’t work – that’s the starting point. Astrology is the thing with something to prove here, and it failed the test.

But even if I were to accept your premise that using the CPI is a flawed way of testing astrology, (and if asking subjects to choose their own chart is flawed too), then astrology is most certainly bogus. I discussed this in my post Testing Astrology – Again, over two years ago. Put simply, if there is no way to test astrology, then:

  1. How were the rules of astrology derived originally? If we can’t test it to see if the rules are correct, then how would anyone have ever worked it all out? Remember, the people who made this up to start with didn’t have modern psychological tests or a culture where that performed large scale surveys of the populace.
  2. How can anyone say astrology works? On what basis do you say that astrology is accurate? How do you know?

Please click the Testing Astrology – Again link and consider the implications of what you are suggesting. I will quote again, my conclusion from that post:

There is one thing I want to be clear about here. If none of these methods are acceptable, and if astrology can’t be tested, then it means that astrology is almost certainly bogus. For one, astrology’s doubtful provenance (no known method by which it is supposed to work, no known way its rules were derived, its absurd premises), mean we need extraordinary evidence that it works. By this I mean better evidence that we demand for many other things. But we are only offered poor evidence – anecdotes that are biased by the Forer Effect and confirmation bias. And for two, if astrology can’t be tested, then clearly no one would ever have been able to work out all the detailed rules astrologers use in the first place. How would they have been able to work out the rules if there is no way of ever testing them to see if they were right?

So either astrology can be tested – so astrologers, please tell us how. (And so far it has failed all well designed tests.) Or it can’t be tested – so astrologers please tell us how its rules were derived and how you know it works.

Which is it? Because it can’t be both.

yakaru, I don't have clients and I don't do readings. I'm just thinking about what astrology means, perhaps like you, only from a different perspective.

I don't know what astrologers you're talking to, but to my knowledge astrology is not about accuracy as you seem to be thinking about it. Accuracy as I think you mean it doesn't help anyone. Astrology is about interpretation. Let's say you look at the client's chart and you see a problem. You ask about the problem to see how the client is handling it. That is their "interpretation." It's now up to you to reinterpret the problem as a challenge and an opportunity. I'm oversimplifying this of course but you get the general idea.

A birth chart for everyone is for the time of birth. It's not something one puts a percentage on. Sometimes a lot of people are born on Tuesdays at 10 am and sometimes a lot of people die together in a disaster. It is what it is.

It's a lot harder to interpret a personality than it is to interpret a specific problem. Astrologers deal with problems and dip into the personality as a resource for resolving the problems. The matching tests, which are purely about personality, are an interesting challenge to astrologers but they are not at all aligned to how astrology is actually used. I don't even think psychology is used like that either, because it deals with problems too, and I think Ertel, who is a psychologist, understands the limitations.

It's almost as if you are thinking of the constants (birth charts) as variables and the variables (interpretations) as constants. Sort of reversed. There are no laws for interpretation, only guidelines.

Big Al, Isn't it true that the more we examine the past the less dim and distant it becomes? It has taken a long time to understand how the pyramids could have been built. One thing we knew for certain was that their building had to have been possible, because they're there. Unfortunately, historians and archeologists haven't taken much interest in ancient astrology.

AvalonXQ, It's kind a shock that the student subjects could not identify their own CPI profiles. Equally interesting is that the control group performed an astonishing 2.3 standard deviations above chance, while the test group did not identify their written chart interpretations any better than chance, even though, in Ertel's assessment, the astrologers were successful to some extent in the previous study in matching CPI profiles to natal charts. Carlson explains this as a "chance fluctuation" but it of course led to speculation that Carlson might have switched the data, perhaps inadvertently. Joseph Vidmar had commented on this profound anomaly, and his article is what got Ertel very interested in the Carlson study. Ertel had never heard of the Carlson study before then.

The Vidmar article is at:
http://www.astronlp.com/Carlson%20Astrology%20Experiments.html

Of course it is good evidence against astrology. The null hypothesis is that astrology doesn’t work – that’s the starting point. Astrology is the thing with something to prove here, and it failed the test.
If the test isn't properly constructed, then what the test shows is exactly nothing. It's not evidence that astrology works, but it's also not evidence that astrology fails to work. It's no evidence at all. If poorly-constructed tests support their null hypotheses, I'll disprove any principle you want to mention with a poorly-constructed test, label myself a successful skeptic, and call it a day.
Of course it is good evidence against astrology. The null hypothesis is that astrology doesn’t work – that’s the starting point. Astrology is the thing with something to prove here, and it failed the test.
I agree. If the effects of astrology cannot be meaningfully seen in a well-constructed test, or if constructing such a test is impossible, then astrology almost certainly doesn't work. But you're making a pretty big leap, there. What I'm claiming is that this study is flawed because it relies on the accuracy of the CPI as judged by non-psychologists; I'm saying nothing about whether astrology can be tested at all. It may be possible to test astrology using a protocol similar to this one, but to use this sort of protocol, we need an objective measure of personality. The CPI didn't/doesn't do the job. There are plenty of perfectly sound studies against astrology, are there not? I would certainly expect so, since it's unsupported woo. So stop investing in this one, which is fatally flawed.

If the test isn't properly constructed, then what the test shows is exactly nothing.

You haven’t shown that this test was improperly constructed.

What I'm claiming is that this study is flawed because it relies on the accuracy of the CPI as judged by non-psychologists;

Yes, I know you are claiming this, but you have not demonstrated it as a fact. The CPI was suggested by the astrologers who were all chosen for their background in psychology and their familiarity with the CPI. There is nothing in this test – nothing – that shows the CPI is unreliable.

I'm saying nothing about whether astrology can be tested at all. It may be possible to test astrology using a protocol similar to this one, but to use this sort of protocol, we need an objective measure of personality. The CPI didn't/doesn't do the job.

What would you use then?

There are plenty of perfectly sound studies against astrology, are there not? I would certainly expect so, since it's unsupported woo. So stop investing in this one, which is fatally flawed.

So you keep claiming. Still waiting for you to demonstrate why it was “fatally flawed.”

AvalonXQ, It's kind a shock that the student subjects could not identify their own CPI profiles. Equally interesting is that the control group performed an astonishing 2.3 standard deviations above chance, while the test group did not identify their written chart interpretations any better than chance,

Not that interesting really, and certainly irrelevant to the portion of the CPI test where astrologers tried to pick the correct CPI as they said in advance they would be able to do.

even though, in Ertel's assessment, the astrologers were successful to some extent in the previous study in matching CPI profiles to natal charts. Carlson explains this as a "chance fluctuation" but it of course led to speculation that Carlson might have switched the data, perhaps inadvertently.

Any evidence he actually did this?

Thanks for answering, Ken.

You say:

Astrologers deal with problems and dip into the personality as a resource for resolving the problems.

Yes exactly. If that is all you are claiming, then everything is cleared up.

Astrology is an arbitrary system for generating information, from which useful information may be selected (cherry-picked). A bit of information only has significance in relation to some objective criterion, like a particular problem, or question of some kind.

Nothing in astrology successfully indicates which bits of information are more important in a particular person's chart. Remove the subject, and there is no more any successful criterion for information selection. The chart itself contains the same amount of potentially accurate information as a randomly selected other person's chart.

So no astrologer should be pretending that it matters which page of the ephemeris you use to prepare a chart. No one should claim that the information it produces is somehow influenced by hidden cosmological processes.

And of course, the stars and planets aren't even in the position the ephemeris says they are anyway. If the position of the sun at my birth really made me act as if under the governance of Aries, it would really be a shock to everyone, including astrologers, as the sun was in Pisces when I was born.

(Currently, the sun is in Ophiucus, not in Sagittarius.)

Big Al, Isn't it true that the more we examine the past the less dim and distant it becomes? It has taken a long time to understand how the pyramids could have been built. One thing we knew for certain was that their building had to have been possible, because they're there. Unfortunately, historians and archeologists haven't taken much interest in ancient astrology.

If astrology were a tenth as vast, impressive, in-yer-face, visible, measurable, tangible, shadow-casting and eye-catching as the Egyptian pyramids, you might just have half an analogy here, AvalonXQ.

I have never been to Egypt or seen the pyramids first-hand, but the evidence is quite overwhelming enough without the need to do so.

And the Egyptian dynastic periods aren't nearly as dim as they are distant, since the Egyptians pretty much wrote everything down, and we can now read hieroglyphics, This is how we now know that the pyramid workers weren't slaves, and we know what they ate from the work group victualling bills.

I don't know the ins and outs of how they were made, although it's plain that ropes and cedar rollers played a large part (Egypt had a flourishing lumber trade with Lebanon at the time).

The Egyptians wrote quite a lot down about building the pyramids: and baking; and stonecutting; and weaving; and marketing slaves; and mummifying pharaohs; and carrying out trials...

They also "did" astrology - but there's bugger-all written down about just how they did that, let alone how they worked it out. The Romans, too.

Why should I therefore class astrology as anyhwere near as enigmatic a puzzle as, say, the origin of dark energy (which, BTW, was not predicted and which genuinely is turning cosmology inside out - but most scientists are loving it because it gives them something new to investigate)?

But there's plenty of evidence for it, so I'm firmly and properly in the "I don't know how it works, but it looks like it does."

The issue may in the end be shown not to be down to "dark energy" - perhaps the constant of gravity changes over time, perhaps the Higgs boson has a half-life... but a strange effect is afoot, and it's undeniable: the expansion of the Universe is not being slowed by gravity as much as General Relativity implies.

Astrology just doesn't have anywhere near enough evidence to make me believe there's a phenomenon to investigate, let alone yearn to investigate it.

Big Al: That was Ken, not me. I haven't been making an argument for astrology; I've just been attacking the Carlson study.

My apologies, Avalon.

Avalon, You said:

"There are plenty of perfectly sound studies against astrology, are there not? I would certainly expect so, since it's unsupported woo. So stop investing in this one, which is fatally flawed."

The problem is there plenty of bad studies and few "sound" ones on both sides. The case against astrology is represented by Carlson, the Astrotest by Nanninga, and the time twins study by Dean and Kelly. That's really about all there is. If the Carlson study erodes away, as I believe is happening, it is a big deal.

Ken, in your "case against astrology" list, you left out a few things:

  • The null hypothesis
  • The lack of any physically plausible mechanism
  • The arbitrariness of constellation and planet designations, which are key to Astrology
  • The well-studied Forer Effect, which uncannily resembles people's responses to astrological readings
  • The well-studied process of Cold Reading, which uncannily resembles astrological readings
  • Astrology's total lack of provenance

And so forth. I just thought you might want to be complete.

The fact that different cultures ascribe different attributes to constellations, depending on what arbitrary dot-pictures they see in them (after all, is the constellation of Taurus really a knock-down unmistakable image of a bull? I don't think so.

Ken, you've ignored all of my questions. You can't answer them, can you.

Not only is astrology missing results and a plausible mechanism, but if positive results were even found, it would actually disprove astrology.

I worded it poorly in my last comment, so I'll try and put it more clearly.

I was born on April 14, so that makes me an Aries. But the sun was actually in Pisces at that time. Nicely illustrated in the graphic on this page - click to enlarge.

This actually means that astrology does not even qualify as a null hypothesis. It is already disproven.

Here is what astrology would look like if it included the positions of the the stars and planets as according to actual reality, instead of according to their ephemeris.

Capricorn: January 20th to February 16th
Aquarius: February 17th to March 11th
Pisces: March 12th to April 18th
Aries: April 19th to May 13th
Taurus: May 14th to June 21st
Gemini: June 22nd to July 20th
Cancer: July 21st to August 10th
Leo: August 11th to September 16th
Virgo: September 17th to October 30th
Libra: October 31st to November 23rd
Scorpio: November 24th to November 29th
Ophiuchus: November 30th to December 17th
Sagittarius: December 18th to January 19th

So does astrology have anything to do with the stars? Or is it ephemeris-based numerology?

Whatever it is, until astrologers deal with this, they don't even deserve the status of a pseudo-science.

Correction:
Here is the link to the Aries/Pisces graphic.

http://www.livescience.com/php/multimedia/imagedisplay/img_display.php?pic=f2-FirstPointof2007C-02.jpg&cap=Figure+2.+If+you+were+born+between+March+21+and+April+19%2C+your+astrological+sign+is+said+to+be+Aries.+But+this+was+only+true+for+a+while%2C+back+when+the+system+was+set+up+in+600+BC.+Today%2C+the+Sun+is+no+longer+within+the+constellation+of+Aries+during+much+of+that+period.+From+March+11+to+April+18%2C+the+Sun+is+actually+in+the+constellation+of+Pisces%21&title=Your+Astrological+Sign+May+Not+Be+What+You+Think+It+Is&title=Your%20Astrological%20Sign%20May%20Not%20Be%20What%20You%20Think%20It%20Is

But, of course, it was astrologers rather than so-called "sy-un-tists" who spotted that their near-100% accuracy had been suffering a little over the last few centuries, wasn't it?

Tell me it was the astrologers! Tell me!

Musta been, Al! I mean scientists have an irrational fear of astrology, don't they. No doubt astrologers noticed that their readings were starting to get a bit off, so they took on the arduous task of walking outside and looking up.

Yup, those narrow-minded egotistical scientists with their conservative materialistic mind-set were probably too busy evading the next paradigm shift.

Well, I can't argue with you there, yakaru. We all know how scientists have their eyes shut, clinging to unfounded ancient beliefs and refusing to change their closed minds in the face of new evidence.

It's those open-minded alties who recognise the flaws in ancient ways of thinking and modify their views to conform with the latest experimental findings.

Oh, my FSM, I can't believe I said that without my face cracking...

Yup, the conservative scientists are just stuck in a rut, doing all these narrow-minded conservative things...like mapping the genome, cloning, extracting Neanderthal DNA, flying around in space ships....

If only they would open their minds, the world would be a better place: homeopathic remedies would finally be accepted and global health would improve to the same level as it would if placebos were universally available. Astrology would become accepted psychological practice and mental health would improve to exactly the same degree as if people were given a randomly chosen psychological profile. Telepathy would be taught in schools, and children would be able to read minds with exactly the same level of success as if they would if they were guessing....

A golden age indeed it would be.

@SKEPTICO: If You want to test astrology, start to study it by yourself. I think taht is the best way to find out whether astrology is useful or not.

soband1,
Read the article. Read the comment thread. Then either
a) Realise you have been fooling yourself
or
b) Try to address the issue.

Just because its so subtle, it doesn't mean people can't find it. People who search often find many a subtle thing, Big Al.

Samuel: And people often see subtle effects where no effects exist at all. That's what blinded tests are used to determine.
If you can't see it in a properly blinded test, chances are very good that you were just making it up.

Lifescape astrology principles have been defined in 2010.

now we are doing extensive testing of these principles .

We could get data of Jayant v Narlikar test of astrology and we have shown correlation with venus.

in case of accidents, we have shown correlation for astrodatabank data.

people interested in testing astrology may send their data to us and we will add explanations and charts

we do not follow convetional astrology.
lifescape astrology is easily testable by even a layman. success rate of above 90% is easy to see.

To self-pimp, and I apologize for doing so, I've had a "debate" raging on for well over a month over at my blog ( here: http://www.lousycanuck.ca/?p=3898 ). Ken McRitchie just made an appearance, and an astrologer by the name of Robert Currie has been actively using the same arguments as Ertel. At that, he has made the point that he was aware of Skeptico's problems with Ertel's lines of argumentation, but is using them nonetheless. If you were interested in the fight I've got going on, I would gladly welcome the exposure, as the astrologers were at one point seriously outnumbering the skeptics in the discussion.

The larger problem I have with my own fight is that I'm not particularly versed in the scientific study outside of prima facie reading, nor had I even heard of Carlson before I started researching to write my logical disassembly of the concept of astrology. Robert Currie claims that his positive evidence for astrology's validity is actually that Carlson study. I normally hate to ask for help, but since you folks are infinitely better versed in the study, please do help out.

Thanks!

Actually Jason, having read through that (long) comment thread this morning, I think you're doing a pretty good job yourself already. I can't think of much that I would have added. I'm also not entirely sure what Robert Currie is getting at, but (as you pointed out in one of your replies) even Ertel admits that his re-analysis of the Carson study doesn't reject the null. Of course, if you look at complex and possibly ambiguous data long enough you can probably find something to support any position, but that isn't how science is done.

What we’re left with, again, is proponents of astrology at best claiming that this test or that test doesn’t disprove astrology.  All sorts of problems are claimed with measuring the results: choice of two or three, not enough people being tested, CPI and/or subject choice unreliable, etc. And what is never explained is, given these difficulties, how did the detailed rules of astrology even get figured out in the first place? Where is the experimental data that shows how astrology’s rules were originally derived? Because if they did it 2-3,000 years ago, what can’t they do it now? 

Ertel would be happy with more tests of the Carlson type (actually known as a "Vernon Clark test," please look that up). So the more discussion on Carlson, the better. More of these tests need to be done.

All Ertel does is point out that Carlson does not follow his own test protocol but changes it without giving a reason. When the success criteria, which Carlson states in his article, is followed, the astrologers score at marginal significance (p = .054) for the 3-choice and even better for the 10-point ranking test (p = .037). It's pretty straightforward and there's no complex, ambiguous data about it. That's all there is, and no skeptic, including Carlson, has been able to refute Ertel's examination.

The best Carlson can come up with at this point is that the astrologers did not perform as well as they thought they would. But in a fair test, maybe they would have. It would be great to do the test again, fairly.

This issue is not going away, so I suggest that both of you, Skeptico and Jason, buy Ertel's article and examine it. I don't see how you can call yourselves skeptics unless you become familiar with the positive astrological studies and this is the place you should begin. This is big.

If you can see where Carlson went wrong and if you do not find fault with Ertel, then you will find it a lot easier to work through your many false assumptions about astrology. Many skeptics like yourselves keep repeating the same things, which are wrong but you don't listen anyway when someone tries to correct you. This Carlson test is important and it just might make you listen. There are other studies even bigger and more important than Carlson's study, but they were not done by a skeptic. So this is where you should start. Defend Carlson. The ball is in your court.

Sorry, the 3-choice test is the "ranking" test and the 10-point test is the "rating" test.

Both methods are good for astrology testing, though rating seems a little more powerful. A serious problem was that in his study Carlson threw out the volunteers' rating tests without showing the data, claiming that the results did not vary enough within each interpretation.

Yet, this type of invariance could be explained if there were good astrologers and not so good astrologers, which would have made that particular data very interesting indeed, as you can well imagine.

His reason for discarding this part of the study does not hold water and the results of the volunteer rating test should have been published.

Hello Ken.

I'm still waiting for your response to my last comment from Dec 3 above.

Yakaru,

Okay, so you are making some claims here, not me. What sort of evidence do you have to back them up? Here's what you said:

=======
1. Astrology is an arbitrary system for generating information, from which useful information may be selected (cherry-picked). A bit of information only has significance in relation to some objective criterion, like a particular problem, or question of some kind.

2. Nothing in astrology successfully indicates which bits of information are more important in a particular person's chart. Remove the subject, and there is no more any successful criterion for information selection. The chart itself contains the same amount of potentially accurate information as a randomly selected other person's chart.

3. So no astrologer should be pretending that it matters which page of the ephemeris you use to prepare a chart. No one should claim that the information it produces is somehow influenced by hidden cosmological processes.

4. And of course, the stars and planets aren't even in the position the ephemeris says they are anyway. If the position of the sun at my birth really made me act as if under the governance of Aries, it would really be a shock to everyone, including astrologers, as the sun was in Pisces when I was born.
=======

1. If astrology is an arbitrary information generating system, then let's say a computer generated interpretation would be no different than random selections. Can you demonstrate this? How can you be certain that the useful information is "cherry picked"? Do you have a demonstration of this type of selection versus others? What's wrong with cherry picking?

2. If nothing in astrology successfully indicates what information is more important in a person's chart, then what have you got that demonstrates this? How do you know what is important versus not important? If you remove the subject from anything (forget about the chart for the moment) then what is your evaluation of "success"? How can you be certain that a genuine chart is as accurate as a random chart? What studies have demonstrated this?

3. What makes you so sure that astrologers are "pretending" to say something? Do you have confessions or some other objective way of ascertaining this? What does it matter if there are unknown processes, or no processes, between the map of the sky and the person? Are you trying to say there can be no evaluation possible? How can you demonstrate that?

4. Are you denying that the positions of planets and stars cannot be measured in celestial longitude (which is what you find in an ephemeris)? How are you going to demonstrate that? If you claim that your Sun was in Pisces by celestial longitude (the system used in Western Astrology), then maybe it would be a shock to everyone that your Sun position makes you act as if it was in Aries by celestial longitude. I might be inclined to agree with you simply because it seems like a contradiction, but again, you would need to demonstrate that.

Skeptico,

I've responded to Yakaru, so now we are waiting for your response to my post yesterday addressed to you and Jason. You've been pretty quiet about the Carlson study, and, like I've said, the ball is in your court to defend Carlson. Maybe silence is your consent.

Sorry Ken, the ball is not in my court. The burden of proof is (as it has always been) with supporters of astrology, to show that their ridiculous made-up magic pictures in the sky fortune telling system is something other than made up nonsense. Good luck with that. You’ve had, what, 3,000 years? And still no explanation of how all the detailed rules of astrology were derived. If the Babylonians (or was it the Greeks?) did it thousands of years ago, why can’t you replicate their work now? All you have is sniping at one study you don’t like. Pathetic.

Ken, you have done a fair job of distilling my criticisms of astrology. What you have failed to do is answer them.

Ok, I'll spell it out:

1. Astrology is a system for generating information. You are claiming that this information is not random, rather of higher value than randomly generated information, but you have shown no positive results, provided no plausible mechanism, are have no idea how the system was initially developed so are bound to accept it purely on authority. These are serious flaws in your system, and these are good reason to judge astrology a pseudo-science.

I have provided a more plausible alternative explanation, which doesn't postulate any unknown cosmic forces, doesn't contradict the known science, is supported by a couple of hundred years of scientific research into human physiological and psychological development, none of which even hints at planetary positions (or goat's entrails, or Greek gods or omens) affecting human behaviour in any way.

2. What is wrong with cherrypicking is that its necessity demonstrates that astrologers have no other way of sorting through the masses of information thrown up by a chart. This is why the properly run studies return success rates identical with random chance.

That is my demonstration, and it is further confirmed by your acceptance of cherrypicking as legitimate.

3. Astrologers may not be pretending; they may be just plain ignorant or uninterested. In any case, it is clear that the study being discussed here used a measure for success way below that which astrologers claim for astrology. Or do astrologers tell their clients to that they are paying for something which may be 1 or 2% more accurate than a random reading?

4. There are two ways of measuring planetary positions. It is up to you to explain on what grounds astrologers prefer one over the other. That will be difficult, because you don't know the answer, and are bound to accept it on ancient authority.

These are the criticisms you have to answer; this is the work you have failed to do.

Skeptico,

Sorry to bring this up again, but this blog thread is about the Carlson study. Ertel has found that the astrologers performed significantly better than chance. This means that the Carlson study, when followed according to Carlson's stated protocol, actually supports astrology. How do you defend Carlson's study against the criticism offered by Ertel? Do you know something that the rest of us, including Carlson, do not?

Ken:

I covered this in my original post, that addressed Ertel's study as rerported by you. Also in a couple of my comments back then. Please read what I wrote over a year ago: it's still valid. The astrologers in both tests performed exactly as chance would predict, Ertel's baseless acusations notwithstanding.

Yakaru,

I was careful to make no claims of my own in my response to you (Aug 23), but only to question you on how you can be so certain of the many claims you made. You have some additional claims. Let's focus on a few of your key claims and bring this back to Carlson's study, which is what this discussion is about.

1. You claim that astrology has "no plausible mechanism" and you claim to know a “plausible alternative” (though you don’t say what it is) that “doesn't contradict the known science.” If I understand you correctly, you insist on a mechanism because mechanisms are used to evaluate effects, and it seems (though it’s not clear) that you are claiming that without a mechanism it is not possible to measure effects. Let’s look at that. Science routinely uses statistics for evaluations where there are no available mechanisms. Most modern sciences are entirely dependent on statistics. To insist on a mechanism is to deny not only astrology but also all studies that rely on statistical evaluations, which includes just about all the new science developed over the last 100 years. Is there some good reason why you want to contradict all that good science and must have a mechanism?

Statistical evaluations were more than acceptable with Carlson. He claimed that his double-blind study would test the "fundamental thesis of astrology." He discovered, let's say "by accident" because that's the way most scientific discoveries happen, that, despite the tests being unfair and the deck stacked against them, the astrologers could match and rate CPI choices with charts at a statistically significant rate (p = .054, ES .15 and p = .037, ES .10). The Carlson study has demonstrated that science supports the fundamental thesis of astrology. QED

2. You claim that cherry picking is "illegitimate" (for what is not quite clear) and that it is the astrologers doing the cherry picking, not their clients (contrary to the usual skeptical claim). I think what you're referring to is the Forer effect ( aka Barnum effect). The claim is that you can present choices that anyone would agree to, so the interpretation is no more specific to one person than another. There's “something for everyone” to cherry pick and supposedly one interpretation is as good as another. That is your belief. You claim that "properly run studies return success rates identical with random chance." That is your belief.

This Forer effect belief is challenged by presenting multiple samples (not just one) in a double-blind test and asking the astrologers to match and rate them. If the astrologers were just cherry picking then, as you said, the success rate would be "identical with random chance" and there would be a Forer effect. However, it turns out, as we saw in point 1, that the astrologers could predict how the volunteer subjects would pick their own very best cherries (in their CPI responses) at a rate that was significantly better than chance frequency. The Carlson study refutes the random cherry picking hypothesis. The cherry picking is not random but statistically predictable.

3. Thank you Yakaru by retracting your claim that astrologers are "pretending." The Carlson study sets a new level for astrological effect sizes (ES .10 and ES .15), even though the test was unfairly stacked against the astrologers. Now that the problems with the Carlson study are known, it would be possible to design a much fairer test. It remains to be seen whether effect sizes will increase if the astrologers are given a fair chance, but that would be the general expectation.

4. You claim there are two ways to measure planetary positions. Unless you are referring to right ascension versus celestial longitude, I think you are referring to what is one of the interpretive frames of reference used in astrology, the zodiac, whether it is the Tropical Zodiac, or one of the Sidereal Zodiacs (there are actually several).

Western astrology has developed the use of the Tropical Zodiac over the past 2000 years. The advantage of the Tropical Zodiac is that it is based on the natural symmetries of the solstices and equinoxes. It is a natural frame of reference. The problem with the Sidereal Zodiacs is that no one can say where they begin or end. The astrologers/astronomers 2000 years ago in the "West" realized that the "fixed stars" actually move and were creeping away from the solstices and equinoxes. They improved their methods to measure with reference to the solstices and equinoxes alone without regard to the fixed stars.

The signs of the zodiac should not be confused with the constellations that have the same names. The Tropical Zodiac is based on natural symmetries. The constellations, some of which have the same names as the signs, are imaginary. The signs of the Tropical Zodiac could be renamed with better names than they have now. The fact that zodiac names are the same as constellation names is not something to base any scientific judgment on. There has been a lot of confusion over this. I don’t know why, but I hope this clears it up for you. It would be better not to harp on things that were understood and fixed in the very ancient past.

Sorry Ken, I can’t let those flat out lies stand:

despite the tests being unfair and the deck stacked against them, the astrologers could match and rate CPI choices with charts at a statistically significant rate (p = .054, ES .15 and p = .037, ES .10). The Carlson study has demonstrated that science supports the fundamental thesis of astrology. QED

Jesus Ken, read the study.  The astrologers picked the correct CPI at the exact rate predicted by random chance – one third. Simply stating that the study was a success for astrology does not suddenly change it from the failure (for astrology) that it was.

Skeptico,

Carlson (and Nature) made an error in the original publication which Ertel has pointed out last year and no one, including Carlson, Dean, nor anyone else, has challenged. Carlson states his protocol (twice) then changed it when he analyzes his results. If the protocol (which is correct) is followed the correct results support astrology.

A scientific study is not written in stone, but open to discourse. In this case the discourse has uncovered a problem that passed peer review and is now known. To correctly understand the study, you need to read not only the original but the whole discourse that followed and that includes Ertel and anyone else who can shed more light on the study.

In case you haven't read the published discourse, what Carlson actually found is that the astrologers succeeded at rating and ranking the CPIs at a rate significantly better than chance.

Skeptico,

Just to anticipate your next argument about JSE, which carried Ertel's response, being a small circulation science journal, it's a small world in any special research like this and the science journal where an article is published is not so important. All the concerned people, Carlson, Dean, Vidmar, Kelly, Hamilton, Sullivan, myself and others knew about it long before it was published, and most of us contributed in one way or another.

A reader just sent me Ertel’s report (thanks Darth Cynic).  I’ll look at it in more detail this weekend, but I’m afraid I can’t let this latest McRitchie / Ertel nonsense stand without the facts being known.  Here’s Ken:

Carlson (and Nature) made an error in the original publication which Ertel has pointed out last year and no one, including Carlson, Dean, nor anyone else, has challenged. Carlson states his protocol (twice) then changed it when he analyzes his results. If the protocol (which is correct) is followed the correct results support astrology.

Wrong.  Carlson did not change his protocol.  He stated (page 425):

Before the data had been analyzed, we had decided to test to see if the astrologers could select the correct CPI profile as either their first or second choice at a higher than expected rate. [Note: italics in original]

Get that?  Either / or.  Ertel, rather sneakily, tries to suggest that Carlson planned to combine the first and second results.  Combine – means add both together.  Different from either/or, which means test one then test another.  Here is Ertel (page 129):

Given the three-choice data, Carlson should have analyzed them, at least, by combining first and second choice frequencies. He himself had intended, “before the data had been analyzed,” to combine them: “We had decided to test to see if the astrologers could select the correct CPI profile as either their first or second choice at a higher than expected rate…” (p. 425). He ignored his own protocol without giving reasons.

[Note: italics for word “or” are gone when Ertel quotes Carlson.]


Note how Ertel states his opinion that the study should have been to combine the first and second places, and then immediately takes an out of context quote by Carlson (“before the data had been analyzed,” ) and places is immediately after Ertel’s opinion of what the test should have been, to make it appear that Carlson had indeed intended to combine the first and second choices.  He then quotes the “..or second choice…” bit later (without the emphasis on “or”) on the assumption that no one would notice that this was not what Ertel was saying Carlson had intended to do.

I didn’t want to do this before for copyright reasons, but I think Shawn will forgive me that I have now placed on this blog, the Shawn Carlson Double Blinn Test of Astrology.  Even on my lousy scanned copy of the photocopy Nature sent me, you can still see the italics in “or”.  Check it – on the last page, mid column, second full paragraph.

I have spoken to a couple of statisticians about this, and I am advised there is no reason that the combined first and second choice should necessarily be a better test than showing them separately.  This is most likely why Ertel has to pretend that Carlson always intended to do the study Ertel’s way, and that he changed his protocol “without giving reasons.”  Well of course he didn’t need to give a reason – he didn’t change anything.  Ertel’s dishonesty just on this one little point does little for his credibility.

And at this point I should emphasize, on Carlson’s test (the actual one, not Ertel’s fantasy test), on both first and second choices, in the exact way Carlson designed the test at the beginning, the astrologers performed exactly to chance.  Sorry Ken.

In reality, Ertel data mined the study to try to find a way, any way possible, that the data could show astrology passing the test.  Of course, that isn’t the way science is done. 

In case you haven't read the published discourse, what Carlson actually found is that the astrologers succeeded at rating and ranking the CPIs at a rate significantly better than chance.

As I clearly showed above - no he didn’t.  The subjects chose their correct profile by one third – exactly the result  guessing would produce. The astrologers chose the correct CPI by one third – exactly the result  guessing would produce. They also chose the second place by one third – exactly the result  guessing would produce.  All we have here is a perfect example of the problems of trying to honestly test a pseudoscience such as astrology.  Even with a meticulously designed test, with the lead tester bending over backwards to be fair to astrology, the best astrologers recommended by an astrology group, peer reviewed and published in Nature that shows astrology performing exactly to chance, the unsinkable rubber ducks of astrology distort the results, cherry pick, data mine and flat out make stuff up to claim that the study showed the exact opposite of what it did, actually, show. Sometimes I wonder why I bother.

I’ll read the rest of Ertel’s paper this weekend, but based on what I’ve seen so far, I’m not optimistic it will amount to much.

Skeptico,

The first or second choice means the two choices combined. This is what Carlson tries to show by stating the rate of the third choice and declaring the combined first two choices to be a rate less than chance. He was calling the result on the first two choices combined, which is how a test of this unusual design has to be analyzed. But he didn't actually present the figure for the first two combined, but leaves it to the assumption of the reader.

The even better result for the astrologers was for the 10-point rating test, which had nothing to do with the 3-choice test and was performed first.

I'm glad to hear that you are reading both papers.

"The first or second choice means the two choices combined."

So, when my brekkie special is - three eggs, toast, home fries, fruit and bacon or sausage or ham - I've been ripped off when they only bring me bacon?

Or is it (and is it?) because the restaurant doesn't italicize the ors?

And am I just confused?

And an unsinkable rubber duck of astrology replies:

The first or second choice means the two choices combined.

That’s it?  That’s your best shot after 24 hours to think about it? “Either … or” means “both”? Really?

Don’t be silly - obviously it doesn’t. It means what it says – what proportion chose the correct CPI as their first choice, and what proportion chose the correct CPI as their second choice?  That’s exactly what it means and it’s exactly what Carlson did.

“The two choices combined” would be “the first and second choices combined.” You need to take some lessons in English reading comprehension.

Right here anyone can see how pointless it is to try to examine astrology rationally to see if it works. No matter how carefully a tester (Carlson in this case) explains what he did, and no matter how clear the result, it makes no difference because an unsinkable rubber duck who will never accept that astrology isn’t real no matter what the evidence, will just state that the experiment showed the exact opposite of what it did, in fact, show. Whatever the actual result, Ken McRitchie will announce it supports astrology. Thankfully, I believe most people will see that it doesn’t.

Skeptico,

Robert Currey has given you a more detailed account of Carlson's intent to test both first and second choices together. Please see his response to you on Jason Thibeault's blog, which you haven't answered.
http://www.lousycanuck.ca/?p=3898&cpage=9#comment-11740

You can post your response there. I'll be interested to see what you have to say in addressing his 4 main points and 5 other points to consider.

Keep in mind that the 3-choice ranking test was just one of 4 test that Carlson reports on. I'd also like to know your response to his 10-point rating test, which turns out to have had an even better result for the astrologers. You haven't commented on it yet.

Skeptico,

As you will see on Jason's site by the link above, Robert's fourth argument is a reiteration of my observation earlier, which you have failed to argue against. I'll repeat it here for you.

Carlson tries to show the results of the combined first two choices by stating the rate of the third choice to be at chance rate and incorrectly declaring the combined first two choices to be at chance rate. He didn't actually present the figure for the first two combined, but leaves it to the assumption of the reader.

Also, I would like you to argue against the 10-point rating test, where the astrologers had an even a better result.

Carlson tries to show the results of the combined first two choices by stating the rate of the third choice to be at chance rate and incorrectly declaring the combined first two choices to be at chance rate.

Well why on Earth wouldn’t it be?

Skeptico,

You're not alone on that. That is an assumption that people made for 25 years. Carlson designed an unusual three-choice test, which is less powerful than a two-choice test (which Clark had already done, favoring the astrologers). Why? You'll have to dig deeper into Carlson to find out.

Carlson says the result will depend on the 1st and 2nd choices, which are the two the astrologers choose. Because of the sample demographics there was a good chance the two could be very similar, and some astrologers quit when they found that, declaring the test impossible. This is why he had to count them both if he was going to persist with a 3-choice test.

Carlson demonstrates that each of the three choices is at a chance rate, 1st, 2nd, 3rd. Not there, not there, not there. Then instead of combining the first two choices and calculating the rate, he directs attention to the third choice, which, as we have already seen, was demonstrated to be at chance rate. Thus he proclaims the first two to be at chance rate.

It seems to make sense, but this of course is not how statistics is evaluated. You cannot assume the rate. You need to calculate it. The rate at which the first two choices were made was actually marginally significant (p = .054).

For the 10-point rating test, Carlson similarly divides it into three, but there actually was no three, there were 10. This test was done before the 3-choice test. The astrologers were correct at a rate of p = .037.

The results favor the astrologers despite the test was unfair. In a fair test, perhaps they could have done much better.

Skeptico, you're a critical thinker, don't you find the sort of scientific investigation that has now surrounded this study interesting?

You seem to be desperate to undermine this scientific evidence which clearly favors astrology. However, you cannot cover-up data that shows statistical significance under scientific conditions even if it is contrary to your deeply held beliefs.

Ken,
I will soon reply to your earlier comment to me above -- but maybe elsewhere because I think I'm pushing the thread too far off topic.

I will say this, though. If astrology actually worked, you would be pointing to a myriad of other studies clearly demonstrating it, be implementing quality-control within the astrological community, railing against atrociously designed astrological studies of the kind published in the JSE and elsewhere and generally trying to raise the standard of astrological discourse among astrologers, and working with psychologists with your practical help for child raising and educational measures.

And a whole lot more. But you are not doing that, because astrology doesn't measure up to the standard required for that.

Instead you are reduced to quibling over finer points of a 25 year old study and just trying to wear down opponents with nitpicking.

Yakaru,

If your responses have to do with the Carlson study, I'd like to see them here in this blog, which is on this study. There is no reason to go off topic.

There are other studies that support astrology, but for the reason that the Carlson study, published in Nature, has been at the very pinicle of skeptical evidence, it is the key study and has earned the scrutiny it is now getting. Everyone will learn from it and this will raise the standard as long as there is discourse that stays on topic and is well informed in all aspects of this very interesting study. Of course, you must read what came before (Carlson and Ertel) before you should continue.

If you'd rather not read the scientific studies, then you will have to give up your claim that your deeply held beliefs are actually scientific. Until you offer informed arguments on why the tests went the way they did, and explain the results, you will remain where you are now, on the outside of science with regard to astrology.

Uhuh, so science supports astrology, and I have a deeply held belief that it doesn't.

Even if your interpretation of the Carlson study were true, it would not mean science supports astrology. All it would mean is that one study (in which the researcher himself must have made serious errors) has returned anomolous results. That does not constitute scientific support.

Yakaru,

There are other studies that support astrology that are better than Carlson's. Yet they were not done by a skeptic and not published in Nature. This places Carlson's study at the highest scientific importance.

When it comes to a study like this, I prefer to study the relevant articles and check the scientific facts rather than make assumptions. Of course, like everyone, you are entitled to your personal beliefs.

Carlson did make errors. However, the errors all favored the null hypothesis, which put the astrologers at a distinct disadvantage.

The results are not anomolous and I am not presenting the Carlson study with any interpretation. That was done by Carlson himself (and Nature) when he stated, "We decided to test what we shall call (for simplicity) the 'fundamental thesis of natal astrology'" (Carlson, 1985: 419). The results are explained as being astrological and are supported by this scientific study.

Ken,

There is no requirement that a scientific study be performed by a skeptic - merely that all necessary steps are taken to ensure the investigator and the subjects are not fooling themselves or each other.

This usually means a double-blind trial.

Where are these other non-skeptical studies? If they are duly disciplined and methodically rigorous, there is no reason why they should not be published in Nature.

"Better" in this sense usually means "more rigorous", not "more favourable to" - is this the sense in which you mean these tests are better than Carlson's?

You say all the errors "all fovoured the null hypothesis". What do you mean - that Carlson always made mistakes that covered up real evidence of astrology? If so, how do you know about it?

Ken:
...This places Carlson's study at the highest scientific importance.

You are way over-estimating its importance in declaring astrology pseudo-science. The rest of science, the entire edifices of physics, chemistry, biology and the behavioural sciences, weigh in against astrology. You seem to have no idea what a herculean task you have taken on in trying to overturn all that in one big swipe.

Anyway, I have looked at enough studies by astrologers (laughably pathetic) and by serious researchers and find no reason to drop my belief that what is written in the textbooks is basically right, as far as they claim to be.

However, like Big Al, I'd also be prepared to read the "better" studies, though I expect shenannigans.

Hi Al,

As we've been discussing on this blog, numerous problems and flaws in Carlson's study have been recognized. These problems covered up or obscured what Carlson actually found, which was significant support (p = .054 and p = .037) for astrology according to his (and Nature's) stated objective to test "the fundamental thesis of natal astrology".

The main problem in one test is that Carlson changed his stated analysis protocol without giving a reason. If the protocol is followed, astrology is supported. In another test he analyzed results based on irrelevant groups that were used in a subsequent test. If the results are analyzed without the false groupings, astrology is supported.

If you cannot see these problems for yourself in the original Nature article, it might help for you to read the detailed scientific reassessment of this study in a published article by Suitbert Ertel.

Yakaru,

The Carlson study is by far the most cited evidence supposedly against astrology (12,200 google results for the search string "a double-blind test of astrology"). This puts it at the pinnacle of importance for any such study. The fact that the study actually favors astrology necessarily has a fairly substantial impact.

I invite you to argue in favor of Carlson's original conclusion and against the scientific reassessment. Only I want you to use scientific discourse and reasoning based on the facts of this study rather than your assumptions of this study based on your personal beliefs.

Ken,
You are accusing Carlson of fraud, an accusation which is taken extremely seriously by professional academics, and will be treated with utmost gravity by the editors of Nature when you contact them.

If I thought you had any kind of case, I would contact themself, not out of sympathy for astrology, but because I expect a magazine like Nature to uphold the highest standards of integrity. However, I will leave that to you, because looking at the arguments you have presented here, it is clear to me that they have been amply refuted by Skeptico, as well as Carlson as quoted here.

Don't be too surprised if the Nature editors reply along similar lines to them.

Yakaru,

I'm pointing at the "problems" and "flaws" in the study. These should not have happened and Nature should have caught them before publishing. Whether the problems were inadvertent or not is irrelevant. They could be either. What is relevant is that the article is critically read to understand the problems and what was actually found. This is the function of normal scientific discourse.

It's a lot easier to see the problems once there are pointed out. Have you read Carlson's article? Can you see the problems? Wouldn't you agree that the Carlson study actually supports astrology in accord with the stated objectives in the study?

http://muller.lbl.gov/papers/Astrology-Carlson.pdf

Ken,

There were a couple of minor procedural glitches in the study, and these are duly described and explained in the article. None of these affected the outcome. Astrologers were of course involved in every aspect of the design of the study, and if the design was faulty, they share responsibility.

Carlson took extraordinary care to make the procedures and design acceptable to astrologers. That there is still controversy suggests that astrology is simply not sufficiently developed as a theoretical system to be studied in any systematic way. In other words, you can't test a pseudo-science. Not because it's "wrong" but because it lacks an adequate theoretical framework on which to base a study.

There are too many variables and too many unknowns for a test to hold anything constant. Personally, I would say that until astrologers can come up with a testable working hypothesis, it cannot (and should not) be tested. Maybe Carlson would and others would disagree, but I find it unrealistic to attempt to simultaneously test the ridiculously long list of assumptions - that the planets have an influence, that this influence peaks at birth and then receeds, leaving an indelible mark on character and fate; that the planetary influences continue to react with this indelible mark throughout life, and even reach outside the individual and shape life circumstances for him, that each planet reacts with the others depending on its position, as well as with the particular consellation inhabiting the arbitrarily drawn 30 degree sector of sky; that each constellation is more influential than other astronomical phenomena in that sector; and so on.....

This is testing the astrologers' ability to measure the influence before the existence of the influence has even been verified or any kind of a plausible mechanism for it has been developed.

This is not to mention the differences of opinion between astrologers.

I would suggest that astrologers should be sorting all these weaknesses out amongst themselves before atttempting to storm the temple of science. If you want astrology to be recognised as a science, try step 1: start acting a bit more like scientists in relation to your own theories. (Hint: google returns 550 000 hits for "quantum astrology", including a site of yours, Ken. You guys don't really take this all that seriously, do you.)

Astrologers were of course involved in every aspect of the design of the study, and if the design was faulty, they share responsibility.

This reminds me of all the people who've failed the Million Dollar Challenge and come back saying the test was unfair. They never mention the fact that they are videotaped saying that they helped design the test and the conditions are fair prior to testing.

Ertel actually took the million dollar challenge back in 2004 in Germany. He failed and then went around complaining that the test was unfair.

That's awesome. I'm sure he too forgot to mention that he helped design the test and agreed it was fair beforehand or they wouldn't have even tested him.

I can't find anything much about it on the net, except for blithering tirades from astrologers. Randi mentioned something interesting though (in response to one of these tirades), about the way Ertel wanted to run the tests but wasn't allowed:

"The sloppy conditions that Ertel had in place for his own “controlled” test were such that I was easily able to accurately predict which numbered ping-pong ball I was about to draw from the target-randomizer-bag that he had constructed. His lack of security was appalling. But I see here no report of the incompetent protocol conducted and designed by this man…"

http://www.randi.org/site/index.php/swift-blog/169-swift-february-29-2008.html

...

Maybe we should all apply for the million dollars with our psychic abilities to predict the way others who apply for it will react when they fail.

Yakaru,

Carlson ignored the the advice of the astrologers. Some astrologers, including the most qualified in the CPI, Teresa Hamilton, quit in protest.

What I need you to look at Yakaru, has nothing to do with that or the other things you bring up. The Carlson study is important because it is so easy to understand. You do not need to know astrology, astronomy, or complex demographics to get it. It's very straightforward.

In the astrologer's 3-choice test, Carlson shows that each of the choices individually was at chance rate within a specified confidence interval. The way he tries to show that the astrologers were unable to choose the genuine CPI as first or second is tricky. He points to the third choice, which as noted was at chance rate. He thus declares that the first and second are also at chance rate. The problem with this is that they're not, and he is just assuming that they are. When actually calculated, they were chosen at a significant rate (p = .054).

In the astrologers 10-point rating test, which was performed before the 3-choice test, but presented in Carlson's paper as if it was after. Carlson unfairly groups the ratings as if they were part of the 3-choice test. The problem is that they had no dependency on the subsequent 3-choice test. When analyzed independently of the 3-choice test, the astrologers performed at a significant rate (p = .037).

Here's a link to the Carlson study again, in case you want to read it again.
http://muller.lbl.gov/papers/Astrology-Carlson.pdf

These are the actual results and they unambiguously support the astrologers in this scientific study.

Astrologers are definitely looking into quantum mechanics, which is a highly statistical science. Many are looking for similarities and believe that there are things to be learned there.

Ken:

I’ve read Ertel’s study and I will respond at some point will a full post.  But it won’t be for a while. I find it hard to work up the enthusiasm to write about this ridiculous piece of nonsense again. It’s depressing. Not to think that you might have something, because you don’t. It’s depressing to know that, once again, I apparently have to sit down and write another piece about this ridiculous pseudoscience that refuses to die, and that I know will be wasted on you anyway.

It’s like being asked again to consider that Santa Claus is real. I mean, really.  We all know the story of Santa is just made up, that there is no possible way for one man to deliver presents to all the children across the world in one night, and even if he could he’s too fat to fit down a chimney (and not all houses have chimneys anyway), plus we know that there are other explanations (stores hire actors to play Santa / parents hide the presents and leave them on Christmas Eve), for the apparent Santa events. But despite these absurdities, several people have attempted to test empirically to see if Santa exists or not, and one person did an extremely detailed, bending over backwards to be fair to the Santologers test that found, guess what, there is no Santa. And yet 25 years later, someone with an interest in proving Santa exists examined the test results again and by analyzing some of the date differently he discovered it mentions some bloke who swears he saw a man in a red suit with a white beard, and a child who insists she heard sleigh bells, and the Santologers are saying that this test now shows that Santa exists. And these are adults who we are supposed to take seriously. It’s just depressing that, again, I’m apparently going to have to try to explain to a bunch of adults that it takes more than a few anomalous reports at P = 0.05 to be extraordinary enough evidence for this extraordinary and clearly fucking ridiculous claim. I will do it at some time, though, but I just not now, or this week or probably even this month. Sorry, but tough.

In the meantime Ken, you need to stop doing two things.  (1) You need to stop claiming that Carlson’s test now supports astrology.  You certainly need to stop claiming that Carlson’s test now supports astrology “unambiguously.”  Don’t be silly.  It doesn’t. Even Ertel’s re-analysis of Carlson’s data doesn’t support astrology – read Ertel, even he doesn’t claim that. (2) You need to stop claiming that your position is somehow the “scientific” one. Science is not decided by P = 0.05 results when there is a very low (read virtually zero) prior probability of your little fairy tale (I won’t call it a hypothesis) being true. If you want to know what I mean, read Provenance. And then describe astrology’s.

Astrologers are definitely looking into quantum mechanics

Of course they are, it's the magical answer to everything wooish

Astrologers are definitely looking into quantum mechanics

No they're not. They have bypassed the "looking into" stage and jumped directly to the marketing phase - a hallmark of purest pseudo-science.

Skeptico,

A p value of .05 is considered to be significant in the social sciences. The astrologers in the Carlson test performed at a significance level of p = .054 and p = .037, which meets and exceeds the requirement. The Carlson study, published in Nature, is no doubt scientific, and the figures for the study are not ambiguous. If you can show otherwise, I'd be interested.

Any study should always provide a history of previous similar studies. Carlson failed to mention the well-known Vernon Clark studies, which supported the astrologers in a very similar study. One really should consider the Carlson study as a successful replication of the Clark study. I think that is what you would call the true provenance of this study.

The other dimension of provenance here is the removal of artifacts, which can be introduced through bias. In astrological studies there have been artifacts found on both sides of the astrological debate. In this case, we must consider the incorrect calculations that led Carlson to an incorrect conclusion, which has now been recognized, and the actual results published.

If there are any artifacts in the study that will bring it back to Carlson's original conclusions, and you have facts to support your own strongly held beliefs of provenance, ambiguity, Santa Claus, fairy tails, etc. I'd be interested to hear about them.

Ken:

  1. You’re ignoring prior probability. P=0.05 means nothing when there is no known means by which something can work, and where we know the thing was just made up. P=0.05 is only reasonable where there is some prior expectation that the thing is valid, for example testing a drug where the chemistry is known and where animal studies have shown prior likelihood of success.
  2. You’re ignoring all the other aspects (ie the main ones) of the test where astrology failed miserably.  That’s cherry picking. And no, they weren’t “incorrect calculations.” The astrologers failed to do what they said they could do.

Read my post provenance.  It is about how the rules of astrology were derived (Hint: they weren’t; they were made up.), not about some other similar lacking in provenance test.

Skeptico,

1. The Carlson study was a statistical study, not a study of mechanisms. This seems to be okay with Nature, and why wouldn't it be? It met the scientific requirements and the astrologers were supported by the facts. If you are looking for "prior probability" you can look at the Clark study, which was also a double-blind study where astrologers matched charts with people also at a significant rate. Carlson replicated it.

2. The Carlson study gave the astrologers only two tasks. They succeeded in both. You are claiming the astrologers had other tasks they failed miserably? What are they?

I read your post on provenance. Let me ask you something about your assertion, "Extraordinary claims require extraordinary evidence," which you attribute to Carl Sagan. Is this not an entirely subjective claim? Isn't it even an "extraordinary claim" in itself? Where is the extraordinary evidence that supports this extraordinary claim?

Recently Steven Hawking responded to the question of "scientific" models beyond the Big Bang, which would be extraordinary, with the following summary of what many scientists have believed for many years on counts in science:

A model is a good model if it:
1. Is elegant
2. Contains few arbitrary or adjustable elements
3. Agrees with and explains all existing observations
4. Makes detailed predictions about future observations that can disprove or falsify the model if they are not borne out.

Read more: http://www.time.com/time/arts/article/0,8599,2017262,00.html#ixzz0zNVMnxpV

Your/Sagan's claim seems extraordinary because it does not fit at all with this. So where is the "provenance" for your claim, other than the authority of Sagan? Where's his?

Let's look at the Carlson study. Some astrologers say they can match charts with CPIs. I find it interesting because I know it can be tested and want to see if they can do it. Sagan knows the claim can be tested, but he asserts the claim is extraordinary. Sagan is entitled to his belief. Either way, does it really matter? This part is just subjective.

Well, then the astrologers do it. They successfully match the charts. I do not find the evidence extraordinary. It's just a matter of scientifically testing the subjects in a blind study. Does Sagan find the evidence extraordinary? You tell me. This is subjective too. Was he expecting something else, perhaps a miracle, which would be extraordinary? He is entitled to his beliefs, but I'll just go with the science myself.

Another example might be a magic act where Sagan is overwhelmed by the extraordinary feat of sawing a lady in half and putting her back together. What has this to do with science? It's pure subjectivity.

Let me ask you something about your assertion, "Extraordinary claims require extraordinary evidence," which you attribute to Carl Sagan. Is this not an entirely subjective claim? Isn't it even an "extraordinary claim" in itself? Where is the extraordinary evidence that supports this extraordinary claim?

I think you have a fundamental misunderstanding about what that phrase actually means. To begin with Dr Sagan - who is deceased so no longer has any beliefs - was not talking specifically about astrology or these tests, it was a general statement for expectations in relation to unusual claims. For instance were I to hear a claim that someone had seen an elephant recently I would need little supporting evidence if any beyond being told. Because we all know elephants to exist, so seeing one is not in general unusual. Now if I were to hear a claim that aliens had abducted someone I would require a significant level of evidence to prove this claim for we neither conclusively know aliens exist, if they do that they have or do come to Terra and if so that they would then abduct random people. Stories and / or fuzzy images of lights in the sky are not good enough.

The claim that at birth, the exact positions of planets and stars in arbitrarily constructed patterns (aka constellations), that they some how permanently imprint on a persons life is extraordinary. The claim that this magical confluence continues to influence a persons life as various planets and constellations moves about is extraordinary. The claim that these changing planetary and constellation positions affects everyone differently based upon their initial birth imprint is extraordinary. Why are they extraordinary claims? Because despite the age of astrology, astrologers are without fail completely unable to demonstrate from whence astrology came*, it has no provenance. They have provided no data as to why this pattern means this or that, they just present assertions that whatever they say is correct. The claims are extraordinary because there is no known mechanism by which these claims could occur, nor has one ever been proposed that I am aware of. That this extraordinary unknown force which can pass vast distances and is strong enough to affect a newborn, is conveniently incapable of penetrating the womb. To claim that because the arbitrarily constructed constellation is here and that planet is there etc, that this some how directly impinges on a person at birth without any reason for doing so is both preposterous and extraordinary when there is no mechanism and no manner of derivation. Hence even by Ertel's generous interpretation of the Carlson study, its fuzziness is not sufficient evidence for the extraordinary claim that is astrology.

You also misunderstand 'provenance', it is not a single prior study which you think indicates the same phenomenon. It is the history of the origins of that phenomenon, the body of supporting data built over time that becomes ever more explanatory as it is refined and added to. Such as electromagnetism where Faraday built on the work of Orsted and Farady's work being built on by Maxwell and so on. Astrology has had centuries if not millenia and yet there is no source, no work, no underlying principles. No, astrology in all that time just asserts that it is real and works. It would be as if the electric motor existed now but there was no history whatsoever of its development or what the underlying principles of its operation were**. Put simply, astrology has no reason for being, it has no provenance. Claiming that it just works and citing two recent studies you interpret as suggesting the same phenomenon at work, is not provenance.

* - By this I am talking about the research that lead to and underpins astrological work, not the history of astrology.

** - This is just a very simple example to illustrate a point, obviously the electric motor works and would still work even were we in the bizarre position of not knowing how. However were the electric motor to lack any underlying work, it simply would not exist, it could not exist.

Ken:

  1. Read Extraordinary Claims Require Extraordinary Evidence. Covers it fairly well. (And no, it is not an "extraordinary claim" in itself. Save your sophistry.)
  2. The astrologers didn’t “successfully match the charts.”  They failed, as you well know. Also, the subjects failed to pick out their own charts. You need to stop lying – Ertel may have found some wiggle room by measuring something else, but he did not alter the basic result of the test which was that the astrologers failed to do what they said they would do.

ken,

you do realize that using a p<=0.05 means that 1 in 20 papers will be dead wrong...right?

that is why when we get a scientific consensus on something there is more than one paper supporting the claim.

even if you were right in your contentions (your are not and frankly your claim have been eviscerated just fine) you'd still be a long way off from supporting the claim the astrology works

Ken, do you believe that if you simply keep blithering on and on and on, Astrology will turn out to be true?

Because I'm sorry to say that it won't.

Here is my 2 cents as an observer who hasn't looked at this study before.
It's definitely a flawed study in many respects. I'm not sure how anyone could misinterpret it's findings, however. Astrologers assigned 40 out of 116 CPIs correctly. That's obviously very close to the 38.5 (1/3 of 116) correct choices predicted by chance, and not very close to the astrologers own prediction of 50% correct (58/116). They set themselves a fairly low target (incidentally that's why the study couldn't be a two choice design) and did not meet it.  Regarding the 10 point ratings, the Astrologers rated incorrect CPIs no differently than correct CPIs. Thus the astrologers can't claim they were forced to pick a poor fit. They thought their first choice was a good fit (7-9 out of 10) whether or not they picked the correct CPI. You can think up all kinds of reasons why they failed, but no reasonable mind can believe this study was a success for the Astrologers. The most obvious flaws were:
1. Subjects (especially young people like those used here) are not very good judges of their own personalities. They couldn't even pick their own CPI at better than chance rates.
2. The CPI may not be the best measure for astrologers to try to match. Looking at the example of the CPI in the paper should make anyone appreciate this difficulty.

If this were an active field of study these and other weaknesses would be openly discussed at meetings and in print, and studies would be designed to replicate the findings while addressing the weaknesses. Such studies are not difficult to imagine. For example, instead of a subject trying to choose his own chart, why not have spouses (married say ten years or more) try to choose each others chart, or have parents choosing the charts of their grown children.
The fact that this single paper remains the focus of discussion after 25 years, and no one has apparently attempted to address it's weaknesses in further controlled studies is an interesting comment in itself. Is the astrology community simply not interested in proving their point?

The comments to this entry are closed.

Search site