Bad Data

Steven Dutch, Natural and Applied Sciences, University of Wisconsin - Green Bay
First-time Visitors: Please visit Site Map and Disclaimer. Use "Back" to return here.

The best defense against being taken in by the pseudoscientist is an awareness of logical fallacies and a strict determination not to fall for emotional appeals. In addition to the logical fallacies, however, pseudoscientists also sooner or later always end up using bad data, and a non-specialist may be hard put to tell good data from bad. Some of the data found in pseudoscience, and some examples, include:

Obsolete Data

The "Ghost Literature" of Pseudoscience

Many of the wonders science supposedly "cannot explain" actually contain nothing to explain. The "facts" are based on obsolete data, hoaxes, coincidence, and the like. In fact, it is useful to have a broad overview of pseudoscience because pseudoscientists commonly cite one another's theories. Erich von Daniken, for example, cited a theory about the capture of the earth's moon that was a popular cult before World War II, and cited numerical coincidences involving the Pyramids that originated in a 19th century Pyramid cult. Creationists cite criticisms of radiometric dating that were originally put forth by Velikovsky supporters. These "facts" look very impressive to the perplexed non-scientist.

Some items of bad data have become dogma in pseudoscience. The "mysterious disappearance of Flight 19", a group of Navy planes which disappeared off Florida in 1945, is a case in point. Complete recordings of the radio traffic between the base and the planes were transcribed in the official investigation into the disaster, and show clearly that the planes headed out into the Atlantic in the mistaken belief that they were west of Florida. Recall that radar was in its infancy in those days. John Wallace Spencer, in Limbo of the Lost, actually quotes a flier as saying "Can't tell whether over Atlantic or Gulf". Yet this incident has become a stock item in the Bermuda Triangle literature. Another stock item are the frozen mammoths of Siberia, supposedly flash-frozen instantly with plants in their stomach that now grow far to the south. One gets the general impression of bananas and papayas, but actually the vegetation in the mammoths' stomachs consisted of pine and spruce twigs and tundra flowers -- normal Siberian vegetation. The best-known specimen was buried when a riverbank caved in on him, breaking his shoulder and hip bones. Somehow these inconvenient details never make it into the popular literature, despite the fact that the frozen mammoths, like Flight 19, are de rigeur in any pseudoscience best-seller. In effect, the literature of pseudoscience is a sort of parallel literature to that of science.

Outdated Science

Another very common approach to data might be called the "lobster trap" approach. Lobster traps are built so that once the lobster gets in, he cannot get out. Pseudoscientists assume that once an idea is published in the scientific literature it is forevermore legitimate evidence, regardless of its original validity or any subsequent findings -- after all, it's "science", isn't it? If the information is refuted or modified later, that just proves once again how fickle science is, so that the final judgment comes down to a subjective one. The experts disagree, so the individual has to decide for himself which alternative to accept. Relativism strikes again.

For example, as the theory of continental drift won widespread acceptance in the late 1960's, a small number of conservative geologists wrote papers arguing against the theory. That was their right and duty as scientists, but most of the arguments they employed were outright balderdash at the time and unlike wine they have not improved with age. To anyone even remotely familiar with the data, these arguments were transparently specious and filled with glaring fallacies. Nevertheless, scientific creationists continue to cite these papers in an attempt to support their own version of earth history with "scientific" evidence. This is a game science cannot win. If scientists revise their ideas, the very fact that they do is taken not as evidence of improvements in knowledge, but as evidence of how changeable and uncertain the results of science are. If science remains firm in a belief, that's evidence not that the idea is correct but that scientists are too pig-headed and dogmatic to change.

Random Events Collected Into Spurious Patterns

In 1932, one of  parapsychology's star performers, Hubert Pearce, guessed 25 consecutive ESP cards correctly. The chances of getting a run of 25 psychic research cards (the standard deck has five each of five different symbols) is one in five to the twenty-fifth power, or one in 295,023,223,876,953,125. Believers in parapsychology insist the probability is so extreme that this incident is proof of a real phenomenon. Skeptics are struck by the inability to repeat these results under controlled conditions and suspect either that the cards were not entirely random (poorly shuffled) or that the subject somehow figured out what the cards were by seeing them reflected in the table top or the experimenter's glasses, or by cuing in on faint nicks, marks, and smudges on the cards. This process need not involve conscious dishonesty. Let's say, however, that the experiment took place exactly as described, without any information reaching the subject by ordinary means, and that the subject actually did overcome enormous odds. What have we proven? Only that an extremely rare event took place. We haven't proven that ESP exists because we are no closer to knowing the mechanism by which the event took place than we ever were. The event could have been chance, or perhaps some other effect completely unrelated to ESP was involved. If we had some theoretical basis to go on we might accept an extremely rare event as evidence because we would have some solid basis for believing the event had a real cause, but in the absence of a workable theory and without the ability to reproduce the event so that we can test different hypotheses, the event just sits in splendid isolation and tells us nothing.

Even the most avowed believers in paranormal phenomena agree that apparent runs of successful guesses come and go erratically. When a possibly real phenomenon is buried amid a great deal of noise, there is a real possibility that the patterns we perceive may be nothing more than the products of our own imaginations. The very existence of the word "coincidence" is a powerful clue to how prone we can be to see spurious patterns in random data.

Real Phenomena Misinterpreted As Anomaly

Revisionist Interpretations of Well-understood Phenomena

Very often, things science "can't explain" turn out to have perfectly straightforward explanations. Despite the claims of Erich von Daniken and many others, there is no mystery at all how the Pyramids were constructed. Museums have the stonecutting tools and rollers that Von Daniken and other Pyramid cultists claim have never been found. Contrary to the claims of creationists, intermediate forms between most major groups of related organisms do exist. We do know what happened to Flight 19, how frozen mammoths got frozen, and so on.

Psychological Phenomena

Pseudoscientists tend to take some kinds of evidence with near-absolute literalness: legends of various kinds, eyewitness accounts of UFO's or mysterious creatures, personal accounts of "after death" experiences, astral travel, clairvoyance or telepathy, or hypnotic accounts of reincarnation. The general rule governing all this sort of evidence is that it is highly personal and subjective, impossible to evaluate or verify rigorously, not subject to controlled experiment, and based on personal experience rather than objective physical evidence. There is probably a close relationship between the preference for this sort of evidence and the widespread feeling that science and technology have mechanized and dehumanized humanity. Like the humanities' support for Velikovsky, there seems to be a desire to see subjective human experiences make a contribution to the physical sciences and thereby "prove" their usefulness and scientific validity.

When I first began researching pseudoscience, I was struck by how often apparently sane people swore they had seen something really outlandish. I wondered how often do otherwise normal people hallucinate? The answer, it turns out, is a lot more often than one might think. In fact there's a whole range of waking and semi-waking hallucinatory phenomena. Hypnogogic and hypnopompic dreams are dreams that occur while a person is dozing off or waking up, and therefore mix waking sensations with dreams. A lot of the alleged instances in which people see angels standing at the bedside are probably of this sort. I once talked with an otherwise very rational person who told of waking up one night and seeing a UFO land near his house; it's very likely this was also an example of a waking dream. I've had a couple of these myself. Generally I'll be in bed (that's where you usually are when going to sleep or waking up) and will "remember" odd things. The "memories" have a harsh, discordant feel to them, and sometimes I'll even wonder "did I dream that or did it really happen?" Or sometimes I'll think the memory is so odd it must have been a dream.

Some people have such vivid fantasy lives that they actually experience some of the sensations of the fantasy. This phenomenon is called confabulation. (If I knew how to induce this state deliberately, I could become obscenely wealthy, but I don't.) After reading an account of the phenomenon in the journal Skeptical Enquirer, one person who frequently experienced confabulation wrote in to describe his enormous relief at not being alone in having these episodes. He described that he had evolved a variety of techniques for telling which stimuli were real and which merely imaginary. It's easy to imagine someone unaware of having the condition, or who experienced it very rarely, believing that a fantasy was the real thing.

Second-hand Data: Urban Legends and Anecdotal Data

The guru of urban legends is Jan Harold Brunvand, a folklorist at the University of Utah, who has published numerous books on the subject. I can do no better than recommend that you read them all. Alligators in the sewer? The kid who dries off the family cat in the microwave? The girl in the 1960's with the beehive hairdo who found critters living in it? The petting couple who are terrified by a radio report of a hook-handed serial killer and drive off, only to find a hook hanging in the door when they get home? They're all there. And not a one is true. Read Brunvand's books! You'll cringe with embarrassment at how many of these hoary myths you've passed along yourself.

The Internet has made it possible to proliferate these myths even faster. A few prominent recent examples include:

All of these might have happened at some time or other. One of the hallmarks of urban legends is that they could be true. But in almost every case, the people who pass these legends along have no direct knowledge of them. They always come from a friend of a friend, and efforts to track the story to its original source never seem to succeed.

Anecdotal Evidence is a single, possibly true instance used to justify a generalization. Everybody who refuses to wear a seat belt has a story of how their Aunt Gertrude was miraculously hurled out of her car into a pile of rose petals just before the car went off a bridge, burst into flames, and sank in quicksand. They simply choose to ignore the far larger number of cases where Aunt Gertrude has to be picked up with a squeegee. To be legitimate evidence, anecdotal evidence has to be true (often it's not) and representative. The anecdote has to illustrate something that happens as a rule, not as an exception.

"The millionaire who pays no taxes" is another popular piece of anecdotal data, especially with political liberals. According to the 1999 Statistical Abstract of the United States (Table 559), the 111,000 taxpayers who reported income of over $1,000,000 dollars in 1996 (the average was about $2.8 million) paid an average of $875,000, or 31 per cent. Taxpayers with $50-75,000 income paid an average of $7300 or 12 per cent. Taxpayers reporting the average income of $35,000 paid $3400 or 9.7 per cent. So millionaires earn on the average about 81 times as much as the average taxpayer but pay 257 times as much tax. Some millionaires do indeed pay no tax because they can offset income against business losses or are paid from tax-exempt bonds, but the anecdote, though true, is not representative.

"Gee-Whiz" Facts are facts that look impressive but have no substance. A prime example is the claim "suicide is the second leading cause of death among teen-agers." Without in any way making light of this problem, think for a second. What can kill a teen-ager? They've already survived childhood diseases and are not prone to diseases of aging yet, so only three things can be significant causes of death among teen-agers: accident, suicide, and homicide. Anything other than that order indicates a real problem, but suicide will always be a leading cause of death among teen-agers.

Another "gee-whiz" fact that pops up during our periodic binges of panic over missing children is "over a million children are reported missing every year." Again referring to the 1999 Statistical Abstract of the United States, about a third of the population, or 77 million people, are 19 or younger. If a million kids vanish every year, in 19 years, 19 million would vanish. One child in four would disappear before reaching adulthood. I think we'd notice that. Yes indeed, a million children are reported missing every year but the vast majority are found within a few hours.

Deliberate Fraud

Parapsychology has seen rampant fraud, with leading researchers repeatedly caught fudging experiments and fabricating data. Even reputable paranormal researchers can and have resorted to fakery. An embarrassing incident took place in 1974, when Walter Levy, Jr., then director of the Institute for Parapsychology in Durham, N.C., was found to have faked experimental results. The experimental design Levy was working on was superb. A rat in a cage had an electrode planted in the pleasure center of its brain, so that the animal would get a pleasurable stimulus when a current was applied to the wire. Animals so wired will do almost anything to keep getting such stimuli, even ignoring food and sex. Levy's rat got a stimulus every time a detector detected the decay of a radioactive atom. If the animal had the ability through paranormal means to increase the number of stimuli the evidence for paranormal powers would be very strong. This is exactly the sort of evidence it would take to demonstrate paranormal effects: the rat is not biased and cannot cheat, and there is no known process that can influence the decay of radioactive materials. The results of such a discovery, to put it mildly, would be far-reaching. 

Workers noted that Levy, a well-known researcher in parapsychology, was unusually attentive to the apparatus, and finally saw him disconnecting a wire to the recorder that registered the results of the experiment. Each disconnection registered a false positive result and therefore made the experimental results look more favorable to Levy's theory. Levy, who had a promising career in parapsychology, resigned. 

Levy was sincerely convinced of the reality of his phenomenon (as most scientific frauds are) but desperate to get results. Uri Geller, who rose to worldwide fame in the 1970's for his alleged psychic abilities, was an outright cynical fraud. Physicists Harold Puthoff and Russell Targ of the Stanford Research Institute (not connected with Stanford University) were very much impressed by the feats of Geller, who could supposedly bend metal through psychic power. Several other prominent physicists involved in paranormal research were highly impressed by Geller. Unfortunately, Geller was seen cheating by professional magicians who were better equipped to spot his sleight-of-hand methods than the physicists, and his stunts were successfully copied, so Geller has since fallen from favor. But not before he did a huge amount of damage. He sued his exposers and lost. They countersued, charging that Geller's lawsuit was frivolous and malicious. They won, but Geller took refuge in his native Israel and never paid up. Still doubt we need radical tort reform?

Just how easy it can be to con believers in parapsychology is shown by a hilarious incident during the heyday of Uri Geller. Magician John Randi, who was then little known in paranormal circles, showed up at the offices of Psychic News in London. He introduced himself as James Zwinge (his real name) and for two hours kept the office in an uproar by bending metal, changing the times on clocks, and duplicating almost all the celebrated feats of psychics using perfectly standard stage techniques. Psychic News ran the story on the front page, complete with claims of "constant surveillance" and assertions that metal had bent when "all could vouch Zwinge had not been near it."

Parapsychologists are insulted by the emphasis on fraud in their field, but the plain fact is the field has been so compromised that the only way to redeem it is to start over from scratch, junking everything done up till now, including the assumption that paranormal phenomena exist at all, and start over with a totally clean slate.

Genuine New Discoveries

One of the heroes of pseudoscience is Ignatz Semmelweiss, who began using antisepsis in the 1850's, a decade before it was generally accepted in medicine. Semmelweiss was driven out of medicine by his fellow doctors. What pseudoscientists don't tell you is that Semmelweiss was in some ways a crank. He had noticed that doctors who went directly from performing autopsies to delivering babies had an appalling record of post-partum deaths. When he insisted that doctors scrub with a caustic solution, his ward soon had the best survival rates. But Semmelweiss theorized that it was the smell of death clinging to the doctor that caused infection, something that must have seemed a throwback to the Middle Ages. Nevertheless, here we have a real case of an apparently crackpot idea turning out to have a real basis in fact. 


Robert Gentry and Radiation Halos?

Return to Pseudoscience Index
Return to Professor Dutch's Home Page

Created 3 February 1998, Last Update 02 June, 2010

Not an official UW Green Bay site