Science: LEARNING BY NUMBERS

Watches stopping when psychics tell them to, lucky floor-tiles in betting shops, dreaming of someone, then they phone out of the blue - what's it all about? Rather than searching into the great unknown for an explanation worthy of the 'X-Files', perhaps we should be looking a little closer to home and studying the laws of statistics

THE EMINENT fertility specialist Robert Winston imagines an ad, placed in a newspaper by an unscrupulous quack. It is aimed at people who want their next baby to be, say, a son and reads: "Send pounds 500 for my patent recipe to make your baby a boy. Money refunded in full if I fail." The money back guarantee is intended to establish confidence in the method. In fact, since boys turn up anyway on approximately 50 per cent of occasions, the scheme would be a nice little earner. Indeed, the quack could safely offer compensation of say, pounds 250 for every girl, over and above the money back guarantee. He would still show a tidy profit in the long run.

I used a similar illustration in my 1991 Royal Institution Christmas Lectures. I said I had reason to believe that among my audience was a psychic, clairvoyant individual, capable of influencing events purely by the power of thought. I would try to flush this individual out. "Let's first establish," I said, "whether the psychic is in the left half or the right half of the lecture hall." I invited everybody to stand up while my assistant tossed a coin.

Everybody on the left of the hall was asked to "will" the coin to come down heads. Everybody on the right had to will it to be tails. Obviously one side had to lose, and they were asked to sit down. Then those who remained were divided into two, with half "willing" heads and the other half tails. Again the losers sat down. And so on by successive halvings until, inevitably, after seven or eight tosses, one individual was left standing. "A big round of applause for our psychic. He must be psychic, mustn't he, because he successfully influenced the coin eight times in a row?"

If the lectures had been televised live it would have been much more impressive. I'd have asked everybody who watched it whose surname begins before J in the alphabet to "will" heads and the rest tails. Whichever half turned out to contain the "psychic" would have been divided in half again, and so on. I'd have asked everybody to keep a written record of the order of their "willings". With 2 million viewers, it would have taken about 21 steps to narrow down to a single individual. To be on the safe side I'd have stopped a bit short of 21 steps. At, say, the 18th step, I'd have invited anybody still in the game to phone in. There would have been quite a few and, with luck, one would phone. This individual would then have been invited to read out his/her written records; HTTHHT HHHHHTTTHHTT which would have matched the official record. So this one individual succeeded in influencing 18 successive tosses of a coin. Gasps of admiration. But admiration for what? Nothing but pure luck. I don't know if that experiment has been done. Actually, the trick here is so obvious it probably wouldn't fool many people. But how about the following?

A well-known "psychic" goes on television, a lucrative engagement fixed up over lunch by his publicity agent. Staring hypnotically out of 10 million screens, our imaginary seer intones that he feels a strange, spiritual rapport, a vibrating resonance of cosmic energy, with certain members of his audience. They will be able to tell who they are because, even as he utters his mystic incantation, their watches will stop. After only a brief pause, a telephone on his table rings and an amplified voice in awed tones announces that its owner's watch stopped dead within seconds of the seer's words. The caller adds that she had a premonition that this was going to happen even before she looked down at her watch, for something in her hero's burning eyes seemed to speak directly to her soul. She felt the "vibrations" of "energy". Even as she is speaking, a second telephone rings. Yet another watch has stopped.

A third call comes through, one more watch has been susceptible to occult forces. It stopped a whole day before, at the very moment when its owner looked at the mystic's photograph in the newspaper. The studio audience gasps its appreciation. This, surely, is psychic power beyond all scepticism, for it happened a whole day early! "There are more things in heaven and earth, Horatio ... "

What we need is less gasping and more thinking and to take the sting out of coincidence by calculating the likelihood that it would have happened anyway. In the course of which, we shall discover that to disarm apparently uncanny coincidences is more interesting than gasping over them anyway.

Any given watch has a certain low probability of stopping at any moment. I don't know what this probability is, but here's the kind of way in which we could come to an estimate. If we take just digital watches, their battery typically runs out within a year. Approximately, then, a digital watch stops once per year. Presumably clockwork watches stop more often because people forget to wind them and presumably digital watches stop less often because people sometimes remember to renew the battery ahead of time. But both kinds of watches probably stop as often again because they develop faults of one kind or another. So, let our estimate be that any given watch is likely to stop about once a year. It doesn't matter too much how accurate our estimate is. The principle will remain.

If somebody's watch stopped three weeks after the spell was cast, even the most credulous would prefer to put it down to chance. We need to decide how large a delay would have been judged by the audience as sufficiently simultaneous with the psychic's announcement to impress. About five minutes is certainly safe, especially since he can keep talking to each caller for a few minutes before the next call. There are about 100,000 five-minute periods in a year. The probability that any given watch, say mine, will stop in a designated five-minute period is about one in 100,000. Low odds, but there are 10 million people watching the show. If only half of them are wearing watches, we could expect about 25 of those watches to stop in any given minute. If only a quarter of these ring in to the studio, that is six calls, more than enough to dumbfound a naive audience. Especially when you add in the calls from people whose watches stopped the day before, people whose watches didn't stop but whose grandfather clocks did, people who died of heart attacks and their bereaved relatives phoned in to say that their "ticker" gave out, and so on.

At this point, I want to invent a technical term, and I hope you'll forgive an acronym. "Petwhac" stands for Population of Events That Would Have Appeared Coincidental. Population may seem an odd word, but it is the correct statistical term. Somebody's watch stopping within 10 seconds of the psychic's incantation obviously belongs within the petwhac, but so do many other events. The psychic said nothing about watches stopping the day before or grandfathers' tickers suffering cardiac arrests.

People feel that such unanticipated events belong in the petwhac. It looks to them as though occult forces must have been at work. But when you start to think like this, the petwhac becomes really quite large, and therein lies the catch. If your watch stopped exactly 24 hours earlier, you would not have to be unduly gullible to embrace this event within the petwhac. The larger the petwhac, the less we ought to be impressed by the coincidence when it comes. One of the devices of an effective trickster is to make people think exactly the opposite.

By the way, I deliberately chose a more impressive trick for my imaginary psychic than is actually done with watches on television. The more familiar feat is to start watches that have stopped. The television audience is invited to get up and fetch, out of drawers or attics, watches that have broken down, and hold them while the psychic performs some incantation or does some hypnotic eye work. What is really going on is that the warmth of the hand melts oil that has coagulated and this starts the watch ticking, if only briefly. Even if this happens in only a small proportion of cases, this proportion, multiplied by the large audience, will generate a satisfactory number of dumbfounded telephone calls. Actually, as Nicholas Humphrey explains in his admirable expose of supernaturalism Soul Searching (1995), it has been demonstrated that more than 50 per cent of broken watches start, at least momentarily, if they are held in the hand.

Here's another example of a coincidence, where it is clear how to calculate the odds. We shall use it to go on and see how odds are sensitive to changing the petwhac. I once had a girlfriend who had the same birth date (though not the same year) as a previous girlfriend. She told a friend of hers who believed in astrology, and the friend triumphantly asked how I could possibly justify my scepticism in the face of such overwhelming evidence that I had unwittingly been brought together with two successive women on the basis of their "stars". Once again, let's just think it through quietly. It is easy to calculate the odds that two people, chosen at random, will have the same birthday. There are 365 days in the year. Whatever the birthday of the first person, the chance that the second will have the same birthday is one in 365 (forgetting leap years). If we pair people off in any particular way, such as taking the successive girlfriends of any one man, the odds that they will share their birthday are one in 365. If we take 10 million men (less than the population of Tokyo or Mexico City), this apparently uncanny coincidence will have happened to more than 27,000 of them!

Now let's think about the petwhac and see how the apparent coincidence becomes less impressive as it swells. There are many other ways in which we could pair people off and still end up noticing an apparent coincidence. Successive girlfriends with the same surname, although unrelated, for instance; or two people with the same birthday sitting next to one another on an plane. Yet, in a well-loaded Boeing 747, the odds are actually better than 50 per cent that at least one pair of neighbours will share a birthday. We don't usually notice this because we don't look over each other's shoulders as we fill in our immigration forms. But if we did, somebody on most flights would go away muttering darkly about occult forces.

The birthday coincidence is famously phrased in a more dramatic way. If you have a roomful of only 23 people, mathematicians can prove that the odds are just greater than 50 per cent that two of them will share the same birthday. I have every sympathy with people who are phobic about mathematical formulas, so I'll spell out an explanation in words.

It's easier to calculate the odds that there is not a pair of shared birthdays in the room. Let's pretend that leap years don't exist, and suppose you and I are among the 23 people in a room. My birthday is 26 March. I don't know when yours is but, since there are 364 days that are not 26 March, the odds are 364/365 (0.997) that your birthday is not mine. But the pairing of you with me is only one of many pairings that we could imagine in our roomful of 23 people. We have to multiply 364/365 by itself for each pairing. How many pairings? A first guess is 23 x 23 (= 529), but this is clearly too many. It allows each person to be paired with himself, which is absurd. So we must at least subtract 23 from our preliminary list of possible pairings, which gives us (23x23) -23 = 506. And our first guess also counts you/me as separate from me/you, whereas obviously, if I share my birthday with you, you must share your birthday with me. In other words it counts each pairing twice. So we must halve our 506, giving 253 as the number of pairings that we must consider. If you multiply 364/365 by itself 253 times you get a number very close to 0.5. This is the chance that there will not be any shared birthdays in the room.

So, unlikely as it first sounds, there's an approximately even chance that at least one pair of individuals in a committee of 23 will share a birthday. It is this kind of intuitive error that in general bedevils our assessment of "uncanny" coincidences.

Now let's take another kind of coincidence, where it is even harder to know how to start calculating odds. Consider the often-quoted experience of dreaming of an old acquaintance for the first time in years and then getting a letter from him, out of the blue, the next day. Or of learning that he died in the night. Or of learning that he didn't die in the night but his father did. Or that his father didn't die but won the football pools. See how the petwhac grows out of control when we relax our vigilance?

Each one of us, though only a single person, none the less amounts to a very large population of opportunities for coincidence. Each ordinary day that you or I live through is an unbroken sequence of events, or incidents, any of which is potentially a coincidence. I am now looking at a picture on my wall of a deep-sea fish with a fascinatingly alien face. It is possible that, at this very moment, the telephone will ring and the caller will identify himself as a Mr Fish. I'm waiting ...

The telephone didn't ring. My point is that, whatever you may be doing in any given minute of the day, there probably is some other event - a phone call, say - which, if it were to happen, would with hindsight be rated an eerie coincidence. There are so many minutes in every individual's lifetime that it would be quite surprising to find an individual who had never experienced a startling coincidence.

Our propensity to see significance and pattern in coincidence, whether or not there is any real significance there, is part of a more general tendency to seek patterns. This tendency is laudable and useful. Many events and features in the world really are patterned in a non-random way and it is helpful to us, and to animals generally, to detect these patterns. The difficulty is to navigate between detecting apparent pattern when there isn't any, and failing to detect pattern when there is. The science of statistics is quite largely concerned with steering this difficult course. But long before statistical methods were formalized, humans and indeed other animals were reasonably good intuitive statisticians. It is easy to make mistakes, however, in both directions.

An example of a true statistical pattern in nature which is not totally obvious, and which humans have not always known is that smoking causes lung cancer. We now know that smoking causes lung cancer, yet it was difficult to pin down because plenty of people who smoke don't get lung cancer and many people get lung cancer who never smoked.

We are not the only animals to seek statistical patterns of non-randomness in nature, and we are not the only animals to make mistakes of the kind that might be called superstitious. Both these facts are neatly demonstrated in the apparatus called the Skinner box named after the famous American psychologist B F Skinner. It is a simple but versatile piece of equipment for studying animal psychology which consists of a switch or switches let into one wall which a pigeon (say) can operate by pecking. There is also an electrically operated feeding (or other rewarding) apparatus. The two are connected in such a way that pecking by the pigeon has some influence on the feeding apparatus. In the simplest case, every time the pigeon pecks the key it gets food. Pigeons readily learn the task. So do rats and, in suitably enlarged and reinforced Skinner boxes, so do pigs.

We know that the causal link between key peck and food is provided by electrical apparatus, but the pigeon doesn't. As far as the pigeon is concerned, pecking a key might as well be a rain dance. Moreover, the link can be quite a weak, statistical one. The apparatus may be set up so that only one in 10 pecks is rewarded. This can mean literally every tenth peck. Or, with a different setting of the apparatus, it can mean that one in 10 pecks on average is rewarded, but on any particular occasion the exact number of pecks required is determined at random. Or there may be a clock which determines that one tenth of the time, on average, a peck will yield reward, but it is impossible to tell which tenth of the time. Pigeons and rats learn to press keys even when, one might think, you'd need to be quite a good statistician to detect the cause-effect relationship. They can be worked up to a schedule in which only a very small proportion of pecks is rewarded. Interestingly, habits learned when pecks are only occasionally rewarded are more durable than habits learned when all pecks are rewarded: the pigeon is less swiftly discouraged when the rewarding mechanism is switched off altogether. This makes intuitive sense if you think about it.

Pigeons and rats, then, are quite good statis- ticians, able to pick up slight, statistical laws of patterning in their world. Presumably this ability serves them in nature as well as in the Skinner box. Life out there is rich in pattern; the world is a big, complicated Skinner box.

Skinner founded a large school of research using Skinner boxes for all kinds of detailed purposes. Then, in 1948, he tried a brilliant variant on the standard technique. He completely severed the causal link between behaviour and reward. He set up the apparatus to "reward" the pigeon from time to time no matter what the bird did. Now all that the birds actually needed to do was sit back and wait for the reward. But in fact this is not what they did. Instead, in six out of eight cases, they built up - exactly as though they were learning a rewarded habit - what Skinner called "superstitious" behaviour. Precisely what this consisted of varied from pigeon to pigeon. One bird spun itself round like a top between "rewards". Another bird repeatedly thrust its head towards one particular corner of the box. A third bird showed "tossing" behaviour, as if lifting an invisible curtain with its head. Two birds independently developed the habit of rhythmic, side-to-side "pendulum swinging" of the head and body. Skinner used the word superstition because the birds behaved as if they thought that their habitual movement had a causal influence on the reward mechanism, when actually it didn't. It was the pigeon equivalent of a rain dance.

Skinner's superstitious pigeons were behaving like statisticians, but statisticians who have got it wrong. They were alert to the possibility of links between events in their world, especially links between rewards that they wanted and actions that it was in their power to take. A habit, such as shoving the head into the corner of the cage, began by chance. The bird just happened to do it the moment before the reward mechanism was due to clunk into action. Understandably enough, the bird developed the tentative hypothesis that there was a link between the two events. So it shoved its head into the corner again. Sure enough, by the luck of Skinner's timing mechanism, the reward came again. If the bird had tried the experiment of not shoving its head into the corner, it would have found that the reward came anyway. But it would have needed to be a better and more sceptical statistician than many of us humans are in order to try this experiment.

Skinner makes the comparison with human gamblers developing little lucky "tics" when playing cards. A one-arm bandit in Las Vegas is nothing more nor less than a human Skinner box. "Key-pecking" is represented not just by pulling the lever but also, of course, by putting money in the slot. It really is a fool's game because the odds are known to be stacked in favour of the casino - how else would the casino pay its huge electricity bills? Whether or not a given lever pull will deliver a jackpot is determined at random. It is a perfect recipe for superstitious habits. Sure enough, if you watch gambling addicts in Las Vegas you see movements highly reminiscent of Skinner's superstitious pigeons. Some talk to the machine. Others make funny signs to it with their fingers, or stroke it or pat it with their hands. They once patted it and won the jackpot and they've never forgotten it.

My informant about Las Vegas has also made an informal study of London betting shops. She reports that one particular gambler habitually runs, after placing his bet, to a certain tile in the floor, where he stands on one leg while watching the race on the bookmaker's television. Presumably he once won while standing on this tile and conceived the notion that there was a causal link. Now, if somebody else stands on "his" lucky tile (some other sportsmen do this deliberately, perhaps to try to hijack some of his "luck" or just to annoy him) he dances around it, desperately trying to get a foot on the tile before the race ends. Other gamblers refuse to change their shirt, or to cut their hair, while they are "on a lucky streak".

Let me return to uncanny coincidence and the calculation of the probability that it would have happened anyway. If I dream of a long-forgotten friend who dies the same night, I am tempted, like anyone else, to see meaning or pattern in the coincidence. I really have to force myself to remember that quite a few people die every night, masses of people dream every night, they quite often dream that people die, and coincidences like this are probably happening to several hundred people in the world every night. Even as I think this through, my own intuition cries out that there must be meaning in the coincidence because it has happened to me. If it is true that intuition is, in this case, making a stastical error, we need to come up with a satisfactory explanation for why human intuition errs in this direction.

As a Darwinian, I want to suggest that our willingness to be impressed at apparently uncanny coincidence (which is a case of our willingness to see pattern where there is none) is related to the typical population size of our ancestors and the relative poverty of their everyday experience. Anthropology, fossil evidence and the study of other apes all suggest that our ancestors, for much of the past few million years, probably lived in either small roving bands or small villages. Either of these would mean that the number of friends and acquaintances that our ancestors would ordinarily meet and talk to with any frequency was not more than a few dozen. A prehistoric villager could expect to hear stories of startling coincidence in proportion to this small number of acquaintances. If the coincidence happened to somebody not in his village, he wouldn't hear the story. So our brains became calibrated to detect pattern, and gasp with astonishment at a level of coincidence which would actually be quite modest if our catchment area of friends and acquaintances had been large.

Nowadays, our catchment area is large, especially because of newspapers, radio and other vehicles of mass news circulation. The best and most spine- creeping coincidences have the opportunity to circulate, in the form of bated-breath stories, over a far wider audience than was ever possible in ancestral times. But, I am now conjecturing, our brains are calibrated by ancestral natural selection to expect a much more modest level of coincidence, calibrated under small village conditions. So we are impressed by coincidences because of a miscalibrated gasp threshold. Our subjective petwhacs have been calibrated by natural selection in small villages, and, as is the case with so much of modern life, the calibration is now out of date.

I guess that there may be another, particular effect pushing in the same direction. I suspect that our individual lives under modern conditions are richer in experiences per hour than were ancestral lives. We don't just get up in the morning, scratch a living in the same way as yesterday, eat a meal or two and go to sleep again. We read, watch television, travel at high speed to new places, pass thousands of people in the street. The number of faces we see, the number of different situations we are exposed to, the number of separate things that happen to us, is much greater than for our village ancestors.

This means that the number of opportunities for coincidence is greater for each one of us than it would have been for our ancestors, and consequently greater than our brains are calibrated to assess. This is an additional effect, over and above the population size effect that I have already noted.

With respect to both these effects, it is theoretically possible for us to recalibrate ourselves, learn to adjust our gasp threshold to a level more appropriate to modern populations and modern richnesses of experience. But this seems to be revealingly difficult even for sophisticated scientists and mathematicians. The fact that we still do gasp when we do, that clairvoyants and mediums and psychics and astrologers manage to make such a nice living out of us, all suggests that we do not, on the whole, learn to recalibrate ourselves. It suggests that the parts of our brains responsible for doing intuitive statistics are still back in the stone age. !