AC Grayling: Never mind the lottery, these are the real secrets of magic

The illusionist's art relies on us wanting to be pleasingly surprised. But Derren Brown's Lotto stunt was a trick too far

If you watched the illusionist Derren Brown predicting the winning lottery numbers last Wednesday, you almost certainly watched him last Friday explaining – one should rather say, "explaining" – how he did it. The Wednesday event was a great trick; the question is, was the Friday event a great trick also?

To answer this question we need to know what, in general, lies behind the ability of magicians and illusionists to do things that invariably make us gasp with surprise and admiration. There are four elements involved. The first is the nature of perception. The second is the fact that almost all of us share a common set of expectations and beliefs, which magicians can exploit. The third is that people want to be entertained, and willingly allow themselves to be led into deceiving themselves accordingly. And the fourth is the skill and dexterity of the magician himself.

Take first the nature of perception. We think that when we are awake with our eyes open, we see our physical environment continuously and as a whole. We do neither. Instead we consciously register only a small part of the environment, the part we are focusing on, and we see it in a series of snapshots that we interpret as continuous, filling in the blanks (for example, when we blink – which we do often) as we go. We not only interpret the snapshot series as continuous with the help of these fillings-in, we also "see" things that are not there but which we expect to find there, or believe are there.

So intense is our focus on one aspect of the environment that we are typically oblivious to the parts of our visual field that we are not attending to. Illusionists and magicians make good use of these facts. To prove it, watch the video at viscog.beckman.illinois.edu/flashmovie/15.php, which shows a group of people, some in black T-shirts and some in white, passing basketballs to each other. Carefully count how many times those in white pass their basketball; then watch the video again, this time not counting the passes, but watching all the people involved. You will be surprised.

The second fact is that the normal human mind possesses a structure of concepts about how the world works – known to philosophers as "folk physics" – which consists of a combination of innate and empirically acquired expectations and beliefs, such as that water is wet, objects fall to the ground if unsupported, one solid object such as a stone cannot pass through another solid object such as a wall, and so on. Illusions exploit these expectations by appearing to contradict them; that is what amazes and delights us when we see a trick performed. As a perfect example of how these facts can be used by the illusionist, watch Professor Richard Wiseman demonstrate a card trick in the video at www.youtube.com/ watch?v=vo AntzB7EwE

The third element is the ordinary human desire to be surprised in pleasing ways, allowing the illusionist to manipulate his audience into deceiving itself. Because the audience is highly willing to be manipulated, it is accordingly amazed and delighted by the illusion, all the more so because they cannot for the life of them see how it is done.

Finally, there is the skill of the illusionist himself. It has been well said that a magician is not a magician but an actor playing a magician, and the skill lies more in the patter and personality of the performer than in the way the trick itself is carried out. Indeed, the patter is an essential ingredient of the trick: it is part of the distraction that prevents the audience from seeing how the trick is done; it raises expectations – often and deliberately the wrong ones; and it prompts the audience to believe it sees something that is in fact not there. This is the art of psychological "suggestion". The patter is sleight of mind, and sleight of mind is more important than sleight of hand.

In his fine book on illusionism, Hiding the Elephant, the master-designer of tricks Jim Steinmeyer says, "Magicians guard an empty safe. There are few tricks that are beyond the capacity of a high school science class, little technology more complex than a rubber band, a square of mirrored glass, or a length of thread. When an audience learns how it's done, they quickly dismiss the art: 'is that all it is!'" But the real art, he says, lies in the finesse with which these objects are handled: "the audience is taken by the hand and led to deceive themselves".

For all the fandango in Brown's Friday night promised explanation, Brown adhered to the illusionists' ethical code of not divulging any secrets. The hour-long "explanation" was itself a trick, and not as good as the lottery trick itself. Note that Brown did not write up the numbers for all to see before the lottery draw took place; there was no question of repeating the experiment a large number of times, as a scientific investigation would require, with control groups of strangers randomly nominating numbers to compare how the "mutually-bonded, automatic-writing" group fared; and Brown was careful to say that picking the lottery numbers would not work if you intend to profit from it – which, of course, is what all lottery players hope for.

Almost all illusionists have assistants, unnoticed and unnamed except in programme credits at the end. Many of the tricks they perform are invented by people such as Steinmeyer and other designers, commissioned to tailor individual illusions or routines for professional performers. Television illusionism is easier than performance on the live stage, but the principles are the same as those mentioned: exploitation of the facts about perception, the common human mindset, people's willingness to self-deceive, and the illusionist's art of patter and power of suggestion.

There are other realms in which illusion is used, for a variety of purposes. From stage design to advertisements, from politicians' use of statistics to novelists telling stories, techniques of distraction and persuasion are commonplace. Next time you look at a figurative painting, remember that the three-dimensional scene you are inspecting lies on a two-dimensional canvas. When looking at a portrait painting, try to connect the limbs to the body under the clothing: you will almost certainly find that the human figure putatively concealed by the clothing is an impossibility.

What painters, politicians and advertisers rely on is the suggestibility, distractibility and "focus-inattention" (not seeing things because of looking too hard at something else) of all human beings, even including those who know about these aspects of human cognition. Although illusionists are not alone among people who use these facts to make their living, they are the most honest: they offer their tricks as tricks, and a great part of their audiences' pleasure is the knowledge that they are tricks, however mind-boggling they seem.

"A few great magicians," Steinmeyer writes, "have always realised that these ephemeral, temporary miracles could be restorative for their audiences. They listen for the brief pause between the end of the trick and the beginning of the applause, the split second when the entire audience shares a gasp of genuine amazement. At that moment there's always been an honorable quality in illusion."

This is a good point: watching a really good magic trick is cathartic and pleasurable. Derren Brown's performance last Wednesday was like that; his Friday performance did not compare, because it seemed – well, like too much of a trick to work.