There's no safety in numbers

Market research is booming, but it'll never be a match for human perversity.

Prepare yourselves as we move into election season for an onslaught of "research shows ..." stories. This month it appears to be showing that while 51 per cent of women who have decided how to vote will vote Labour, Tony Blair's personal appeal ("satisfaction rating") is plummeting. Labour no doubt finds comfort in the former, while the Daily Mail finds enough solidity in the latter to run a headline "Why women find this man so smarmy". One could add that another paper, the Independent, reported that while Blair's satisfaction rating had indeed fallen from plus 23 to plus 2, John Major's remained at minus 29.

Numbers are great - nice, solid things that you can get your head round, quote in pub arguments and, if you are a politician or Today programme interviewer, use as rhetorical ping-pong balls.

Seventy-six per cent of Britons think it is immoral to lie to their husband or wife; 75 per cent of French people would not be shocked if the Queen had had an affair; 41 per cent of couples have had sex in the kitchen; 34 per cent of schoolchildren worry a lot about their parents' marriage.

Who provides us with these numbers? Everybody's friend - the market researcher. Market research is one of the great boom industries. Its British revenues grew by almost 15 per cent last year, to pounds 420m. It is also useful. A MORI survey revealing that two-thirds of teenage smokers considered the risks from smoking to be unimportant could trigger government action. A drinks company might spend half a million pounds on research before changing the shape of a bottle. That makes sense for the shareholders: get it wrong and sales could drop by millions a year.

And market research can be fun. A snigger of light was beamed into the dour post-war world by the Kinsey report on sex, which found that while 23 per cent of men claimed to have received fellatio, only 12 per cent of women said they had administered it.

Where would the glossy magazines of today be without their sex and diet surveys?

But when we read any of these surveys, we would do well to keep our eyebrows well raised. For market research is like economics - it tries very hard to be a science, but human nature and the complexity of the world keep on leaving it behind. Techniques have made great strides in the last couple of decades - but they still failed miserably to predict the result of the last election, and they did not stop Coca-Cola investing millions in a drink hardly anybody wanted.

The danger lies in the numbers, which is why one genus of market researcher must be considered on its own. Qualitative researchers, the "Quallies", shun figures. They gather "focus groups" of perhaps a dozen people and get them into a discussion about the best shape for a Guinness bottle or a new Ford advertisement. They observe the groups through a two-way mirror, and then write nice fat reports for their clients. Qualitative research is often used by companies as part of expensive launches or relaunches.

Focus groups can produce subtle, unquantified results - though human nature takes its toll. It takes an extraordinarily skilful "moderator" to stop individuals following the common view, fibbing to impress, deceiving themselves - in other words, generally behaving like human beings. Sometimes different focus groups on the same subject will produce utterly different results, which is why so many of the industry's clients demand large-scale quantitative surveys.

The "Quanties" are the boffins. Quantitative market researchers write complex computer models to analyse the figures and to produce a neat set of results. But as Terry Hanby, an experienced market researcher now at the Judge Institute of Management Studies at Cambridge University, says, "The danger with numbers is that they miss much of what is truly descriptive." If you are asking about a complex issue, he says, you will likely get a range of views - average them and the result is meaningless.

Quantitative research should work well where there is an unambiguous choice. "Who are you going to vote for?" "Do you prefer this drink or this one?"

These questions could hardly be simpler - but they have caused much redness of face. Four polling organisations failed to predict a Tory victory in 1992, and the industry hurled itself into an orgy of navel-gazing to find out why afterwards. It came up with a number of ideas - late swing, Conservative voters less likely to give their intention away, "don't knows" voting disproportionately for the Tories. But no one really knows, and April 1992 remains researchers' mensis horribilis.

The commercial equivalent was Coca-Cola's launch of "new Coke". It tested the taste on 190,000 people before launching it, and it still flopped. Coca-Cola assumed people bought Coke just because of the taste. They did not: image is at least as important, and it seems new Coke just wasn't cool.

The research industry has been working hard to make sure these cock-ups are eliminated, and to ensure that their methodology is beyond reproach. Twenty years ago it was all too easy for a researcher to invent interviewees - now, thanks to various systems of double checking, they know they could be caught out.

In addition, the best outfits are alert to distortions that may arise from the client's prejudices. "I resign if we are told to ask biased questions," says Bob Worcester of MORI, eminence grise of the industry. He cites a favourite from a 1938 Gallup survey: "Do you favour direct retaliatory action against Franco's piracy?" But, he says, the phrasing of a question is not always crucial. If it relates to a basic "value", such as the death penalty or abortion, it does not matter how it is phrased. A pro-abortion group could carry out a survey asking if people believed in choice, and most people would understand what the issue was and would vote according to their beliefs. The same would be true of an anti-abortion "pro-life" questionnaire.

Some subjects just cannot be measured properly. Ask most people how much alcohol they drink, and shame will make them underestimate. Ask others, and they will boast. A third group will not have a clue, because they drink so much they cannot remember.

Income is another tricky one - interviewees may exaggerate to impress, or play it down in case the Revenue is listening. Then there is sex: the fellatio phenomenon is consistent across surveys, with men always claiming to have more oral sex than women. Is this because some women oblige dozens of men ... or just male wish fulfilment?

There are many ways of gathering research information but the foot soldiers, generally, are middle-aged women doing their level best to get their clipboards filled in. They will, if necessary, return to ring the same doorbell over and over again - because a computer has determined that this is the household that must be quizzed. Other times they will go to great lengths to make sure they have built up their "quota" exactly to spec. If you see a women with a clipboard desperately scanning a crowd, she is probably looking for that 25-year-old Asian C2 man she needs to complete her work.

They will drag people off the street into halls, and ask them to taste cupcakes or drinks. They will ring MPs, peers, journalists or other "opinion formers", requesting an interview, offering some inducement. MPs will be offered about pounds 60, though, nowadays, will probably ask that it be sent to charity (MORI gave away pounds 15,000 like this last year). Peers and journalists are more likely to take the money, and meekly provide "perceptions" of companies they have barely heard of.

More arduous for researchers and respondents are the omnibus surveys, the source of those "Seventy-six per cent of Britons ..." headlines, as well as of opinion polls. These are large regular questionnaires with pre-selected interviewees. Companies or charities can "buy" questions for pounds 800 a go. The research organisation will include all sorts of riders in its press release of the findings. But by the time a news editor has his hands on it, it will probably be a bold shrieking headline.

Whatever the pollsters do, though, they find themselves struggling with good old human perversity. "Ten per cent of people believe ICI makes bicycles," Mr Worcester says. "You show them a list of products like paints and fertilisers, throw in bicycles as a dummy and one in 10 will tick it." Fifty years ago, an academic surveyed Americans' attitude to the Metallic Metals Act - 38 per cent said it should be passed. There was no Metallic Metals Act.

Next time you read, for example, that 90 per cent of hot-water bottle fatalities are caused by scalding - a statistic that rated a newspaper item some years ago - consider what you have learnt. It sounds true, scientifically precise, and solid enough to base a business plan on. But then consider the fellatio factor. Consider the Coca-Cola factor. And ask yourself, what on earth could have caused that other 10 per cent?