Cameron is far happier than his party

The Tory leader – like Tony Blair and Nick Clegg – is not a great fan of his party. He is no tribal politician, so finds coalitions easy and party conferences tiresome

After a week's uncertainty at Labour's conference in Manchester – "Why did we lose?", "Have we chosen the right leader?" – the circus has decamped to Birmingham. And, although the Conservatives are in power, there's an oddly similar air of uncertainty. "Why didn't we win outright?" is the question activists want answered.

Tories, after all, are used to exercising power. It's their default mode. When out of government, they can't quite believe voters have been so foolish as to have been taken in by Labour. That's why they find it so hard to change in opposition. Some of them are still saying that, if only the election campaign had focused more on immigration, Europe and tax cuts, they might have done better.

Yet the voters with whom they did worst – public-sector workers, Scots and ethnic minorities – would hardly have been converted by promises of lower immigration, lower tax and greater Euroscepticism. If anything, the problem was that the Tories hadn't detoxified their brand enough. A post-election survey of voters who had considered voting Conservative found that they feared the party was still for the rich and not for ordinary people, and that it would cut spending and remove benefits such as tax credits and bus passes.

Given how propitious the election was for the Tories, though, surely they should still have won a stonking majority? A year before, they had been 20 points ahead in the polls, and this May, they faced a shambolic Government, an unpopular Prime Minister and the worst recession since the 1930s.

Well, the first thing to say is that they didn't do badly at all. Not since the 1930s has there been a swing to the right of the scale David Cameron needed. For a simple majority of one, he needed a 6.9 per cent swing from Labour; for a tiny majority of 20, he needed 7.6 per cent. In the event, he achieved 5.1 per cent, only just smaller than Margaret Thatcher's 5.3 per cent in 1979 after the IMF had been called in, the rubbish was piling up in the streets and the country was crying out for competent, authoritarian government. Cameron actually gained more seats than Thatcher – at 97, it was the largest net gain for his party since 1931.

But Labour fought an effective defensive campaign. They knew they were going to lose, so concentrated not on the most marginal seats, but on the middle rank of constituencies that would deprive the Tories of an overall majority. And it worked. As the party's campaign co-ordinator, Douglas Alexander, likes to put it, Labour won a 1992 share of the seats on a 1983 share of the vote.

Labour was up against a much richer rival: the Tories outspent them three to one. And Labour's national campaign was pretty dire. But what the media never spotted was how well Labour was doing in engaging with individual voters.

The party had a new database system called Contact Creator. Every time a Labour candidate put a voter into the system, it was logged at national level. So headquarters could measure exactly how many voters had been spoken to in each constituency, and what their concerns were. The national campaign then rewarded the candidates who had uploaded most contacts by offering them extra help with direct mail and phone banks.

It made a big difference. In the top 100 seats for voter contact, the average swing away from Labour was nearly two points less than the national swing, while in the top 10 seats, there was actually a small swing towards Labour. As the Slough MP Fiona Mactaggart puts it: "If you talk to voters on the doorstep and give them a chance to slap you, they're less likely to slap you in the ballot box."

This was the culmination of work that had started years before. Between 2008 and early 2010, as Dennis Kavanagh and Philip Cowley point out in their new book The British General Election of 2010, Labour sent out 8.3 million letters to voters. A further 7.4 million were delivered during the campaign. Three-quarters of the party's election spending went on direct mail targeted to play to individual voters' fears about what a Tory government might do. One leaflet about cancer treatment sent to middle-aged women made the headlines, but the rest fell below the media's radar. Yet a Conservative told the book's authors that a Labour leaflet on tax credits was "the most powerful thing they did".

The Tories, too, concentrated on their so-called "battleground" seats, with money from Lord Ashcroft. They had some success, but not as much as they had hoped. "A mixture of achievement and disappointment" was the verdict from one campaign strategist.

Ashcroft believes that the local campaigns were battling against the lack of a clear message at a national level. Voters overwhelmingly wanted a change, but they weren't convinced they wanted a change to the Conservatives. The Tory campaign spent too much time attacking Brown and not enough persuading voters that the Conservatives had changed and were on their side. This allowed Labour's scare tactics to resonate.

Ashcroft also blames Cameron for agreeing to the television debates, which gave a platform to Nick Clegg. As Kavanagh and Cowley relate, when Jeremy Hunt stood in for the Lib Dem leader at the Tories' dress rehearsal, he gave an uncannily accurate performance. Afterwards, Anita Dunn, one of the party's American advisers, said, "Well, I'm voting for Nick Clegg."

Yet still the party did little to deflect the threat. "We weren't disciplined enough to work out what to do about it," admitted one of those involved. Cameron went into the first debate with too little preparation and no idea how to take on a rival who was promising change more convincingly than he was.

So part of the blame for the Tories not now governing on their own can be laid at the door of Cameron and the people around him. The irony is that it is Cameron and the people around him who are most relieved that they are not governing on their own. Whatever their public protestations, they like the way their coalition with the Lib Dems gives power to moderates at the expense of the right. With a minority or a small majority government, the right would have held the balance of power.

Cameron – like Tony Blair and, indeed, Nick Clegg – is not a great fan of his party. He is no tribal politician, which means that he finds coalitions easy and party conferences tiresome. But he can use this week to explain to his activists why the current situation has, despite everything, turned out to be for the best.

Had the Tories formed a minority government and held an autumn election in the hope of increasing their seats, they would probably have been disappointed. The last two times this has been tried – in 1910 and 1974 – the share of seats barely changed. Even if he had won a small majority, he would still have been held hostage by the right.

And suppose he had run a better campaign and ended up with a small majority in May, he could never have got away with forming a coalition. The result would have been a government elected by fewer than 40 per cent of voters enacting virulently unpopular measures. He could hardly have called it, as he did yesterday, "a government in the national interest".

Now he is leading an administration representing 60 per cent of the population, with only one party in opposition. And the coalition is helping to detoxify the Tory brand. The alternative was one that would have been lambasted as "same old Tories, same old cuts" by two parties in opposition. In that light, it's not surprising Cameron has more of a spring in his step this week than most of his party activists.

m.sieghart@independent.co.uk