John Rentoul: The man who could have changed it all

Alan Johnson ducked his chance to be leader of his party, and David Cameron was the lucky beneficiary

(AFP/Getty Images)

Most writing about politics creates the impression of certainty, of great historical forces rolling towards inevitable outcomes. Politicians often conspire in this myth-making. The most notable example was James Callaghan's "sea change" in 1979. In a feeble excuse for his own mistakes and failures, he observed morosely, when the votes went against him, that there are times when the public mood changes, this was one of them, and the change was "for" Margaret Thatcher.

How convenient was such a reading of his brief, Gordon-Brown-like premiership. His turfing out of No 10 was nothing to do, then, with his loss of nerve, failing to call an election when he could have won it in 1978 before the winter of strikes. And that season of discontents: that was nothing to do, we are supposed to suppose, with his own Faustian pact with the trade unions in destroying In Place of Strife, Barbara Castle's reforming White Paper, nearly a decade before.

Yet the real interest of politics is in its uncertainty, in how marginal choices produce outcomes that must appear entertainingly arbitrary to the gods who know what would have happened if the coin had fallen the other way, or if a human impulse had chosen a slightly different expression. And what a case study the past year provides.

It seems much longer ago, partly because it was before the formation of the apparently solid Tory-led government, as Ed Miliband would have us call it, but the year began with the last plot against Gordon Brown.

The same trick of perspective worked for him. Because he looks as if carved out of Scottish granite and has a deep voice, it was assumed at the time that the attempt by silly, disaffected people to unseat him was a deluded enterprise. Yet now we know better. The early histories, by my esteemed colleague Steve Richards (Whatever It Takes) and the historians Anthony Seldon and Guy Lodge (Brown At 10), reveal that Patricia Hewitt and Geoff Hoon nearly succeeded in bringing down their Gulliver.

It was on last New Year's Eve that the plotters met at Harriet Harman's second home in Suffolk over a roast goose supper. A few days later, Hewitt and Hoon issued their call for a vote of confidence of Labour MPs in Brown's leadership. If people had behaved only slightly differently on that day, 6 January, Brown would have been gone by the end of the following week.

At the time, it seemed to fizzle out embarrassingly. What was really happening was that everyone was waiting to see what everyone else would do. Not only Harman, but Peter Mandelson and Jack Straw had had enough of Brown and had decided not to strive officiously to keep him alive. But no one, apart from Tessa Jowell, would tell him it was time to go. As her friend Hewitt told Seldon and Lodge: "It did not succeed because the various people who had told us they were willing to speak to Gordon to ask him to stand aside did not do it."

Perhaps most critical was the failure of either of the most credible alternative prime ministers, David Miliband and Alan Johnson, to make themselves available for service. Miliband had decided, wrongly in my view and as it turned out, that he could not avow a bat's squeak of ambition at any point, and that if he were to secure the leadership it would be John Major's way, after someone else had wielded the knife.

Johnson was not so definite. At one point, he went for a walk up Victoria Street to clear his head, to ask himself if he wanted to be prime minister, well aware that it was close to his grasp. He decided not that it was tactically mistaken to go for it, but that he did not want the job. Of all the things that happened this year, this was perhaps most worth regretting. Johnson's abiding fault has long been a lack of confidence. He often laughs it off with self-deprecation that goes just a little too far, such as when he said he was going out to get an economics primer on being appointed Shadow Chancellor. But it is a shame. He is no less able than David Cameron; he was just brought up differently.

I bridled on his behalf when Gordon Brown called him a "lightweight" and Tony Blair failed to contradict him as they argued over the succession on the day of the September coup in 2006. Blair had been a lightweight once, and his floaty lack of definition had been one of his great strengths. And Brown was heavy only in the sense of heavy going.

If Johnson had replaced Brown in January, he would have been a more formidable opponent for Cameron. A man of the people at ease with the televised debate format, able to offer the best of the Labour brand – that it wanted to keep people in jobs – while shedding Brown's personal unpopularity. Then the election would have been rather different.

We can't know how much better Labour would have done, but the Callaghanite reading of 2010 would have been exposed as a fraud. Tony Blair, in his memoir, wrote: "What the public ended up doing, in that remarkable way they have, is electing the government they wanted. They were unsure of the Tories, so they put a strong Lib Dem showing alongside and urged them to get together."

The idea that a Liberal Conservative coalition was the expression of the semi-mystical will of the people is bunkum. Indeed, one of the reasons why so many voters feel betrayed by Nick Clegg is that, even on the actual result this year, it could be argued that the true consensus was to cut public spending more slowly. If Johnson, Harman, Mandelson and Straw had played their hands only slightly differently, Clegg might still be defending a U-turn on tuition fees, but under a reforming Labour government that had capped them at £6,000 a year.

There was nothing inevitable about this coalition. It was a numerical and historical accident; the outcome of the fragile interplay of a thousand factors. That was no sea change; it was David Cameron's lucky break.

twitter.com/JohnRentoul; independent.co.uk/jrentoul