Steve Richards: The shadow of Mrs Thatcher still hangs heavy over the party of David Cameron

Earlier this week, Michael Portillo presented a gripping BBC documentary on Margaret Thatcher, her rise, fall and continuing impact on the Conservative Party. Like the best evocations of the recent past, the film shone an unexpectedly bright light on the present. Gordon Brown and David Cameron should watch it. Tony Blair should get hold of a tape and reflect on what might have been. Media bosses who only commission films if they portray politicians as corrupt and mad should take note also.

Portillo's thesis was not especially original. He argued that Thatcher inspired a younger generation of Conservative politicians. The nature of her departure and her subsequent hold on some in the party helped to bring about the calamitous fall of the same doting admirers.

Portillo argued that only now under Cameron is the Conservative Party escaping from her long shadow. What turned the film into a highly charged political drama were the candid, self-critical conversations between Portillo and his former cabinet colleagues, run at length with anguished pauses included. I know people with only a passing interest in politics who were gripped.

There is a message here for some in the media, especially those who run Channel 4. On the basis of its recent political films, Channel 4 would have commissioned a narrative on Thatcher only if Portillo had agreed in advance to prove that she was an alcoholic war criminal. Such a film would have made news stories, and yet would have been boring because based so obviously on the parochial desire of an insecure channel to look bolder than it really is. The BBC is to be congratulated for making the film and giving it the space to breathe.

What came across most vividly was the extent of near terminal turmoil in the Conservative Party after its slaughter in the 1997 election. In the programme, Portillo and William Hague discussed openly their disagreements in the run up to the following election in 2001. Hague acknowledged that Portillo had probably been right in urging him to move towards the centre, but suggested that if he had acted in that way the appalling Conservative defeat could have been much worse. Hague feared the core vote would have deserted the party in protest.

The next leader but one, Michael Howard, was an engaging interviewee in the film, confirming my sense that, against caricature, he is a thoughtful, likeable and self-critical politician. He told Portillo that he came to recognise that he could not be a credible agent of modernisation. He was personally happier with the right-wing themes he chose in the 2005 election. But he acknowledged that he had tested to destruction the idea the Conservatives could win on this basis.

As I watched these reflections, I recalled how much Tony Blair and the rest of the New Labour entourage continued to fear the Conservatives throughout this period. Brought up on election defeats, Blair and most of his colleagues could not see the political space that had opened up in front of them. Blair was wise, of course, to fight a "war on complacency", but he was far too swayed by fears that the Conservatives were on the verge of recovery.

I remember having a cup of tea with Blair during the first term, when he was still seen widely as the messiah walking on water and 30 points ahead in the polls. The meeting was interrupted by an entourage that rushed into the room. They were all in a state of fearful panic because Hague had changed his party's policy on rural chemists. Anyone would have thought World War Three had broken out. On one level, such assiduous professionalism explains Labour's electoral triumphs, but it was also a paralysing factor behind Blair's caution.

The fearful obsession got worse as Labour won more elections. At the start of the second term, Blair told the former cabinet minister, Chris Smith, that he was worried Iain Duncan Smith would seem closer to the US than him now that President Bush had replaced Bill Clinton. Blair told Smith one of his main priorities for the second term was to give IDS no space by showing that a Labour prime minister could work closely with a republican president.

We know what followed. A fear of Iain Duncan Smith's electoral potency partly explains why Blair moved so close to Bush. For too much of its time in office, Labour has been frightened by the Tories when Portillo's film shows it had no need to be.

By implication, the film also showed why the Conservatives are not further ahead now. David Cameron did not inherit a semi-reformed party as Blair did in 1994. On the contrary, he took over a party which had fought the 2005 election on more or less the same platform as the one contested in 2001.

In some ways, Cameron has done well to establish a lead of any sort after inheriting a party that had made no significant headway since its first election slaughter in 1997. But as the political editor of The Spectator, Fraser Nelson points out on its excellent website Blair had a 40 per cent lead at the same stage of the political cycle in February 1995. Currently the Conservatives' average lead is 7 per cent. They are nowhere near ready to win an overall majority.

Politics has a context. Portillo's programme shows the traumatic background in which Cameron's too modest reforms are applied. It would have taken a titanic visionary with a magic wand to propel them into a position where they could claim with genuine confidence that they are ready to win with a significant majority.

Blair inherited a party already well ahead in the polls, following years of painful internal reform, and facing a government in disarray (Portillo's contempt for Major came over strongly in the programme). Cameron took over a party that had fought one disastrous election and then another on Europe, immigration and tax cuts.

This is where I part with the Portillo thesis. He suggests that at last the Conservatives are escaping from Thatcher's shadow. I wonder about that. Large sections of the party are still obsessed with Europe, immigration and tax cuts. Cameron knows that the message needs to be more rounded and less shrill, but he too cannot escape his past entirely, the years in which he first followed politics when Thatcher was the one walking on water. The over-excited way in which he and George Osborne described the pragmatic, reluctant nationalisation of Northern Rock as a leap back to the 1970s suggests that the voters have moved on more quickly from Thatcher than they have done.

Mind you, the timid nervousness of Gordon Brown in relation to the state ownership of Northern Rock shows that he too is trapped by the past. Neither Labour nor the Conservatives have moved out of Thatcher's shadow. Even Portillo could only do so by leaving politics altogether for the safer terrain of television.

s.richards@independent.co.uk