Sean O'Grady: Millbank's Mafia has left the building, but the Millbank Tendency remains

Politics has a habit of appropriating the most unlikely of places. Even after 40 years, the mention of Orpington to a Liberal heart will stir the memory of Eric Lubbock's stunning by-election victory. Cable Street in Whitechapel, east London, has few of its original buildings, but it is well remembered as the scene of heroic resistance against Oswald Mosley's fascists in the 1930s. Jarrow, Aldermaston, Stormont, Selsdon Park, Orgreave, Greenham Common and many others still resonate. So too, rather less heroically, does Millbank Tower, London SW1, until this weekend the headquarters of the Labour Party, and an address now synonymous with the black arts of spin and media manipulation.

It entered the lexicon of politics and journalism almost as soon as Tony Blair officially opened the party's media centre in January 1996. "We never believe that message is a substitute for substance, but it's important that both go together," he declared. I'm not sure many of us believed that then, or even that he did, and we certainly don't now. The thousands of speeches, images, soundbites and photo-opportunities that have spewed from the spin factory since have all contributed in their own small way to the conviction that Labour is indeed more about message than substance. It has fed the very media obsession with process over policy that ministers such as Charles Clarke (himself no mean spinner in his day) complain about. Will the move from the ugly 1960s glass tower block to the classier surroundings of 16-18 Old Queen Street help the party shrug off its association with the worst ways of modern politics?

It might. In fact, Old Queen Street itself was the HQ of the Tory party during the Second World War and from 1930-81 was the home of the Conservative Research Department, the power base for Neville Chamberlain and R A Butler, and boasting such notable alumni as Chris Patten and Michael Portillo. Little of that old association now survives.

Labour certainly hopes people will similarly forget all about the Millbank Tendency and the Millbank Mafia. "The days of Millbank are over," says Edward Morgan, assistant general secretary of the party. Spinning till the end he adds: "It was somewhat unfairly identified as a symbol of an obsession with spin, control freaks and sleaze. Changes were needed and everyone is delighted to be moving to the new building."

But was it as bad as all that? Which would you rather have running the Labour Party: the Millbank Tendency or the Militant Tendency? For most Labour members and supporters, I suspect, the answer is simple. "Neither" would be their honest reply, but, if pressed, I suspect that they would agree that Labour couldn't go back to the bad old times. They wouldn't, we can safely assume, like to return to the days when a gang of Trotskyists with no hope of winning electoral support under their own colours tried to hijack Labour for their own barmy ends. They were a malign cancer that almost killed the party. Say what you like about Tony Blair, Alastair Campbell, Peter Mandelson and the rest, but they did actually want the party to succeed and return to power so that it would be in a position to do something for the people it was supposed to represent. It is only fair to mention that they did actually succeed in that.

None of which is to say that the behaviour of the Millbank Tendency has always been wholly admirable. Just like Militant before them they have a less than healthy respect for democracy. Some of them with a background in hard-left politics retained some vestigial affection for Leninist disciplines of democratic centralism. Never forget that Peter Mandelson was once a member of the Hampstead branch of the young Communists. A lot of outrageous things were done in the name of New Labour up in Millbank Tower. They lied to journalists – not that we hacks expect much sympathy for that. When they weren't lying they could be spectacularly and entertainingly abusive.

This is an appropriate moment to retell a story told to me by a female colleague who had run into the usual brick wall of silence when she was writing a piece about Labour's then impending move to Millbank seven years ago. She went ahead anyway and, when the offending article came out, complete with accurate details of the cost of the premises, Tony Blair's new press secretary, Alastair Campbell, told her that she could "stick it [the 387ft-high Millbank Tower] up your arse". There was also a lot of self-important posing at the back of press conferences by the likes of Campbell and Charlie Whelan. Hardly a crime, but a bit pretentious. The spin doctors blackened or "down-spun" colleagues' reputations. They tried to bully the media into pulling stories they didn't like. Just politics, I suppose, and the media should be big and strong enough to stand up to it.

But some of the excesses were genuinely repugnant. The fixing of the party's nominations for London mayor and leader of the Welsh Assembly were a disgrace. By twisting arms , calling in favours and relying on the trade union block vote, the party's apparatchiks overruled the membership, abandoned their supposed devotion to one member, one vote and shafted Ken Livingstone and Rhodri Morgan, only to see the electorate bite back by delivering miserable Labour performances in the London and Welsh elections. When I put it to two of the sultans of spin shortly afterwards that they might have something to learn from the Livingstone episode they agreed, but said that the lesson was that they had to be better, more effective, control freaks. The antics of Stephen Byers and Jo "a good day to bury bad news" Moore seemed to epitomise all that was wrong with the Labour operation.

It was all dreadful, but not, it must be said, as dreadful as having Margaret Thatcher and John Major running the country for 18 miserable years. There was a reason why the Labour Party moved to Millbank and that was that it had to modernise the way it worked, and learn the lessons of the Tories' success and, in particular, of the Clinton campaign's war room in the American election of 1992. The rabbit warren of Walworth Road, as Blair's polling expert and strategist Philip Gould called it, was too far away from the media pack at Westminster and too shambolic to serve a media more extensive snf demanding than ever. It was the home of decline and failure, of splits and rows. The Labour Party lost three out of the three elections it fought from Walworth Road; it won both the elections it fought from Millbank Tower, and by landslides. Millbank should be seen not just as a powerful symbol of spin and excess, but also as a powerful symbol of electoral success. Labour should not forget that.