Behind closed doors: The bewildering dance between Gaddafi and MI6

Britain and the Gaddafi regime were not always the enemies they are now. Gordon Corera recalls the secret deals and clandestine contact – and the key role played by the Colonel's son Saif

With Tripoli falling and the Gaddafi family on the run, those inside British intelligence who have worked on Libya over the last 10 years might be excused a wry smile at the way events have panned out.

Gaddafi has morphed from enemy to friend and back to enemy again in an almost bewildering dance, in which MI6 and Britain's spies have sometimes been leading and sometimes following. That dance is now approaching its end as the secret service hunts the man with whom it once sat down to deal...

Back in March 2003, as the bombs were about to fall on Baghdad, a phone call had come into MI6 headquarters at Vauxhall Cross. It was from a Palestinian who had a message from the Libyan leader Colonel Gaddafi. It was to be the start of a strange entanglement with an old foe.

The leader's son, Saif, wanted to deliver the message in person. The rendezvous was arranged in the private room of an upmarket Mayfair hotel.

Present were two MI6 officers. Saif was nervous. He had not met with anyone from the British secret service before and had been brought up to think of them as the enemy bent on destroying his father's regime. Now he was sitting opposite them. He explained that his father wanted to talk about weapons of mass destruction. On the day that "shock and awe" was unleashed on Iraq, a plane took off secretly for Libya.

The two British intelligence officers on board headed for Sirte, desert headquarters of the eccentric colonel, where some wonder if he may now make his last stand. There he would hold court in a huge Bedouin tent with camels roaming outside. His decision was all about survival. Gaddafi was worried about being next on the hit list after Saddam and wanted to see if he could buy his way back into the international community by giving up a weapons programme that he thought no one knew about. But it was not as secret as he thought. The nuclear programme had been supplied by Pakistani salesman, AQ Khan, whose network had been penetrated by MI6 and the CIA.

Gaddafi told the leader of the MI6 team, named widely as Mark Allen, that his Libyan counterpart for the negotiations would be the intelligence chief Moussa Koussa. Allen went with MI6 chief Richard Dearlove to Washington a few days later, briefing President Bush, who assigned CIA officer Steve Kappes to lead their side.

Koussa's connections to Britain and America went way back. He had studied in the US and been expelled from Britain for threatening the murder of a dissident. Libyan intelligence's links to international terrorism were also well known. For the British – who had counted the cost of the bombs and guns supplied to the IRA by Libya – sitting down with Koussa meant swallowing their discomfort.

The Libyans were nervous about revealing what they had procured (even though it was far from operational), because they feared that their opponents could simply walk away from secret negotiations and use the information as a pretext to attack. Libya's evasiveness, in turn, inspired mistrust from MI6 and the CIA, who knew how much the Libyans were hiding. At one point, a retired British military figure had to be sent out, as a gesture of good faith, with the message that a deal really was on the table. As months passed, the mercurial, unpredictable – but always calculating – Gaddafi may also have been watching the "victory" in Iraq turn sour and begun to wonder about the appetite for another war.

By the summer of 2003, a frustrating impasse had been reached. A risk needed to be taken. A source revealed that a consignment of nuclear parts was destined for Libya. The boat carrying them, the BBC China (which had nothing to do with the BBC or China) was diverted in the middle of the night to an Italian port, where five cargo containers were prised open to considerable relief. Allen called Moussa Koussa to confront him. The Libyans agreed to allow in a team of inspectors.

A small, unmarked plane carrying a joint CIA-MI6 team of technical specialists departed from Northolt airfield for Libya on 19 October. Only a handful of officials on each side of the Atlantic knew about the trip. The team was taken to sites by Libyan guides by day, before filing reports back to Langley and Vauxhall Cross by night. But the Libyans continued to be evasive. After 10 days, the group returned home, frustrated. The enigmatic Moussa Koussa was invited to Britain to discuss the problem. Intelligence was literally laid out on the table before him, to highlight the differences between what Libya was claiming and what was known to be true, including a recording of a conversation between AQ Khan and Libya's nuclear chief.

Koussa conceded a second inspection visit which began on 9 December and proved nearly as frustrating. Rain fell on the last full day, as the possibility of failure hung over the visiting team.

In the dark before dawn the next morning, they headed to the airport. As they prepared to board, the Libyans handed over a stack of brown envelopes about a foot high. Inside was the design for a nuclear weapon. It was what they had been waiting for. The final outlines of the deal were hammered out a few days later in a private room at the Travellers Club on Pall Mall. On 19 December the secret diplomacy finally became public (although not until Gaddafi had waited for a football match to conclude).

Within a few weeks, President Musharraf of Pakistan was pressurised to put AQ Khan out of business. These were crucial victories, but at the time only a sideshow to the growing crisis over Saddam's absent weapons of mass destruction.

The strange coda to the tale of Moussa Koussa, Gaddafi and MI6 would come years later. Over time, the Libyan leader became disillusioned with the deal he had struck. Western oil and gas companies welcomed him with open arms and Tony Blair visited him in his tent, but Gaddafi never felt he got as much as he wanted. The deal had only ever been about survival and when that was threatened this year by protests, the leopard showed his spots with a violent crackdown. If he was now a problem again, at least – those who worked on the 2003 deal reflected – he was a problem with no chemical weapons ready to fire or an active nuclear weapons programme.

Because he had been the interlocutor, Moussa Koussa had been closely associated with the deal and had paid the price in the preceding years as it soured.

He had been shifted from intelligence chief to Foreign Minister (Libya, as with other security-states being one of the few countries where such a move is a demotion). He still did not flee immediately when the uprisings began – he was too calculating for that – instead he made his move when it was clear the regime was under serious threat and NATO began its air campaign.

Despite his history with MI6, Britain was not his first choice. The French had originally approached him at a conference in Ethiopia and then later at the end of March in the Tunisian capital. But at the last minute Paris demanded a public denunciation of Gaddafi which may have been a step too far for the wily operator. So, stuck in Tunisia, half-in and half-out and in a tricky situation, he turned to the British. Events moved extremely fast, as Koussa demanded an immediate exit.

Over less than 24 hours, plans were put in place (officials dislike saying a deal was struck) and he was flown to Britain. This appeared a useful coup for MI6, after a disastrous start to the Libyan campaign, when one of its officers – who was helicoptered into rebel held Libya – was embarrassingly captured and held for a few days by rebels. On arrival – and after some problems with the paperwork – the former Foreign Minister was taken off to be interviewed. Officials say the debriefing process in Britain was, in their words, "worthwhile", providing valuable information on the state of the regime.

I met with Koussa a few days later. After being approached by a member of his family, I was then contacted by a police officer who asked me to wait outside a nightclub on a well-known square in central London from where, he promised, I could then follow his car to a secret rendezvous point to which Koussa would be brought. The rendezvous turned out to be in the basement of a hotel less than a hundred metres away.

There were no British officials present, save for his security detail as Koussa arrived, dressed nattily with a flat cap, thick glasses and a scarf (whether for disguise or fashion was unclear). Ever the controlling spymaster, he remained cool and detached and declined to answer any questions, instead insisting simply on making a statement in Arabic. Its tone was careful – not too condemnatory of Gaddafi, clearly trying to position him as a dealmaker.

The following day he was gone. The temperature was too hot, with headlines describing him as "fingernail-puller-in-chief" and "up to his eyeballs in murder and torture" and politicians questioning why he was being allowed to stay here.

There had been no offer of immunity from prosecution because of the outstanding questions relating to the Lockerbie bombing and the killing of WPc Yvonne Fletcher. If Koussa had not been personally involved, then he would certainly know who had been. Using the pretext of a meeting of opposition groups, he made his way to Qatar. At first, British officials said they expected him to return soon, but that talk soon faded. And what of his former master? Now, as the Gaddafi regime collapses, MI6 is no doubt hunting the men with whom it once met. Saif – who sat in that Mayfair hotel room – is on the run and indicted by the International Criminal Court. His father's whereabouts remain unclear.

After its initial failure, MI6 sent another team into Benghazi to coordinate with the rebels. The support of western countries and Qatar has largely been covert, but its impact has been significant in turning a rag-tag band of fighters into a group that could take Tripoli. MI6 will also have tried to recruit Libyans as agents to provide information. If they have been successful, they may even have people close to Gaddafi. Those contacts will now be pressed for information on where he might be. GCHQ will be doing its bit to help locate him.

The final chapter of the strange entanglement between Britain, MI6 and Gaddafi is now being written.

The Art of Betrayal – Life and Death in the British Secret Service by Gordon Corera is published by Weidenfeld and Nicolson