Scarlett's best service would be to resign

Mention MI6 and henceforth the response will always be the smile, the 'well-they-would-say-that, wouldn't-they'

For all his adult life, John Scarlett has served his country. During 30 years he worked solely for the Secret Intelligence Service (SIS). Then in 2001 he became chairman of the Joint Intelligence Committee (JIC). Now he stands ready to take over as chief of the SIS. The Prime Minister has announced his appointment. Lord Butler, in his review of intelligence on weapons of mass destruction, supports him. Unfortunately Mr Scarlett is damaged goods. His last, best service for the state would be to step down.

For all his adult life, John Scarlett has served his country. During 30 years he worked solely for the Secret Intelligence Service (SIS). Then in 2001 he became chairman of the Joint Intelligence Committee (JIC). Now he stands ready to take over as chief of the SIS. The Prime Minister has announced his appointment. Lord Butler, in his review of intelligence on weapons of mass destruction, supports him. Unfortunately Mr Scarlett is damaged goods. His last, best service for the state would be to step down.

Leaving aside for the moment the so-called dodgy dossier, the work of the JIC during Mr Scarlett's period as chairman was at best patchy. Its task is to provide the Government with intelligence assessments that distil all that is known about a particular situation. It draws on published sources and on material provided to the Government in confidence. It supplements this with the work of the SIS and the Security Service (MI5). It also has available the electronic surveillance carried out by the Government Communications Headquarters (GCHQ). In addition it can see the analysis carried out by the Defence Intelligence Staff, part of the Ministry of Defence.

A good assessment draws fully on open and secret material, it brings together human and technical intelligence, it offers a view on the quality of the intelligence sources, it identifies past errors and corrects them, it considers alternative hypotheses and scenarios, and, above all, it is balanced and measured. During Mr Scarlett's time, the JIC assessments of Iraq's nuclear capabilities were done in the classic manner. But assessments of the threat from chemical and biological weapons were of lower quality and sometimes misleading.

Looking at one such assessment dated 21 August 2002, just before work on the dossier began, Lord Butler's committee states that it was struck by the relative thinness of the intelligence base supporting the JIC's judgements in this area. It notes the inferential nature of much of it. It also found that the JIC did not reflect material in one of the reports available to it suggesting that most members of the Iraqi leadership were not convinced that it would be possible to use chemical and biological weapons. This last observation should have been mentioned even if only to dismiss it.

More generally, the JIC developed a bad habit of taking a worst-case estimate, describing it as such first time round only to drop the qualification in later reports. As a result, "there was a risk of over-cautious or worst-case estimates, shorn of their caveats, becoming the prevailing wisdom" - or "groupthink" as the American committee investigating the CIA called the process.

Thus Lord Butler's report suggests that what the American author Bob Woodward called the "they-can-attack-in-45-minutes shit" was not an isolated aberration. Rather it was the worst example of a gung-ho style that was present during Mr Scarlett's chairmanship and had perhaps begun to develop before he arrived. The claim in the JIC's assessment was that "chemical and biological munitions could be with military units and ready for firing in 20-45 minutes". But firing by what means exactly? If by ballistic missiles, then the report would be both surprising and frightening. If by battlefield weapons with a range of a few miles, then much less so. The relevant JIC assessment was unclear on this crucial point.

This poor to middling performance by the JIC would normally have passed unnoticed by the general public. Alas for poor Mr Scarlett, he emerged too soon into the limelight by taking full responsibility for the notorious report entitled "Iraq's Weapons of Mass Destruction" issued by the Government, with a foreword by the Prime Minister.

Mr Scarlett insisted on undertaking this task but he was wrong to do so. He had no experience of preparing material that would be easily understood by the general public. As a result, to make good this deficit, he found himself bullied by the Prime Minister's staff into a drafting activity that the BBC's Andrew Gilligan correctly described as "sexing up". In the understatement of the year, Lord Butler's team comments that "this will have put a strain" on the JIC.

Indeed it did. The Butler report finds that it was a serious weakness that the JIC's warnings on the limitations of the intelligence were not made sufficiently clear in the dossier. In other words, Mr Scarlett had taken responsibility for a document which, when published, had the effect of deceiving the public. I say "deceiving" because the drafting was done by people who know very well how words work.

Lord Butler says Mr Scarlett shouldn't be blamed for this. Lord Butler's wonderfully convenient theory of collective responsibility - convenient, that is, for the sort of people with whom Lord Butler has spent his life, ministers and high civil servants - covers Mr Scarlett. In a passage which confirms that Whitehall is a blame-free area, he states that the chairman of the JIC should not bear personal responsibility for what was a collective decision.

Much of what I have described above was already suspected. But unknown until a few days ago was a further gross error of judgement by Mr Scarlett. He failed to tell the Hutton inquiry which began its work in August 2003 that some of the findings published in the dossier had been withdrawn by the SIS in the previous month. The relevant "sourcing chain" had been discredited. Such information would have influenced the way counsel for the inquiry conducted their examinations of witnesses. Admissions might have been secured that would have changed Lord Hutton's opinion of the story he was hearing.

In summary, what have we learnt about Mr Scarlett? That he was an ineffective chairman of the JIC. That he prepared a dossier for publication by the Government that misled the electorate. That he wilfully failed to bring significant information to the attention of the Hutton inquiry. Incompetence, deceit, contempt of a judicial process - not bad going for a high civil servant.

What would be the consequences of his appointment? There would be a diminution in the trust that is placed in the work of the SIS. With Mr Scarlett in charge, people would always wonder whether its output wasn't being tailored to tell ministers what they wanted to hear. Mention the SIS and John Scarlett and henceforth the response will always be the wry smile, the shrug of the shoulders, the well-they-would-say-that-wouldn't-they.

Worse still, there would be a further small decline in the respect with which the public holds ministers and senior civil servants.

Their world, where no one says sorry for their mistakes and no one accepts responsibility for their actions, is not where the rest of us want to be.