Defeat: Why They Lost Iraq, by Jonathan Steele <br/>Muqtadr Al-Sadr and the Fall of Iraq, by Patrick Cockburn <br/>After Iraq, by Gwynne Dyer

How the war was lost

In the weeks up to the end of March, a narrative of impending victory in Iraq was promoted by Republican commentators in the US. Bush's troop surge was working, casualties were down and there was a prospect of political progress. From their perspective things looked good – there was every sign that the Iraq war would no longer hamper John McCain as he campaigned for the White House.

Better still, he could even claim that both Obama and Clinton might well go for a rapid military withdrawal that would snatch defeat from the jaws of victory. General David Petraeus was due to testify before Congress, and all seemed far better than even the most neoconservative commentator could have expected at the start of the year. In the space of two weeks, it all went sour. Violence erupted in Basra and Baghdad as the Iraqi government tried to crush the Mehdi Army, civilian casualties went back up, the secure Green Zone in Baghdad lost its aura of safety, and Petraeus cautioned against excessive optimism.

In this sixth year of a seemingly interminable war, these three books offer a range of explanations of what went wrong and what might be the consequences. Jonathan Steele has been a frequent visitor to Iraq, reporting during some of the most dangerous of times. Defeat is his detailed analysis of the early years of the war. For Steele, the most common arguments of those criticising the war – the failure to plan for the post-war era and a series of monumental blunders in the immediate aftermath of regime termination – are wrong. He has a more fundamental criticism: that occupation of an Arab state such as Iraq in the early 21st century is fundamentally untenable. It simply cannot work.

From an Arab perspective, while the end of Saddam Hussein may have been welcome, for his regime to be replaced by a Western occupation involving the two key interferers in the Middle East – the US and Britain – is wholly unwanted and a source of deep affront. In an impeccably written and thoughtful book, Steele concludes that there is no alternative to withdrawal, expressing enduring surprise that neither Blair nor Bush can see what they have unleashed.

Steele's book was published as the surge appeared to be working, but with the renewed violence it retains both its relevance and good sense. The Independent's Patrick Cockburn is more fortunate with his study of Muqtada al-Sadr, in that the timing is impeccable. Iraqi government conflict with al-Sadr's Mehdi militia in Basra and Baghdad has sucked in the US military to its most violent exchanges for months, just as its surged troops had finally begun to withdraw.

The title is somewhat misleading: this is much more than a study of al-Sadr, with most of the book being a broad discussion of Shi'a Iraq, even if the emphasis moves steadily towards the Sadrists. Cockburn is unable to get close to the personality of al-Sadr, a disappointment common to every Western analyst, no matter how expert. Where he scores is in his knowledge of post-invasion Iraq, enhanced by a long-term interest in the country.

Three aspects of al-Sadr and the Mehdi militia are key to understanding his influence. One is the impeccable family credentials, particularly his father, who was assassinated by Saddam, and the manner in which these resonate with a substantial minority of Iraq's population. His family did not exile themselves, even in the worst periods of the Saddam era, in marked contrast to most leading Shi'a figures.

Second is his formidable connection with Iraq's Shi'a underclass – the millions of people in Sadr City and elsewhere who have been marginalised not just by the Sunni elite of the Saddam era, but by those Shi'a political parties favoured by Washington. Finally, al-Sadr is much more astute than most people have presumed and this, combined with his caution over links with Tehran and his nationalist leanings, make him perhaps the most important political figure of the era.

Cockburn is pessimistic about Iraq's future, fearing that the divisions are now too deep for any sense of national unity to evolve. Gwynne Dyer's After Iraq goes beyond the immediate circumstances, assumes that the American occupation will not last, and then asks what is the future for the Middle East. His answer, in essence, is that the region is best left to itself and any belief that Western states can determine its future is doomed to failure. Even the Islamist radicalisation may not survive – while much of it may be a reaction to the rule of local elites, the movement has been aided by external interference.

Dyer is most pessimistic in his assessment of Israel's precarious position, doubting whether current US support will persist. He is also dubious about the military significance of the region's oil reserves, arguing that whoever rules the countries of the Persian Gulf will still have to sell their oil. Both views are open to question – the Israel lobby has got a considerable boost from the growing influence of Christian Zionism in the US, and there is a 35-year history of US military interest in Gulf oil.

Even so, Dyer writes with a racy style and provokes as much as he informs. He does not share the experience of Steele and Cockburn – among the best in British journalism – but he does leave one amazed at the sheer incompetence of the American and British governments of recent years.

Paul Rogers is professor of peace studies at Bradford University; his latest book is 'Why We're Losing the War on Terror' (Polity)