Leading article: Yemen's autocrat who has played on Western fears to cling to power

Saleh need have no approval over his successor. After 32 years in power his exit is overdue

After Presidents Mubarak of Egypt and Ben Ali of Tunisia, could it be the turn of Ali Abdullah Saleh to be forced out of office by popular unrest? Yesterday, the Yemeni president made a Mubarak-like promise to stand down – but not just yet. The regime has been a key ally of the West in the counter-terrorist struggle against al-Qa'ida; a slide into civil war, which is what Saleh is warning would follow his departure, would be a cause for alarm in Washington and other Western capitals. Yet, after 32 years in power, President Saleh's exit is surely overdue.

For seven weeks, his country has been in the grip of anti-government protests. The situation reached a crisis after a bloody assault by government henchmen killed more than 50 democracy protesters after prayers last Friday. Scores of high-ranking government officials, including five top military leaders, have defected. Among them are the president's half-brother and Yemen's envoy to the Arab League. President Saleh has reluctantly promised to stand down, but not until after parliamentary elections next year.

Dogged by poverty, tribalism and central government dysfunction – Yemen has been the base from which al-Qa'ida launched attacks against targets in the US and Saudi Arabia. The groups known as AQAP (al-Qa'ida in the Arabian Peninsula) are regarded by Washington as a graver threat even than Osama bin Laden's affiliates on the Afghan-Pakistan border. Yemen was the source of failed plots to place bombs on board US airliners in 2009 and 2010 and a Yemen-based cleric has been held responsible for instigating the shooting rampage at Fort Hood in November 2009.

Yemen is the Arab world's poorest country. Its little oil and natural gas is running out, as is water in the north of the country. Many areas have no access to electricity. It is poorly developed, with high rates of illiteracy and unemployment at 35 per cent, and higher for young people. As a society it is violent, poor and tribal: its north has Sunnis and Zaidi Shiites; the middle has a mix of Sufis and hardline Salafis; the south has a separatist movement around the southern port city of Aden. President Saleh has for three decades attempted to manage all this by balancing tribal rivalries rather than through development, state-building or national unity.

But Washington's classic regional ambivalence over the choice between stability and democracy appears to have reached a tipping point after the massacre of protesters. Saleh, who has received millions in US aid for his co-operation in the fight against Islamic militants, is on the point of being abandoned. The Yemeni president has remained defiant, insisting that he cannot stand down without knowing who will replace him, which is why he says he won't go until after elections. The problem is that Saleh has tried such ruses before. Last week he offered a new constitution giving more powers to parliament, and announced an array of handouts. But few Yemenis believe he will deliver.

He need have no approval over who will succeed him. It is true that the struggle of competing visions for the future of Yemen will not end when Saleh leaves. But there is a greater risk of this strategically key country, which borders the world's biggest oil exporter and several major shipping routes, slipping into chaos if Saleh does not go soon and a bloody conflict ensues. That could even see the country split into separate zones along tribal, military or regional lines. A failed state looms.

The genuine participation by all sides in an open and transparent process that addresses the concerns of the Yemeni people is now required. Al-Qa'ida has thrived in Yemen in opposition to the US-backed autocracy. Here, as in other parts of the Arab world, democracy is the best antidote to al-Qa'ida.