Leading article: An unlikely partnership that could mark the start of a treacherous era

We should not imagine that this will be an easy arrangement. The social democratic wing of the Lib Dems and the Tories' unreconstructed Thatcherites will make for awkward benchfellows

In the end, the progressive coalition failed the stress test. The parliamentary arithmetic for a Labour-Liberal Democrat coalition simply did not add up to "strong and stable" government. The prospect of such an administration being forced to rely, vote after vote, on the capricious support of Scottish Nationalists and Ulster Unionists was too off-putting even for those from both parties who yearned for this historic partnership of the two progressive forces of British politics. Just a dozen more seats for the Liberal Democrats might have made the alliance feasible. But that was not the hand the parties were dealt by the election result.

This is a cause for real regret. A Conservative-Liberal Democrat coalition will not serve the cause of progressive politics in the way that a Labour-Liberal Democrat alliance would have. Liberal Democrat and Labour policies are closer and sounder on a host of crucial areas, from the approach to the economy to our relations with Europe. And of the two larger parties, Labour offered by far the more serious package on electoral reform. A Labour-Liberal Democrat coalition would have left Britain thrillingly close to the long-sought goal of a fair voting system. The Conservatives' pledge to hold a referendum on moving to the Alternative Vote – though better than nothing – leaves us with a mountain to climb as far as serious political reform is concerned.

Yet there are reasons why progressives should not be entirely crushed by this outcome. What many Liberal Democrat and indeed Labour voters feared in the run-up to this election was a rampant and doctrinaire Conservative government of the sort experienced by Britain in the 1980s. This is not what they will now get. David Cameron does not swagger into Downing Street, but rather limps across the threshold. This politician, who once thought power was heading inexorably in his direction, this week briefly stared into the political abyss. That should be a chastening experience for the new Prime Minister.

And Mr Cameron remains very much on probation. The wilder excesses of what is still, in many ways, an ideologically hard-line Tory party should be curbed by the presence of the Liberal Democrats. The fact that there will be Liberal Democrat ministers in the new Cabinet ought to act as a restraining influence. It is hard to see any stunts to repatriate powers from Brussels with pro-Europeans such as Nick Clegg and Chris Huhne in positions of authority. And George Osborne will find it much harder to unleash his right-wing ideological furies on a still fragile British economy with Vince Cable at his elbow. A shackled Conservative government is not a prize to bear comparison with a historic progressive coalition, but it is a prize nonetheless. And the presence of the Liberal Democrats might, ironically, make it easier for David Cameron to face down his own vociferous right wing. This coalition might end up helping to civilise the Conservative Party.

Yet we should not imagine for a moment that this will be an easy arrangement. The social democratic wing of the Liberal Democrats and the unreconstructed Thatcherites of the Conservatives will make for awkward benchfellows in the Commons. And the Labour Party, some of whose MPs are already savouring the prospect of being the "progressive opposition", will not make life easy for the new coalition government.

There is also likely to be considerable external pressure on the Liberal Democrats in the coming months. This ideologically uneasy coalition will severely test the unity of Mr Clegg's party. It will also test the Liberal Democrat support in the wider country. Few people voted Liberal Democrat last week in the hope that it would help deliver a Tory government. Yet that is what their vote has ended up doing. This is a perilous moment for Mr Clegg's party. Smaller coalition partners have a habit of getting squeezed. The Liberal Democrats will have to be careful that they do not get outmanoeuvred by the Conservatives and end up taking the blame for unpopular decisions.

There will be those from across the political spectrum and in all parties who hope this alliance will be strangled at birth. But they should be careful what they wish for. We have heard a great deal of talk about the "national interest" from our political leaders in recent days. It is easy to be cynical about this phrase. But a national interest – shared goals above partisan politics – does exist. And it exists especially on the economy at the moment.

Britain's public borrowing is at record levels and there is a mighty economic storm on the Continent that could easily turn our way. This new government needs to make it clear to the financial markets and the wider public that it will do what it takes to get the deficit under control. This does not necessarily mean cuts this year, despite what the Tories have argued. It means a coherent plan to reassure investors in British debt that our politicians are capable of working together and making the inevitably unpopular decisions.

In the longer term, this coalition will be judged on the extent to which it delivers a new way of doing politics. And there have been encouraging signs in recent days that this could be possible. The manner in which the Conservatives reached out to the Liberal Democrats after it became clear that the result of the election was a hung parliament suggests that British politicians, despite their inexperience, are capable of coping with coalition politics. This matters because if we are to get electoral reform this sort of negotiation in the wake of elections will be the norm rather than the exception. The Liberal Democrats have done their bit too. By recognising that the Conservative Party, as the winner of the greatest number of seats and votes in the election, had the first right to seek to form a government, they showed that they understood their own responsibilities.

Yet the acid test of this coalition is one of good faith. The pledge from the Conservatives, extracted by the Liberal Democrats, to hold a national referendum on changing the voting system is of fundamental importance. That vote must take place. There can be no delays, nor attempts to get around it. Britain finds itself in an unfamiliar and potentially treacherous political world this morning. But, just like in the old one, this newspaper will be pressing relentlessly for a more open and democratic way of conducting politics.