Never mind Ukip, it's the Lib Dems' Brexit agenda that Labour should be afraid of

The danger is that socially liberal voters opposed to Brexit will give up on Labour and drift on mass to Farron’s party

Liberal Democrat candidate Sarah Olney with her husband Ben and party supporters celebrate after winning the Richmond Park by-election (PA)

Last Thursday's Richmond Park by-election sent convulsions through Westminster. The focus has turned to whether the Liberal Democrats’ victory will force Theresa May to recalibrate her Government’s approach to Brexit, raising the spectre of renewed infighting and a fracturing of the Tory coalition, as affluent pro-Remain voters turn away from the Conservatives. With a Commons majority of 13, the Prime Minister’s political acumen will be tested to the limit. Yet for all the Tories’ travails, it is the Labour Party that ought to find the Richmond result most perplexing.

Labour’s share of the vote collapsed to 3.6 per cent, depths not plumbed for a generation. For an opposition that aspires to be in government, losing your deposit in a contest fought on an issue of overwhelming national significance ought to give pause for thought. So what are the lessons Corbyn’s party should digest?

The first is that Labour is having its electoral coalition ripped apart. Some party figures argue it is Ukip that poses an existential threat: policy on Europe and immigration must change to regain the trust of working class voters. Not all working class areas voted for Brexit, nor is hostility to immigration wholly determined by class identity. Moreover, the hard reality is that socially liberal voters outnumber those hostile to diversity, immigration and Europe, as Professor John Curtice has pointed out. Analysis by YouGov demonstrated that in a general election, where the Lib Dems took a strong position against a hard Brexit while Labour equivocated, Corbyn’s party would be beaten into third place. The danger is that socially liberal voters opposed to Brexit will give up on Labour and drift on mass to Farron’s party. Whatever electoral calculations the party’s leadership make, Labour will never be a convincing Eurosceptic or anti-immigration force.

The second lesson is that Labour is making its position worse by failing to develop a coherent Brexit policy. Waiting for the Government to come to grief is hardly a viable strategy: Labour has to articulate a plausible alternative. The Lib Dems won Richmond because they outlined a distinctive pro-European approach, committing to a second referendum on the terms of Britain’s departure. With their niche middle class appeal, the Lib Dems can afford to take the risk of ignoring the referendum outcome. But that doesn’t excuse Labour’s ineptitude. Shadow Cabinet figures cannot agree about whether the priority in Brexit negotiations should be to preserve access to the single market, or to accept new restrictions on EU migration. John McDonnell, the combative shadow chancellor, indicated a hard Brexit might actually be advantageous for Britain. Voters are baffled about Labour’s actual position.

The consequence is that the party appears increasingly irrelevant to the national political conversation. Too few voters see Labour as a viable party of government; even fewer regard Corbyn as a potential Prime Minister.

In the meantime, Labour has little comprehension of the historical forces reshaping British politics. If the fundamental question in political debate becomes whether you are "for" or "against" Brexit, the traditional divide between left and right will have less resonance. As identity politics encroaches on economic issues, traditional voter loyalties and allegiances break down. Brexit has cast a long shadow, threatening to provoke a structural realignment of the party system.

Ingrained tribalism prevents the Labour Party from co-operating with other progressive forces on the centre-left. Labour’s antipathy to the Liberals originates in the early 20^th century, but on an issue of such profound national importance as working to prevent a hard Brexit, voters find Labour’s sectarian outlook incomprehensible. David Marquand’s influential book The Progressive Dilemma demonstrated in the early 1990s that where the liberal left vote is divided, the consequence is long-term Tory rule.

Yet the much vaunted concept of a "progressive alliance" is scarcely a panacea: in 2015, the Tories destroyed Ed Miliband’s chances of victory, portraying the "nightmare vision" of a minority Labour Government "held to ransom" by the SNP and the Greens. Any centre-left coalition will be defeated if the parties are not perceived to be economically credible, while the Tories offer competence and security. If Labour cannot get its act together to provide convincing opposition, other progressive centre-left forces will emerge to try to prevent this century being dominated by the Conservative Party, as was the last. Corbyn’s party is living on borrowed time.

Patrick Diamond is Lecturer in Politics at Queen Mary, University of London, and a former Downing Street policy adviser to Tony Blair and Gordon Brown