The BBC is facing its most deadly political threat since its foundation – but its spirit should be revived, not crushed

Thanks to a blinkered and self-serving management, the corporation is now under siege from neoliberals intent on reducing both its remit and its competition with the commercial channels, says Peter Oborne

The main entrance to the BBC headquarters in London (Getty Images)

John Whittingdale, newly appointed Secretary of State for Culture, Media and Sport, is one of the more devout followers of Margaret Thatcher. As a young man he worked in her office, and was inside No 10 Downing Street at the time of her political assassination in November 1990. He has always been clear that he owes his career to the former prime minister.

Whittingdale is also close to the court of Rupert Murdoch. As chairman of the Culture, Media and Sport backbench committee, he would mingle with News International henchmen to an extent that raised eyebrows. If Whittingdale was subsequently embarrassed by the revelations of criminality inside the News International empire, he never showed it.

Since his surprise elevation to the Cabinet in May, Whittingdale has made some emollient comments about the BBC. Until then, however, he was an unremitting critic. He shares the antipathy of the Murdoch media empire to the licence fee, claiming that he dislikes it on the basis that it is a form of regressive taxation, comparable (so he asserts) to the poll tax.

This is odd. Whittingdale's record suggests that on other matters he strongly favours any tax regime that favours the rich. For instance, he did not complain about the reduction in the top rate of income tax to 45p. He has not yet explained why, on the narrow matter of the BBC licence fee but on nothing else, he should find the thought of regressive taxation repellent.

John Whittingdale's views on the BBC repay inspection because the organisation's charter comes up for renewal next year. The new Culture Secretary is the spokesman for a very powerful body of right-wing opinion that has long been determined to weaken and even to destroy the BBC as a national institution, and clearly sees its opportunity in the wake of the 2015 general election victory.

There are two fundamental grounds for this assault on the BBC. One is connected to neoliberal dogma. Right-wingers ask why a publicly funded body should be asked to produce mass entertainment, when the private sector is capable of carrying out the same function just as well if not better. The neoliberals would therefore like to slim the BBC down to a handful of core functions, above all news and current affairs, which the market is unable to provide.

This faction questions the very existence of the licence fee and admires the broadcasting model in the United States, where public broadcasting has a tiny presence and news reporting is influenced by major advertisers and other commercial interests. Judging from his statements before becoming Culture Secretary, John Whittingdale is susceptible to this line of argument. He has even questioned whether the BBC should show Strictly Coming Dancing.

The second argument is about political bias. Many Conservatives (as well as some from the left) believe that the BBC routinely flouts the statutory guidelines that insist on impartiality. This is complicated territory because judgments about bias tend to be subjective. Many of those who attack the BBC belong to the newspapers or media, groups whose own political reporting is hopelessly skewed.

To make matters murkier still, the loudest critics tend to work in organisations that stand to gain most from the destruction of the BBC. For example, the end of the licence fee has long been a strategic objective for the Murdoch empire. Significantly, Murdoch's News International has reforged its historically close relationship with the Conservative party now that the phone-hacking scandal is past.

To sum up, the BBC now faces a more deadly political threat than at any time since its foundation in the 1920s. This comes at a moment when, in common with all traditional media, it is struggling to deal with the explosive rise of the new technologies connected to satellite broadcasting and the internet. It finds itself fighting for a presence in an ever more chaotic market that no one really understands.

BBC’s nemesis: John Whittingdale, Secretary of State for Culture Media and Sport, was criticised for his closeness to the Murdoch empire when he was chairman of the Culture, Media and Sport backbench committee (Getty) (Getty Images)

Three books have been published at a moment of mortal crisis: Jean Seaton's Pinkoes and Traitors: The BBC and the Nation, 1974 – 1987, Roger Mosey's Getting Out Alive: News, Sport and Politics at the BBC and Charlotte Higgins' This New Noise: This Extraordinary Birth and Troubled Life of the BBC.

In the first of these, Jean Seaton has taken over the role of official historian from Asa Briggs, whose massive study took five volumes to cover the ground from 1922 (the year of the BBC's foundation) to 1974.

Professor Seaton takes the story forward a further 13 years to 1987. She is wise to treat her subject matter thematically. Politically, there were exhausting rows over the coverage of the IRA terrorism, as well as relentless Conservative hostility culminating in the brutal removal of Alasdair Milne as Director-General in 1987.

Nevertheless, it deals with a period when the BBC had a powerful presence at the heart of British national life. There were only four TV channels: BBC1, 2, ITV and (from 1982) Channel 4. Sky TV had not yet happened. During this time the BBC commissioned an endless list of glorious programmes. This was the era of Fawlty Towers (1975-9), Yes Minister (1980-84) David Attenborough's Life on Earth (1979), Ronnie Barker's Porridge (1974-7), The Two Ronnies (1971-7) and the Alec Guinness portrayal of George Smiley, the hero of John Le Carré's Tinker, Tailor, Soldier, Spy (1979).

That's Life, presented by Esther Rantzen, a programme that drew snobbish sneers at the time and whose importance is still hugely underestimated, precisely fulfilled Lord Reith's founding mission statement to combine entertainment and instruction. The BBC superbly staged great national events, such as the 1981 royal wedding. It televised the Proms.

There are omissions. It is a pity that Professor Seaton's exuberant work does not contain a section on the BBC's European coverage, and in particular the 1975 referendum on British membership of what was then called the Common Market. The corporation (in common with the mainstream press) became an instrument of the pro-Europe propaganda machine, and there are surely lessons to be learnt ahead of the impending EU referendum. Nevertheless, Jean Seaton does justice to a glorious period in the corporation's history in optimistic and vivid prose.

Roger Mosey's agreeable memoir takes the BBC story into the more troubled modern era. Now Master of Selwyn College, Cambridge, he embodies the most attractive qualities of the modern BBC: its decency, classlessness and an earnest but muddled determination (often thwarted) to get things right. Educated at Bradford Grammar School and Oxford University, he joined the BBC as a reporter for Radio Lincolnshire, then became an editor on the Today programme and later its editorial director.

He has written a thoughtful book, which is not blind to the corporation's mistakes: "I believe most people in the BBC are fair-minded," writes Mosey, "and journalists have a professionalism that means they steer well clear of party political bias. I have never had a problem with an editor in favour of the Conservatives, Labour or any other party. But there can be a default to 'groupthink' – a set of assumptions that seem reasonable to everyone they know."

This groupthink, suggests Mosey, embraces the EU and immigration. The rise of Ukip, with its fierce opposition to both, came as a profound shock to the BBC mindset.

George Entwistle's tenure as BBC Director-General lasted just seven weeks (Getty) (Getty Images)

When Ukip gained its first major breakthrough, scoring 16 per cent of the national vote in the 2004 European elections, the BBC literally wrote it out of the script. Its election graphs recorded the Ukip vote as "other", even though it had won more votes than the Liberal Democrats and not that many fewer than Labour. For some years thereafter, BBC political reporters continued to ignore Ukip – until Nigel Farage's rise to prominence in the run-up to the 2015 election forced them to come to terms with a phenomenon they did not like and could not comprehend.

Mosey's mild criticisms made headlines when his book was serialised in the Murdoch-owned Times. In essentials, however, his book is a love letter to the corporation he served so conscientiously all his life. The importance of Mosey's story deepens as he rises up the ranks of the corporation, eventually becoming a privileged spectator of the disintegration and moral collapse of the BBC over the Jimmy Savile affair four years ago.

Trouble had been brewing for some time. The BBC's large salaries and generous pay-offs (for instance to Caroline Thompson, chief operating officer, and to Mark Byford, Deputy Director-General) were already a minor national scandal. They paled into insignificance beside the revelation that Savile had been a sexual predator, targeting vulnerable victims including children and hospital patients.

This was revealed by ITV, the BBC's main rival. Yet more embarrassing, it emerged that the BBC's flagship current affairs programme Newsnight had been working on its own investigation into Savile which had, however, been suppressed. More embarrassingly still, the BBC had run tribute programmes to Savile in the wake of the quashed Newsnight investigation.

Roger Mosey does his best to defend his former colleagues, in particular George Entwistle, the BBC Director-General who lost his job over Savile: "To most of us inside he was so obviously the best candidate that there was rejoicing in the sane choice that Chris Patten had made.

Mosey's account of the Savile affair is a picture of honest bungling by well-intentioned men and women. To this day, that remains the consensus BBC view of events. Perhaps it reflects well on Mosey that for very human reasons he is determined to put the best possible case for his former colleagues.

'That’s Life' fulfilled the organisation's founding mission statement (Rex) (Rex Features)

The Mosey verdict relies heavily on the subsequent report by Nick Pollard, a former BBC journalist who rose to become head of Sky News. Mosey quotes Pollard's soothing conclusion: "The decision to drop the original investigation was flawed and the way it was taken was wrong, but I believe it was done in good faith." It is not surprising there were no resignations in the wake of the publication of Nick Pollard's essentially forgiving and benign review.

In hindsight the Pollard review, which cost £3m of licence fee money, was an unsatisfactory exercise that suggests that the BBC had no real intention of lancing the Jimmy Savile boil. Nick Pollard never established which BBC employee was responsible in December 2011 for shelving the Newsnight programme in which the Savile allegations were first meant to be aired.

As the freelance journalist Miles Goslett has established in a painstaking series of reports, the problems with the Pollard review go further still. One of the key questions that Pollard was meant to answer related to what the then BBC Director-General and editor-in-chief Mark Thompson knew about the Savile allegations from December 2011 until he left the BBC in September 2012. After all, the BBC had not just shelved the Newsnight film, it had also decided to air tribute programmes to Savile instead. The buck stopped with Thompson, as editor-in-chief.

In evidence to the Pollard review, Thompson, now chief executive of The New York Times, maintained he was unaware of allegations that Savile was a paedophile until October 2012, the month after he quit the BBC.

Nick Pollard accepted this, concluding in his report he had "no reason to doubt" Thompson. However, before publishing his findings, Pollard had received a letter from a lawyer acting for Helen Boaden, the BBC's most senior female executive.

The letter, dated 14 December 2012, stated that when Thompson rang Boaden before Christmas 2011 she had informed him that Savile had been accused of sexually abusing girls at a Surrey school and that this suggestion was to have featured on Newsnight. Despite this last-minute revelation, Boaden's evidence was excluded from the Pollard report when it was published five days later.

This is mysterious. Pollard should have dealt with the Boaden evidence in two ways. He could have taken it seriously, in which case Mark Thompson's claim to have been ignorant of the Savile allegations would have come under question. He could have dismissed Helen Boaden as a fantasist or a liar. He chose neither course, meaning that one of the central questions relating to the Savile affair remains hanging in the air to this day. As a result, the BBC has failed to drain the Savile swamp.

Broadcaster Richard Dimbleby talks to Irish linen workers for a BBC television programme in 1950 (Getty) (Getty Images)

The Savile affair, and the lamentable Pollard inquiry, were both symptoms of a much larger problem. There is little doubt that something did go wrong with the BBC in the first decade of the 21st century. By this stage the BBC hierarchy had abandoned at least part of its Reithian heritage, which was rooted in the provinces and therefore capable of stretching out to the country as a whole. By the turn of the century the BBC had been captured by a self-selecting, affluent London-based clique.

To be fair to the BBC, this was part of a much broader and more insidious trend in British national life. Exactly the same process took place inside the Conservative and Labour parties, both of which deliberately abandoned the regional roots that had nurtured their growth.

The effect was uniformity. "Female TV researchers all wear black and say 'fuck' a lot," one news anchor remarked to me several years ago. As for the men, they wore dark suits, but no ties, with three-day beards. (Mark Thompson was a good example of this prescriptive sartorial style.)

In the same way, the mental parameters were conformist – centre-left, pro-European, pro-immigration. All ideas were, paradoxically, required to be "brave" and "edgy". In theory this meant that they were required to challenge convention. In fact this edginess and bravery was a new language for conformity and corporate cowardice.

All this meant that by the early 21st century the BBC management was more regimented than the British Army, far better paid and less original. To put it in another way, the BBC top management had turned its back on its viewers and listeners. Mosey does his best to defend Mark Thompson, Director-General when this homogenous culture was at its height.

His attempt does not work. Until becoming Director-General of the BBC, Thompson had enjoyed a well-planned but meaningless media career. Having joined the BBC as a graduate trainee in 1979, he did approximately 10 different jobs, each lasting around two years: long enough to win rapid promotion, but not enough to make a lasting difference.

Known as a devout Roman Catholic, Thompson was surprisingly tolerant of the greed and corporate excess that was the special characteristic of his term of office. Obscure middle-ranking managers could easily earn a salary of several hundred thousand pounds a year. At Newsnight, Jeremy Paxman pulled in approximately £1m a year, while Jonathan Ross, a presenter who in 2006 asked David Cameron whether he "had a wank" when he thought about Margaret Thatcher (the BBC defended him), collected £6m annually.

Meanwhile run-of-the-mill producers (as an occasional BBC reporter I have worked alongside many of these hard-working, self-effacing and admirable people) were at best moderately paid, and sometimes badly treated. The BBC high command had become a club for the well-connected and fashionable. In this, as in other ways, it mirrored the trajectory of the mainstream political parties.

Chris Patten, who became BBC Trust chairman on the advice of David Cameron in 2011, was no help. He offered zero leadership (and is one of the very few who emerges poorly from Roger Mosey's account). In my own view, he was arrogant, lazy and complacent. He wrongly considered that his job could be combined with a long list of other public and private duties. He never sought to get to grips with the contradictions embedded in the role of chairman of the BBC Trust (set up in haste after the disaster of the Hutton Inquiry), between being the internal guardian of standards and the external voice of the BBC. As a result, he carried out both roles very poorly. By the end of the era of Thompson and Patten, the BBC had no corporate direction and negligible moral authority.

In summary, two factors have coalesced to throw the BBC into crisis. One has been the collective and possibly unconscious decision of recent senior management to subvert their magnificent Reithian inheritance. This betrayal has provided the public ammunition for the second factor: a malevolent assault by a neoliberal political clique overseen by George Osborne, the Chancellor of the Exchequer.

Osborne has already forced a gravely weakened BBC to take on the burden of funding the licence fee for pensioners over 75 years old. As with so many of the BBC's recent problems, Mark Thompson bears much of the blame. As Director-General he conceded the crucial principle that the licence fee could be used for government purposes when he agreed to bear the cost of the BBC World Service, over £200m previously paid from the Foreign Office budget. Funding pensioners will strip approximately one fifth out of the BBC budget.

Meanwhile, John Whittingdale's Green Paper has opened up a fresh wave of attacks on the integrity and funding of the BBC. His statement to the Commons on 15 July even challenged the proposition that the BBC should provide a universal national service. If this vital principle is lost, the BBC will increasingly come to resemble the Public Broadcasting Service in the United States, which unlike the BBC has virtually no influence on the general standard of other broadcasters.

Jimmy Savile at the BBC in 1974 (Evening Standard/Getty) (Evening Standard/Getty Images)

The attack on the BBC is part of a much wider assault by the new Tory Government on what the political philosopher David Marquand has labelled "the public domain": that common space, open to all, which is independent of both the state and the marketplace.

Lord Reith, founder of the BBC, understood this idea well and explained it his 1924 volume, Broadcast Over Britain: "It does not matter how many thousands there may be listening. There is always enough for others. It is the reversal of the natural law, that the more one takes, the less there is for others… Wireless is a good thing, but it may be shared by all alike, for the same outlay and to the same extent… the genius and the fool, the wealthy and the poor listen simultaneously… There is no first and third class."

Charlotte Higgins quotes these words in her beautifully written history of the BBC from its formation in the years after the First World War, through the arrival of television in the 1950s, to today's collapse of institutional self-confidence. She stresses how the BBC came about almost by accident and that Britain, too, might have suffered the fate of the US, where "a cacophony of competing commercial stations grew up, strung between coast and coast".

Her conclusion is worth quoting at length: "Without the BBC we would be poorer in spirit. We would know less about the world: our cultural, musical, political lives would be diminished, our curiosity neither so piqued. It threads itself through all our lives."

As Higgins demonstrates, the BBC is one of the glories of Britain. Along with the monarchy, Parliament, the National Health Service and the rule of law it has become one of the institutions that defines us as a nation. That is something that Burkean Conservatives, though of course not Hayekian neoliberals, can understand, respect and seek to preserve.

Behind the battle for the BBC are two rival visions of Britain and what we stand for. George Osborne and his neoliberal backers are not simply attacking the BBC. They are launching a concerted attack on Britain's democratic public culture. They want to destroy our public space, and put it to commercial use.

Let's leave the final words to Charlotte Higgins: "The BBC needs constantly to reinvent itself; like a church, it must earn, and cannot wholly take for granted, the patience of the faithful. Those who love it expect much from it: we expect more from it. We cheer it on, but we urge it to better. We still believe. We do not wish to see it stumble. We do not wish to hear its melancholy, long withdrawing roar."

The BBC is one of a tiny number of institutions that holds us together as a nation. For all its faults it is one of the jewels of our common national life. It is time for British citizens to take sides, come out and fight for the BBC, and if necessary to man the barricades.

First published on www.opendemocracy.net