John Birt: The return of Mr Blue Sky

He has interviewed Mick Jagger, brought his own brand of management-speak to the BBC, and thought the unthinkable for Tony Blair. Now John Birt is helping EMI get the most out of Robbie Williams and Lily Allen. Ciar Byrne reports

In his autobiography, The Harder Path, John Birt recalled how, in 1962 at the tender age of 17, he earned overtime by working as a bouncer at Southport's Cambridge Hall where a little known band called the Beatles was playing. He was "electrified" by the sound: "I felt the sharp shock of the new." But, to his shame, the young Birt abused his newfound position of power. The Fab Four were already attracting crowds of young female fans and one of them was known to Birt and his fellow bouncer as "the most beautiful girl in Formby". When she pleaded to be let in to see Paul McCartney, Birt's friend agreed in return for "a snog and a feel". Birt admitted: "I participated in the encounter despite her inert response."

There is a kind of poetry in the fact that the once oversexed young Beatles fan has now been charged with reviewing how EMI, the Beatles' record label, treats its roster of artists – including Lily Allen, Pete Doherty, Robbie Williams and Kylie Minogue. In the age of internet piracy and talent using the Web to bypass traditional record companies, Lord Birt's task is to compare the way in which pop stars are marketed, promoted and looked after on all of EMI's different labels – and to report on it all in the new year.

His appointment follows the buy-out of EMI by Guy Hands' private equity group, Terra Firma, which Birt joined 18 months ago as a general adviser. It is just one of many hats he has worn since he quit as the BBC director general in 2000, including working as an adviser to companies ranging from the consulting firm Capgemini to the Waste Recycling Group.

But it is Birt's role as a visionary "blue skies" thinker and special adviser to Tony Blair from 2001 to 2005 that has attracted the most interest and, all too often, derision. Blair, by all accounts, found Birt's advice invaluable. One insider commented that the reason the former prime minister got on so well with the ex-director general was that Birt would present him with papers on various subjects that were "a statement of the bleeding obvious", providing clarity the Civil Service had failed to deliver.

Among certain members of the cabinet, however, Birt was held in slightly lower regard. In 2005, when the ceiling of his Downing Street office collapsed, leaving the taxpayer with a bill for nearly £3,000, the Cabinet Office minister John Hutton joked to fellow MPs in the House of Commons: "I'm afraid Lord Birt's ceiling fell in at his office in No 10 and I suspect that will probably help him in his blue skies thinking."

As a special adviser rather than a civil servant, Birt was under no obligation to appear before House of Commons select committees – Gwyneth Dunwoody, the chair of the transport select committee, was one of those who tried in vain to call the peer to give evidence, despite the fact that he had been asked to come up with a long-term transport strategy.

Lance Price, a former BBC correspondent who worked in political communications for Downing Street and the Labour Party, believes that despite the distrust of some in government, including the present Prime Minister, there was no doubt that Lord Birt brought sharp, strategic thinking to Downing Street.

"He was certainly held in very high regard by the [former] prime minister, although he was treated with suspicion by some other members of the cabinet," said Price. "Gordon Brown had suspicions of what he was about, but he was there to do some very radical thinking on behalf of Tony Blair. He [Blair] was concerned that the second half of his premiership should be more radical and Birt was a guy who always knew how to get things done."

It was certainly unusual for a former BBC director general – and one dubbed a "croak-voiced Dalek" by the playwright Dennis Potter at that – to enter politics after leaving the broadcaster.

"It was surprising that he should go from the BBC into government, when most people go into the commercial broadcasting arena, although it was obvious from his previous career that he was interested in politics," said Price. "It says something about what drives him, which is to get things done and make his mark. He's very, very smart, but not terribly good with people. He wasn't very good at inspiring people at the BBC. People who worked closely with him respected him enormously, but the majority of staff were suspicious of him."

Born in Liverpool in December 1944 into a family which straddled the city's sectarian divide, Birt was raised a Roman Catholic, following the religion of his father's family. Perhaps explaining the supreme self-confidence for which he is renowned, in his autobiography he writes: "When I was born – the first grandchild on either side of the family – I was the centre of attention for the numerous women I was surrounded by." His father worked for the Pru, but his wages were barely enough to cover the mortgage of the family's semi in Formby. The young Birt was educated by the Christian Brothers at St Mary's Senior School, before winning a place at St Catherine's College, Oxford, to study engineering.

Two weeks after his arrival at Oxford, Birt met a young American artist called Jane. By Christmas of his first year, he had proposed marriage. Jane became his first wife and the couple went on to have a son and daughter. But in 2005, their 40-year marriage was brought to an abrupt end when he admitted an affair with Eithne Wallis, the former head of the National Probation Service and a divorced mother. Birt went on to marry his mistress in a low-key ceremony at Islington register office in December 2006, which was not attended by either set of children. Peter Mandelson, a long-term friend of Birt from his days at London Weekend Television, and Trevor Phillips, the head of the Commission for Equalities and Human Rights, both attended the reception at the fashionable St John restaurant near London's Smithfield Market, but many former colleagues refused to attend when their wives were not invited.

Upon leaving Oxford, Birt applied to work at the BBC and made it through "to the last 10 or so", but was turned down. Instead, he won a job on Granada's flagship documentary strand World In Action in Manchester. Here he secured his first scoop, persuading Mick Jagger, who had just spent three nights in Brixton prison for possession of drugs, to be filmed in conversation with the editor of The Times, William Rees-Mogg, for a programme hailed as a "dialogue between generations".

Birt was poached from Granada by David Frost to work at London Weekend Television, where, together with Peter Jay, he published three articles in The Times, arguing that television news and current affairs should have a "mission to explain". In his review of The Harder Path in The Observer, the former chief creative officer of Big Brother creator Endemol, Peter Bazalgette, described the pieces as "possibly the most misguided articles ever written about the practice of television", adding: " The basic error they made was that the amount television can 'explain' is limited by the number of spoken words you can fit into a one-hour slot."

In the late 1970s, Birt took leave of absence from LWT to work with David Frost on The Nixon Interviews. In the recent film version of Peter Morgan's play Frost/Nixon, based on those interviews, Birt is played by the former Spooks star Matthew Mcfadyen who was obliged to strip for the role – Birt was allegedly so thrilled by the success of the interview dealing with Watergate that he took off all his clothes and dived into the Pacific Ocean.

In 1986, Birt was poached from LWT to become deputy director general of the BBC, where he headed news and current affairs. No sooner had he arrived than he entered into a row with Michael Grade, an old friend from his LWT days, who was campaigning for Birt's deputy Ron Neil to be appointed controller of BBC1. Birt won the argument, but "a chasm had opened up between two old friends".

Within six years, Birt had succeeded Michael Checkland as director general and set about implementing unpopular policies including "producer choice ", which required BBC producers to use outside suppliers when it was cheaper to do so – an initiative which led to crazy anomalies such as it being more expensive to borrow a CD from the BBC library than to buy one from a high-street shop. He also split broadcasting and production, meaning that programme-makers within the BBC had to sell their programmes to commissioners.

Controversy arose when it emerged that Birt was employed as a freelance consultant, which meant he paid less tax, and he came under political pressure to become a BBC employee. His memos were infamous, using management-style language that was soon satirised by Private Eye as Birtspeak.

In his favour, during his tenure, Birt oversaw negotiations with the government which secured a long-term increase in the licence fee. He is also widely acknowledged to have pioneered the BBC's online and digital strategies, developing BBC News 24 and BBC Parliament, as well as the early prototypes of BBC3 and BBC4 – BBC Choice and BBC Knowledge.

One former associate said: "John is clearly a hugely talented man, especially when it comes to analytical and strategic work on policy and business problems. His thinking is rigorous, and he is very determined to pursue solutions, once he has figured out what to do. In fact, he is relentless, once he has made up his mind. Sometimes this means that there are human casualties, but he regards this as an inevitable consequence of necessary change."

In August 2005, Lord Birt (he had been knighted in 1998) delivered the annual James MacTaggart Memorial Lecture at the Edinburgh International Television Festival. In an attack which did not go down well with the audience of TV executives, Birt bemoaned the "tabloidisation" of British intellectual life and said the media had become too hung up on " the desire to humiliate".

In Armando Iannucci's political satire The Thick of It, a former business guru turned "blue skies" thinker, Julius Nicholson, played by Alex MacQueen, descends on the fictional Department for Social Affairs as part of an investigation into the structure of all government departments. It bears an uncanny resemblance to a planned shake-up of Whitehall which Birt worked on but which never went ahead. One acquaintance commented: "He was willing to attack embedded problems which governments usually ignore in areas like crime and transport, but he was not able to get many of his policy ideas actually implemented."

Perhaps, at EMI, Lord Birt will have better luck.

Granada to EMI: John Birt's CV

* 1944: Born at Walton Hospital, Liverpool, 10 December

* 1963-1966: Takes degree in engineering at St Catherine's College, Oxford

* 1965: Marries Jane Lake in Washington DC

* 1966: Production trainee at Granada Television, Manchester

* 1967: Makes World in Action with Mick Jagger and William Rees-Mogg

* 1971: Leaves Granada for London Weekend Television

* 1974: Becomes head of current affairs at LWT

* 1977: Produces the Frost-Nixon interviews on leave from LWT

* 1981: Appointed head of programmes at LWT

* 1987: Joins BBC as deputy director general

* 1992: Becomes 12th director general of the BBC

* 2000: Leaves BBC in January

* 2001-2005: Works as "blue skies" adviser to Tony Blair

* 2006: Joins the private equity firm Terra Firma as an adviser

* 2007: Asked to review how EMI treats its roster of artists