'Most people still see me as a bit of a lad': David Baddiel returns to stand-up comedy

He swapped the fame of Fantasy Football, 'Three Lions' and arena comedy gigs for a quiet life writing books and raising kids. So what has tempted David Baddiel back behind the mic?

David Baddiel can remember the precise moment that he gave up stand-up comedy. It was Christmas 2003 and he had been paid £14,000 to entertain an audience of boozed-up bankers. The gig was a bad-tempered affair, and when one heckler bellowed "Why don't you just piss off?", Baddiel couldn't think of a good reason not to. "Performing is very stressful," he says now, on the eve of what's been billed as his stand-up comeback. "When I was doing Baddiel and Skinner Unplanned [his show with Frank Skinner in which they'd take questions from the audience], the stress was just ridiculous."

The pressure exacerbated an underlying depression, for which he has since (successfully) undergone therapy. In the meantime, Baddiel has written four novels, one screenplay (2010's The Infidel) and helped raise (with his partner Morwenna Banks) two children, Dolly and Ezra, both of whom feature in Baddiel's return to the stage, Fame: Not the Musical.

We meet at Quo Vadis restaurant in central London, just a few doors down from Soho Theatre where I had watched him preview what he calls "a proper essay about my weird relationship with fame". He is loath to call it stand-up – in fact, given the painful sciatica running down his leg, he is loath to stand up at all. "I thought maybe I should sit down, in a kind of Dave Allen way, but it doesn't feel quite right."

There will be 11 performances of his "theatrical monologue" at this year's Edinburgh Festival, and a one-off performance at the RSC in Stratford in October. "I'm refusing to tour it in a normal way," he says. "I'm not sure it's suitable to tour. Also, I'm very protective these days of my life… I want to go on holiday with my kids."

Stand-up or not, it's an entertaining and thought-provoking show. Harry Enfield was in the audience on the night I went, while Ricky Gervais was in two nights later. Gervais is the subject of one of the show's best anecdotes, which involves the comedian's pithy reply to Madonna (and Madonna's equally pithy comeback) while backstage at the Live 8 concert in London in 2005, when the singer told Gervais "I'm your biggest fan". It may sound like an orgy of name dropping but, says Baddiel, that's the nature of the beast.

"It's a kind of complicated thing to talk about being famous," he says. "I don't really feel like I as a person was deeply changed by becoming well-known. So I guess the way that the show works [is that] I'm saying to people, 'If you were me this is probably how you'd be thinking in this ludicrous situation'."

Baddiel cites his former comedy partner, Rob Newman, as someone who didn't cope with celebrity. "He was immensely talented, but I would say he found fame pretty toxic," says Baddiel. "The reason we ended up not working together was a lot to do with the fact that I couldn't quite handle the way that he ballooned as a result of being famous. I get very anxious if I feel I'm veering away from me." And fame is restricting, he says. "No one has the time to see these people as three-dimensional beings who can change with time. You get pinned as something that's not you."

So how is Baddiel perceived, if at all – for he freely admits that he is no longer as famous as he once was? "Sixty per cent of fame for me is white van men and builders going 'Whey – Dave!' as I walk past them and they are seeing me absolutely as the bloke who did 'Three Lions' [the England football team anthem he wrote with Frank Skinner], or was on Fantasy Football and a bit of a lad. Other people will still have an idea of me from The Mary Whitehouse Experience… slightly brash… rock'n'roll comedy."

Quoting Philip Roth, he compares this fame to "mistaken identity" and to wearing a mask. "At its worst my mask is between a shouty lad and a racist stereotype of an arrogant, smart-alecky Jew – I think that is the bad mask that feels like it's pressing on my face sometimes."

There is one moment in the show that beautifully illustrates the weirdness of fame – when Baddiel was visiting Auschwitz concentration camp and a bloke sidled up to him in the gas chamber and, instead of sharing some weighty thought about man's inhumanity to man, asked instead: "Dave… when's Fantasy Football coming back?".

Some people, it seems (Andrew Lloyd Webber to the fore) confuse Baddiel with Ben Elton. "They think I'm really right-on," he says. But is he political? "I used to be very left-wing," he says. "I used to be a member of the Young Communists. I was part of lots of direct-action things at university, but as time has gone on I've become disintereste d – certainly in party politics."

One subject, however, he has decided to be more forthright about in his new show – and that is his own Jewishness. His Twitter page (300,000-plus followers) bears the simple self-description 'David Baddiel… Jew', while in his show he talks about the geographically correct (he lives in Hampstead) but weasily euphemistic journalese of 'north London' to denote Jewish. "When a piece describes me as 'having a self-obsessed, north London drone' I know exactly what that means, and it's kind of amazing that that would be OK," he says. "At the same time it's very un-English to go on about anti-Semitism… slightly distasteful."

Is Britain anti-Semitic? "In a very discreet way, and it always has been," he says, going on to point out that Jewish humour has never been celebrated in this country, as it has in America. "My first understanding of comedy was Woody Allen and Lenny Bruce and then Jackie Mason," he says. "I used to love [them] as a young Jewish lad in north London and feeling slightly alien. Americans revel in it and I've tried to do it a bit in this show – I take the gloves off."

His maternal grandparents were some of the last refugees out of Nazi Germany, arriving in Britain in August 1939 – Baddiel's mother, Sarah, just a baby at the time. His father was from working-class Swansea, climbing the social ladder as a research chemist with Unilever before being made redundant in the 1980s. "I had a very, very ordinary lower-to-middle-class upbringing," says Baddiel, who is one of three brothers. "It was just very, very Willesden and Jewish and not at all glamorous."

He did well enough at school, however, to gain a place at Cambridge, achieving a double first in English and embarking on a PhD before being diverted by comedy. The prospect of fame didn't enter his head, he says. "When I started being a comedian all I wanted to do was the next thing… just step by step. I just kept going." But celebrity did arrive fairly promptly, first with Rob Newman, and then as part of The Mary Whitehouse Experience (along with Newman, Steve Punt and Hugh Dennis), before teaming up with Newman again, and performing one of the very first arena comedy shows – selling out a 12,000-capacity Wembley Arena. Commonplace nowadays, but a novelty then. In fact, stand-up comedy has changed a lot since Baddiel last performed."What's slightly galling is that me and Rob Newman started that arena thing," he says. "We did all right out of it, but not like Michael MacIntyre."

Watching his shows was a teenage Russell Brand – now a close friend. "Russell was one of the few comedians who would cite my work as a formative influence," says Baddiel. "He can quote virtually every bit of stand-up I've ever done… stuff I've forgotten." The two comedians met in 2004 in Edinburgh. "I saw his show and thought it was really good. We had a drink afterwards and he confessed to me that he was a big fan and I was something of a hero of his. But even whilst he was saying that, he was looking over my shoulder at women. I'd never quite come across sex-addiction like that before."

In his show, Baddiel uses the story of attending Brand's wedding to Katy Perry in India as another example of the weirdness of fame, and how the press couldn't quite believe that the nuptials weren't being attended by a starry band of Hollywood actors and pop A-listers. "But apart from Russell and Katy, I was the most famous person there," he says. "It was a really beautiful wedding and a real shame, in a way, that that went wrong."

Cue the part of his show that deals with our lack of empathy with famous people – whether it be people treating the failure of Brand and Perry's marriage as a form of entertainment, or trolls jumping into Baddiel's Twitter page to share such witticisms as 'Oi, shit beard. Shut it'. To underscore the reality of his own common humanity, Baddiel – controversially perhaps – ends his show with his 11-year-old daughter singing Elton John's "Your Song". She has an amazing voice. "Obviously the point of the whole show was to get Dolly an agent," he jibes. "Actually she's been asked to audition for two West End shows and because me and Morwenna know about this world and all its pitfalls, we've done our level-best to keep her away from it.

"What's complicated for a parent is to feel you're not inhibiting her – because she loves all that stuff. She hasn't asked to do Britain's Got Talent, but if she was to go on, we wouldn't fit the narrative. It would only inspire hatred, because they base themselves on rags-to-riches in this very old-fashioned way."

Suddenly Baddiel is keen to talk about other projects – and he has a few. "I'm doing a ridiculous amount of stuff, which I think is partly a mortality thing," he says. "I've got to my late forties and thought if I don't action these ideas now I never will." And so he's writing the second draft of a romantic comedy for Paramount; a "Hangover-style thing" for the makers of The Inbetweeners; a musical of his film The Infidel (he's writing the lyrics while Erran Baron Cohen – Sacha Baron Cohen's brother – composes the music); and a new panel show for Radio 4 called Don't Make Me Laugh, in which comedians have to talk about witty subjects without getting a laugh for as long as they can. And then there's a children's book out next year, as well as a sitcom for Channel 4 and a drama for ITV. "It's sort of too much, isn't it? It sounds a bit mental when I say it." A bit of a mid-life crisis perhaps? "It is a mid-life crisis," he says. "But it's also a way of writing myself out of a mid-life crisis."