Laugh? I nearly cried: How a tour of the Fringe reveals a tortured nation

Johann Hari toured the Edinburgh Festival Fringe in search of Britain's funny bone. He left with the feeling that, behind the smiles, as a nation we are dissatisfied with our lives and uncomfortable with each other

In August, every last granite crevice of Edinburgh – this 18th-century Scottish city on a hill – becomes infested with comedy. Every building, hut, toilet and phone box is requisitioned for laughter, and if there's any room left, they fill it with new metal huts to make you chortle and spew some more. As the stand-up Nick Doody puts it: "If you're in a car accident in Edinburgh and the ambulance is a bit late, they'll set up a show in your guts. When you come round, the doctor will say: the bad news is you're going to lose your legs. The good news is you're now venue number 1,674, and the show's getting rave reviews."

If you know what makes a country laugh, you know what it fears and dreams and despises. So I decided to go in search of the funny bone lying deep in the flesh of Gordon Brown's Britain. Let's start our trawl as most of its practitioners do – with a long lash of depression.

The skinny Simon Amstell is best known as the acidic host of Never Mind the Buzzcocks, but here he is offering a late-night dose of despair. "Is there anything worse than being alive?" he asks. "People think death is worse because they don't know what it's like. I'll tell you what it's like – less bother." Looking bleakly out into the auditorium, he declares: "When somebody says to me, 'I'm having a baby,' I say, 'Really? You know who else had a baby? Everyone. You're just furthering the cycle of misery.'"

The source of this misery is a man. Amstell began to row with his boyfriend last year, and started trawling into the fetid bogs of Eastern spirituality for a salve. His logic was: "There's a personality clash here. So I had to take my personality out of the equation." He began to trawl into the Buddha's thought and concluded: "The idea we are individually significant is illusory. You know this is true when a friend shows you their photographs. You think, I don't care, I can't be bothered – oh, is that me?" He tells a man in the audience: "You think you have an identity. You have glasses, so you're the guy with glasses. But when a crisis comes along, you think, I'm more than that. Actually, you're less. Much less."

Amstell could easily get away with serving up a few lukewarm gags for a drunk, late-night audience who just want to see that bloke off the telly – but instead he offers a probing, cerebral show that has stretched far beyond that. "I thought about buying a cat," he explains, "but then I realised that a cat isn't going to make me any less lonely. It'll just provide a mascot for my loneliness."

He speaks not just for everyone who's ever been dumped, but for a disconnected, disengaged chunk of Britain, sitting in their own trendily furnished boxes waiting for the world – or some meaningful cause – to come bursting through the door. It doesn't. He ends declaring: "If you ever feel you're too depressed to go on – don't. Obviously, don't kill yourself after this show, though, because that's not a good review for me. It's one thing to say, 'I died laughing,' but not to say, 'I laughed then I died.'"

Wil Hodgson seems, at first, to be from another Britain far away. He is a big-bellied 29-year-old with a pit-bull face and a high, high mohican, dyed pink. "At many points in my life I've been a miserable failure," he says in his warp-speed Essex burr. "I was a miserable failure at being a skinhead at 17, a miserable failure at wrestling school in my mid-twenties, and since the age of five I have been a miserable failure at collecting My Little Pony dolls."

I sit up. There's something original and odd here. Hodgson whizzes through his obsession with My Little Pony, interspersed with anecdotes about what particular animal tattoos mean in prison, before adding with a frown: "Skinheads are a much more liberal group than you have been led to believe, but they draw the line at My Little Pony lesbian weddings." So many "character" comedies feel contrived; this has the acrid tang of authenticity.

Hodgson guides us through the pubs and kebab shops of Chippenham: one of them "is the kind of place you should go if you are ever pissed off with having two eyes". Another is somewhere "they might as well put the Rohypnol in the drinks when they serve them."

Hodgson introduces a theme that keeps recurring in Edinburgh's comedyathon: a sense that we are all living in a processed, plastic environment, trapped in our own inauthenticity. He caught a late-night show on ITV2 called The 150 Worst Moments of the 1980s, and explains: "If you're a bankrupt coal-miner watching your son inject himself with heroin, don't worry – at least you're not Timmy Mallett, who was officially the worst thing about the entire decade. That makes him the worst person of the 1980s, too, which must be a great relief to Peter Sutcliffe."

Hodgson's is a voice pining for reality – including real women. "I'd climb over Paris Hilton to get to Fern Britton, should such an unlikely scenario ever occur," he says. "Having sex with Paris Hilton would be like sleeping with a giant bar of Toblerone you'd dipped in Ronseal." Hodgson drags on to the stage a country dismissed – not least by repellent snobbish stand-ups like Jimmy Carr and Tom Stade – as "chav Britain". "If you see any posh kids wandering around dressed as 'chavs' to promote their show, kill them," he says. I cheered.

But Britain is racked not just by resurgent snobbery, but by the resurgence of foaming religion, too. As always, its greatest comedic foe here is Stewart Lee, who boasts in the title of his show that he is now officially "the 41st best stand-up ever", according to an inane Channel 4 list show.

Wandering on to the stage with his hands in his dark-suited pockets, he explains he is currently itching to begin shooting a documentary called March of the Mallards. The Christian right seized hungrily on the movie March of the Penguins, because as it followed our waddling Antarctic friends, it found they believe in monogamy and family values. See! It's natural!

Well, Lee tells us: "Mallards are the only creatures that reproduce exclusively by gang rape. They have been captured on film indulging in some recreational homosexual necrophilia. So I want a film where Morgan Freeman says in the narration, 'There goes that mallard taking the other dead mallard in the ass – in a dance as old as time.' There's nature." Lee's V C atheism is part of an old English empiricist tradition. He doesn't offer the angry anti-clericalism of the French philosophers, but a wry and mocking raised eyebrow at the absurd claims of the faithful.

After Lee's laconic style, it takes a moment to acclimatise to the hoarse Aussie bark of Brendon Burns – although his rage against religion is just as great and just as righteous. "Somebody trying to blow themselves up to convert us all to Islam, and not killing anyone else, is hilarious," he yells. "If you see any comedian who isn't joking about it, they're not doing their job. I want tickets to the next one. I'll be there roasting marshmallows. You want to convert Britain to Islam? They haven't even converted to Christianity yet. They haven't even converted to the fucking metric system. And you're offering no drink, no sex, no music, and at the end you burn yourself alive? Sign me up!"

One of the most dessicated clichés in the critic's vocabulary is to call a stand-up show "a blast" – but Burns's really is a blast. He screams and shouts and punches his way across the stage in a recovering-alcoholic rage. His show poster makes it look at first like a tedious anti-PC rant: it shows Burns in blackface, then in a wheelchair impersonating a disabled person, then mocked up as Christ on the cross, and asks: "So I Suppose This Is Offensive Now?"

But actually Burns is offering something much more interesting than that. With a blunt honesty, he is trying to tease out what we can legitimately joke about together. He loathes racism – "I don't celebrate Australia Day, in much the same way I don't celebrate Hitler's birthday. Two hundred years of genocide? It's not for me" – but he wants to be able to mock cultural differences he finds depraved, like forcing women into burqas. He says: "If you take the piss out of culture, people get uncomfortable – but isn't culture something I do because my granddad did? And don't we take the piss out of grandad a lot?"

Burns illustrates this conflict within comedians with a real, live twist, as shocking as the shockers in The Crying Game or The Sixth Sense. It would be a crime to give it away, but the audience leave thinking about racism – and their reactions to it – in a burning new light.

All over the Fringe, comedians are being forced to defend their right to offend. The excellent, underrated young stand-up Nick Doody has been chastised for saying that Madeleine McCann's parents were right to visit the Pope "because if anyone can find a paedophile..." He explains in his set: "Just because you make a joke about something doesn't mean you find the subject itself funny. To laugh at a 'knock knock' joke, you don't have to find doors hilarious. But people are looking for offence. Why did the chicken cross the road? I can't believe you'd say that! My mother was pecked to death by a chicken!"

But Jerry Sadowitz – the Glaswegian-Tourettesian veteran stand-up – takes this offensiveness shooting off into the stratosphere. He wishes death on Heather Mills-McCartney ("She's two-thirds of a fuckin' person") and says things about the McCanns and assorted other public figures that cannot be published in a family newspaper. In fact, they cannot be published in any newspaper, except perhaps the Cromwell Street Gazette. "My glass is fuckin' empty and cracked and it's no' even there because some bastard stole it," he rasps in between offering card-tricks to the audience, as if he was the bastard child of Tommy Cooper and Fred West. There is one moment of accuracy in his set; Sadowitz yells: "The only way I'm getting back on telly is if I'm kidnapped by fucking Iraqis."

If Sadowitz is edgy, though, Rick Shapiro at the Green Room is over the edge, off the cliff and bleeding to death in the valley below. Shapiro is a former prostitute and heroin addict who shambles on stage and snaps: "You guys are looking at me like a bunch of radiation victims." He mumbles and flails for a few minutes, before pleading: "Stay with me because I am so desperate to snort cocaine and shoot heroin and crawl off the stage and die right now. Imagine if suicide was, for you, just eating lots of chocolate ice-cream. That's what it's like in my head." It sounds smooth on the page. Imagine it said with a stammer and a crazed flickering stare.

This isn't stand-up – it's crawl-across, with Shapiro dragging himself across the stage and twitching. He periodically yells: "Help me!" His director has to yell: "Come on Rick!" from the back of the room from time to time, causing him to snap back: "You don't know what it's like to be off your medication when the voices in your head aren't friendly." It's hard to separate the comedy from the disintegration. At one point he has a hacking cough and says: "I got Aids, but I'll beat it." It takes the audience a few beats to realise this was actually a joke. There are a few good lines – "If a woman's a good lay, don't call her a whore. Call her again" – but all I really remember is my growing fear that this might be the first stand-up gig since Tommy Cooper where the act literally dies on stage.

If Burns and Shapiro water-cannon you with angry testosterone, the most intriguing trend on the Fringe is towards a very different kind of comedy – one that offers you a warm bath of oestrogen. Josie Long is the queen of this new school, which I think of as Organic Comedy. It usually involves lots of home-made props, hand-written diagrams and a whimsical, upbeat glow. It is the polar opposite of the glib, flavourless speed-comedy of a Jimmy Carr, offering instead something slower, more intimate and stripped of the comedy-pesticides of cynicism and sneering.

Josie (and with her easy intimacy, you quickly think of her as "Josie", not "Long") is a scatty, dreamy twentysomething who begins her show Trying Is Good absent-mindedly saying to the techie: "Oh God, sorry, I was miles away." The central image of her show comes from something she spotted in the local swimming-pool at the start of the festival. They have built a floating assault course for the children to play on, and there is a man – "an actual adult man", she says – whose job is to hose the children off if they get too far. "What does he say at dinner parties when people ask his job? 'Actually, I power-hose children off a floating assault course.'"

Josie spotted one fat kid clinging desperately – and successfully – to the course, with a look on his face that said: "At last – victory! Of sorts." And she figured: "I would always rather be a fat kid on a slide than the bastard with the hose." Her show is a celebration of people who invest effort, "no matter how misplaced". It is a catalogue of people who delight her, and whatever the opposite of misanthropy is – proanthropy? – she is full of it.

This kind of comedy is hard to capture in written snippets: you can't bottle charm. Josie takes her audience along great rainbows of whimsy. A taster: she says she loves roller-coasters because "it's like somebody you don't know coming up to you and saying, 'If you give me £2 I will shake you.' And you say, 'Yes.' Then they say, 'If you give me £2.50, I will give you the chance to win a toy to the value of 10 pence, along with the illusion that you are good at sports.'"

Josie's organic comedy style is spreading into something like a movement here, and it is an artistic cousin of the mumblecore movement in American independent cinema. Both are stripped-down reactions to a crass commercial culture of pulp products and pulped minds. Both are trying to retrieve the personal and the intimate buried beneath the plastic and tinsel and wrapping paper of our corporate culture, and they are all frightened of being squeezed into a deadening nine-to-five culture that smothers personal creativity.

Isy Suttie's show Love Lost in the British Retail Industry is one of the most beautiful examples, consisting of nothing but a sweet, slight Northern girl, a guitar and a voice you want to swim in. With this, Suttie creates a love story in a supermarket in Matlock that washes away all the bad taste left in your mouth from enduring foul rants about "chavs".

Hers is a dreamy world, depicted with love, that reflects a forgotten chunk of Gordon Brown's Britain – the one working on tills but dreaming of being in a band, who sings with a wry smile: "Somewhere over the rainbow/ There's global warming./ And a black charcoal cloud slowly forming.../ If there's a pot at the end of the rainbow/ It's full of piss." Yes, this is a school of comedy that makes you gurgle rather than belly-laugh – but it also offers a world view that urges you, for all its sadness, to find happiness in the scattered moments of authenticity and tenderness between human beings. It's not just a style of comedy; it's a philosophy of life.

But what is missing from this picture of British comedy? There are only a few straggling political stand-ups left here: the always dependable Andy Zaltzman made the best point of the festival when he said: "Next time scientists have a report on global warming they should issue it as a fuzzy video from a mountain lair in Afghanistan. Then we'll be terrified."

Yet mostly, political stand-up has stood down. I think I know the root of the problem; George Bush has killed political comedy (and hundreds of thousands of people). The laughs in his strangulated English – and in a Vice-President who shoots an old man in the face, mistaking him for a quail – are so obvious they don't need to be amplified by a guy with a mic. The same goes for the Carry On Up the Jihad attacks on London and Glasgow in late June.

Instead, we are left with a softer, sadder picture of Britain from the clouds of laughter that hang over Edinburgh. It is of a country that is, in the main, wealthy – but oddly unsatisfied with our lives. We are uncomfortable talking to each other, not even sure of what language to use. We know there are terrible threats out there – global warming, jihadism – but don't feel we can do anything about them. There is no grand national story here – just the scattered stories of depressive break-ups with our boyfriends, kebab shops in Chippenham, and an attempt to retrieve some reality from a mass-produced husk. I have found Britain's funny bone – and it's black and fractured.

The Edinburgh Festival Fringe continues to Monday (www.edfringe.com). The if.eddie comedy award short-list is reviewed in Extra