Why the short story is still alive and well

Publishers and critics denigrate them, but short stories offer readers a perfect love affair, argues Al Kennedy

The short story - it gets a bad press. Well, mainly it gets no press at all, which is worse. But if it ever is discussed, the short story is usually dismissed as the commercially untouchable equivalent of finger-painting.

In an age when responsible adults will happily read Harry Potter books in public, the short story is supposed to be something for children - possibly because they're short, too. Plus, it's condemned to labour under the pedantic curse that demands a beginning, a middle and a twist in the tail. The magazines that used to print stories have largely disappeared and they're left to be harried by endless small-scale competitions that merrily dictate size, content, themes and even title options. (Would you do that to a novel?) Stories are allegedly what novelists knock off when they're feeling lazy, me journalism with a dash of purple prose, something to read in the toilet, a waste of trees.

At which point I get a migraine and then ask you to bear with me for a moment, because together we have to rediscover what the short story is really all about. So go and get a glass, maybe one with a stem, if you're in that kind of household, but definitely a glass, not one of those plastic things your children chuck at one another. I'll wait here.

Sitting comfortably? Then I'll begin. Tap the glass gently with your nail, or a pen. If the glass has a fault or a crack, it won't make much of a sound. If it's flawless, it will sing, resonate beyond itself. That's the best way I can show you the nature of the short story. It may be small, fragile, but to create that kind of seamless clarity - that's a massive challenge to any writer, and a remarkable gift for any reader. Publishers fear the short story, critics overlook or denigrate it, bookshops see short-story collections as a step down from filling their shelves with rancid meat, and authors know they are held to be the professional equivalent of an embarrassingly botched suicide, but make no mistake, the short story is an exercise in perfection.

Poetry has the sexy reputation, impressively obscure technical terms and ancient Greek - it may make no money, but it's supposed to be the premier form, the one you should get the big bucks for festival performances. The novel is for grown-ups, an intellectual and imaginative marathon that addresses the weighty issues, explores historical timescales and the state of mankind and nations.

The short story, we're led to believe, is the wheezy orphan of the literary world, slowly withering between Byron and Tolstoy. Presumably, we're meant to think of Chekhov only as a playwright and forget his short stories, especially the ones that are far too long to fit in a colour supplement or a Radio 4 slot. Forget the short work of Stevenson, Hawthorne and Melville - even if it's some of their finest. Dodge the stories of Tobias Wolff, Richard Ford, Richard Bausch, Alice Munro, Mavis Gallant. O Henry? O Who? Saki? That's a disease, isn't it? Hemingway? Concentrate on the novels. Dorothy Parker? Cheap laughs.

Kick the paralysingly beautiful stories of Alasdair MacLeod into touch - and what can you do with Raymond Carver beyond ignoring his extraordinary life's work and wondering what he'd have made of the novel? And the short stories that go the extra mile and are genre fiction? Well, ditch Dashiell Hammet's remarkable short prose, skip Raymond Chandler, dump Richard McKenna (one of the 20th century's most remarkable sci-fi writers). Stephen King, anyone? Yes, he's commercial - but only because of the novels.

The short story - second best every time, because size does matter and, to casual observers, something that isn't big may seem unimportant. I would point out here that a bullet isn't very big either, but if someone has the skill to sling it into your head at speed, it will be impressive.

Short-story authors are literature's snipers. The form can offer the artistry and intensity of a poem, the themes and weight of a novel and all in a space so small that there is nowhere to hide a single error. No smoke and mirrors, no tricks. Character and narrative have to be developed as they would in a novel, but from a standing start. Sweeping, lyrical, brisk, photographic, still, psychological, miniature, savage - the short story's possibilities are close to limitless. Not exactly a wheezy orphan.

And yet the form faces a perfect storm of opposition. Publishers, who rarely pay well for short-story collections, are then under no pressure to recoup an investment and rarely promote them, so they sell poorly, thus fulfilling the prophecy that collections sell poorly. Agents encourage authors to write anything other than short stories - because they pay badly and sell poorly. The magazines that used to publish literary short stories have largely vanished, and if you're keen on genre short fiction, start taking out subscriptions now: standardised high- street newsagents won't be carrying your magazines, either.

If you write short stories that are longer than 2,500 words - don't. Nowhere will take them. If you write short stories that don't appear to be thinly veiled versions of your own life or that of your relatives - don't. The lifestyle magazines will find them unacceptable, if not alarming. If you write short stories that involve swearing, or strange acts and disturbing details - don't. BBC radio won't touch them, ditto for mags and newspapers. If you save up the varied, strange, long, lively stories that came to you to be expressed and get them published as a collection - well... still don't. You'll be very lucky to get reviewed and the big prizes that get publishers' and publicists' attention are reserved for the novel. And you'll sell poorly.

But you'll write the stories anyway. You know in your heart as a writer and a reader that short stories play a vital role in our literature. Their intensity and brevity may make them uniquely suited for a restless age. As television becomes ever more vapid and ignored, as newsprint "fact" becomes ever more fabricated, and as magazines brandish covers with headlines such as "My Hubby's One-Legged Love Child Tried to Behead Me", there could hardly be a better therapy than an hour with a short story. There, sitting quietly in your hand: imaginative energy, fiction that takes you away far and fast, the facts of the human condition laid out for you as a challenge, a puzzle, a commiseration, a confession, a howl and who knows what next - the word as a power in your mind, alive again, no jingles, no spin.

I was lucky. I started to write with short stories. I found that they were a huge part of my education as a writer, and I was able to get my first collection published. I was lucky to find a publisher who would invest in promoting my short fiction as energetically as my novels, which has meant their sales have been equivalent throughout my career. It's not exactly hard to deal with the problem that commercially invisible books sell poorly.

And now, after the terrifying and counterproductive efforts to "save" the short story, we have the Edge Hill Prize, an annual award of £5,000 to the author of the year's best collection of short fiction. No gimmicks, no tricks, just respect for the form, the authors who are trying to work with it and the readers who still love it. I'm proud to be one of the judges in this, its inaugural year.

I won't tell you my favourite. I'll just give you the short list, so you can buy them, and get reacquainted with the form. Colm Toibin brings a wealth of experience and skill to "Mothers and Sons", Jackie Kay exercises her dense poetic voice in "Wish I Was Here", Neil Gaiman offers us the exuberant and playful "Fragile Things", we travel into the dark, into the future, into other realities and the bewildered mind with "Mortality" from the avid short story author Nicholas Royle, and Tamar Yellin explores cultural dissonance, inheritance and identity with "Kafka in Brontëland" So, in case you were wondering, the short story is still out there, alive and well and waiting for you.