Howard Jacobson: The best fiction doesn't need a label

Much educated discussion at the moment about the composition of this year's Man Booker longlist.

Not just what's on it and what isn't, but whether what's on it adequately rewards genre fiction: crime, horror, fantasy, etc. And whether it is acknowledging one genre – say, the thriller – at the expense of another – say, science fiction.

I'm staying out of it. The current Miss World doesn't comment on her likely successor and I have two months of my reign still to run. So I stay erect and dignified, mind my business, continue to give my views, when asked, on world peace, and make sure my tiara doesn't slip.

But there is an issue here – to keep it general – that relates to what we mean by "genre fiction" and whether the term isn't self-limiting and therefore, when it comes to the dishing out of prizes at least, self-defeating. In the fine arts the word "genre" carries a degree of condescension: a genre painting is one that depicts ordinary people going about their lives in an ordinary manner, the style in which they're painted being essentially realistic and often, according to the Tate's online dictionary, "small in scale". Toss in the imputation of "moralism" and "sentimentality" and it's clear that however brilliant the genre painter happens to be, he is not scaling the heights of art, not making us see as we have never seen before, not upsetting the apple cart and in the process, as modernism decrees, upsetting the bourgeoisie.

The word "genre" when applied to literature has come to mean, to those who use it freely, the very opposite. It is only in one or other of the genres, they argue, that the really surprising and innovative work is being done. "Oh, not another novel about adultery in Tufnell Park," was once a ritual complaint at Man Booker time, which to my mind did scant justice to the many and exciting forms which adultery in Tufnell Park could take. But not as many, was the genre aficionado's assumption, as could be found on the planet Tralfamadore or in the Hanging Gardens of Edinburgh, that's if boring old adultery was of any interest any longer anyway.

Though internecine war will sometimes break out between the genrists – paranormalists deriding the moralistic pretensions of dystopians, for example – the common enemy, which is "literary fiction", usually brings them together at the last.

That the novel is a form always in need of rejuvenation, and should welcome it from whichever direction it comes, I don't see how anyone interested in the future of literature can question. But there is something contradictory in the proposition that "genre fiction" is likely to provide that rejuvenation when you consider that what makes genre fiction genre fiction is its formal predictability, that it answers, genre by genre, to specific expectations, gives its readers exactly what they have come to love and hope for more of – often the same hero, working in the same city, and suffering the same flutterings of existential despair. Forgive me if I don't sound as though I know what I'm talking about: I'm not a reader of the serial novel. But others clearly are, as the windows of our biggest bookshops show, and their loyalty is proof of a simple fact: genre fiction is popular because it's reliably generic. You know what you're getting.

It will be argued that the best exponents of this or that genre escape the confines of their chosen form and turn it into something else. They write more adventurously than do many non-alternative novelists, their fans insist, comparing their prose to that of Melville or Dickens. In this recommendation I detect a certain irony, for its logic is that the more accomplished the genre writer is, the less of a genre writer he becomes. Fine by me, ironical or not. And this should really be the end of the matter. Yes, the best writers must find ways to overleap the expectations of their genre, if they have one, because those expectations are themselves debilitating.

Take the "experimental" novel – an invented concept which in my experience infuriates no one so much as the so-called experimental novelist. Every year the call goes up for more experimental novels to appear on prize lists. But an experiment is no more than work in progress: successful experiments in fiction go unnoticed and you only call a novel experimental when the experiment has failed.

It is the same with the "literary novel". Who wants to be called "literary"? There was a time, before 20th-century publishers dreamt up the "literary novel" as a distinct genre – in the process putting off intelligent readers and consigning 1,000 good writers to the scrapheap of oblivion – when to be called "literary" was the gravest insult. To be literary meant you were precious and overwrought, bloodless and derivative. In the "literary novel" you heard, not the throb of life, but the turning of some other writer's pages. When such a novel carries all before it, the genrists are right to complain. But every other writer, including those clumsily misdescribed as literary, will feel the same.

The truth is, the best novels will always defy category. Is Great Expectations a mystery or The Brothers Karamazov a whodunnit or The Scarlet Letter science fiction? Does Kafka's Metamorphosis belong to the genre of fantasy? In reality men don't turn into giant insects. And it's funny. Does that mean it's a comic novel? God help us all when it comes to that genre. We've said it enough times in this column: any novel worthy of the name is comic somewhere in its being. And to maintain that the comic frightens more judges than it should is not to argue for a specific comic genre. For a good writer of comedy, too, will jump the restrictions of the form and foil every attempt to box him in.

At a time when reading is in trouble, those readers left should define themselves less rigidly. So long as the words live in their own skin, are "neither diffident nor ostentatious", to quote TS Eliot, are "exact without vulgarity ... precise but not pedantic", and on occasions rip your heart out, then the name we give the genre in whose service they "dance together" is immaterial.