Class struggle: Jake Arnott analyses the class repression of one of the heroes of Empire

(STEVE SCHOFIELD)

At a recent party, Jake Arnott noticed a group of Old Etonians. They exuded a sense of entitlement only an elite education can provide. A short time later, he observed another group, dressed to impress. Their eagerness betrayed them. "I looked at them and thought: grammar school boys, made for middle management," the author of The Long Firm says. He adds, with the hint of a sneer, "That's why I dropped out of grammar school. It just seemed geared to creating middle managers."

Arnott tells the story during an interview about his new novel, The Devil's Paintbrush, which is set in Paris in 1903 and based on a meeting between the infamous occultist Aleister Crowley and Sir Hector MacDonald – or "Fighting Mac" of Omdurman, the alleged model for the soldier on Camp Coffee bottles – on the eve of the soldier's suicide. MacDonald fled to Paris from Ceylon via London, under threat of a court martial for allegations of homosexuality. Faced with public disgrace, the hero of the Empire shot himself.

Though masculinity, sexuality and repression are themes in Arnott's previous four novels, The Devil's Paintbrush is equally about class and the hidden ciphers that exclude the uninitiated. MacDonald made a rare transition: a Scottish crofter's son, his bravery in battle led to promotion. In an Edwardian society rigidly divided along class lines, he was an aberration even before his sexuality was unmasked: one mistake and he'd slide down the snake faster than he climbed the ladder.

Homosexuality at the heart of Empire has been nagging at Arnott as a theme for some time. "In The Long Firm, Harry Starks comes out from watching Lawrence of Arabia in floods of tears because he identifies with the great queer hero," he says. "For a long time I have thought about this subject. Hector MacDonald was a great gift for a novelist because there is no definitive biography – I don't think there can be because he left so little behind."

Placing his next novel in the distant past had another attraction for Arnott, who was sued when it emerged that the band leader Tony Rocco, an unsavoury minor character in his novel Johnny Come Home, shared a stage name with a real former band leader. Though there was no intent to libel, intent is no defence in British law and the entire print run was pulped. The experience still smarts. "I realised that nobody can libel the dead," he says with a bitter laugh.

If The Devil's Paintbrush were recent history, Arnott would be in more trouble. Few Empire heroes emerge unscathed. Kitchener is portrayed as a Machiavellian repressed homosexual, whose lack of judgement in battle almost ends in disaster, but who is protected by his Band of Boys – young officers who are instantly banished from the inner circle if they marry. Baden-Powell is a closet queer who drags up for "skirt dances". It is a world that existed unquestioned under Queen Victoria, the maternal centre of this homosocial world. "The temptation is to call her an old fag hag," Arnott jokes.

For boys raised in the brutal, homoerotic atmosphere of an English public school, the rules of this world are easy to decipher. Not so for a rough Scot raised on a smallholding. The boundaries of propriety are opaque to him: so fearful is he of betraying his sexuality, he buries his emotions beneath a granite expression and fearlessness in battle. He is a tragic hero, terrified of himself.

"The reason Hector is such a tragedy is not that he got in trouble, but that he didn't have any protection," says Arnott. He leans towards me and says in his hint-of-geezer accent: "He doesn't have very much money, which means he doesn't have any protective world that he can retreat to, so he is in really big trouble when the accusations come out."

The inclusion of Aleister Crowley in Hector's downfall is a stroke of genius. Crowley's wilful flouting of propriety and easy acceptance of his own confused sexuality ("Crowley for me is genuinely bisexual," says Arnott) throws MacDonald's struggles into sharp relief. Not only is Crowley shielded by wealth and position; he has a letter written by the Duke of Clarence, Edward VII's dead son, to his gay lover. It is essential insurance for a man of excessive tastes. This revelation is just one of the many ways that The Beast, as Crowley refers to himself in the novel, emerges as a playful, narcissistic reprobate. As MacDonald represents a dying age of Empire, Crowley, with his self-regard, vanity and lack of serious substance, foreshadows our own age of celebrity excess and self-obsession. He would fit nicely into the Celebrity Big Brother house.

"Crowley connects with a very modern world but he's also come of age in the Victorian period so he has these very conflicted feelings," the one-time mortuary attendant Arnott observes. Crowley is not the first conflicted Arnott character: Harry Starks, the monstrous psychotic gangster in The Long Firm trilogy, "talks about being homosexual, but he doesn't understand gay culture because he doesn't belong to that. He doesn't feel he connects." In that, he resembles MacDonald.

It is hard to believe it is 10 years since The Long Firm. Riding on a crest of gangster chic, it subverted rather than celebrated the Lock, Stock and Two Smoking Barrels image of masculinity. Though Arnott is two years shy of 50, he looks 10 years younger. Much has changed since I last saw him, and he admits that defining male sexuality interests him because of his own recent narrative: last time I saw him he had just started seeing the writer Stephanie Theobald, a relationship that caused some raised eyebrows: though bisexual, he was regarded as the darling of the gay lit scene, while she was a bi-baiting lesbian. Though both have written about their relationship, when asked about it he answers hastily: "I don't want to become a bisexual spokesperson. Neither me or Stephanie want that."

Theobald advised him about a Black Mass in the penultimate chapter: "The way she describes it is much funnier." The Mass is not her only influence on The Devil's Paintbrush. "To be honest, the thing with Stephanie has raised my game as a writer," he says. Arnott is not comfortable about talking about his own feelings, though he betrays their depth when he rhapsodises that the author of Biche and A Partial Indulgence is "brilliant". He adds: "Even before I knew her I thought she was an extraordinary talent."

His championing of Theobald reflects not only love, but a general frustration with British literary culture. "I remember the same time as the last Granta list, Nick Blincoe and Matt Thorne launched the New Puritans." The group and their manifesto challenged the literary traditions of the old establishment exemplified by Martin Amis, and were rewarded with general criticism. "The idea of having some kind of movement or philosophical statement, they don't want that. They'd rather have a list of the 10 best puddings this summer," spits out number 87 on the 2005 Independent on Sunday Pink List of Britain's 100 most influential gay people.

We are back on the class system, and I realise it isn't just about observing Old Etonians: Arnott isn't comfortable with being in the middle. It is not the ambivalence he hates, it is the constraints. "When you are on the list you worry about who you are above and who you are below. That is sad in our literary culture. If you want to be treated well as a literary writer, go abroad." Writers may have more in common with hounded gay war heroes than at first appears.

The extract

The Devil's Paintbrush, By Jake Arnott (Sceptre £15)

'... A great, simple, lion-hearted man with the spirit of a child, thought the Beast as he caught sight of Major-General Sir Hector MacDonald taking lunch alone in the dining room ... The General sat to attention, ramrod-straight, looking ... so strong and resolute. Only one of his hands betrayed him. It clawed at the white tablecloth, as if grasping at a ledge'