In the shadow of a lost city: The secrets and struggles behind Carlos Ruiz Zafón's success

Mystery and melodrama in bookish Barcelona have made Carlos Ruiz Zafón a global bestseller and now the author's second Gothic tale appears in Britain

Millions of grateful readers around the globe have the Academy of Motion Picture Arts to thank for the bewitching Barcelona mysteries of Carlos Ruiz Zafón. As well as running the Oscars, the Academy awards its annual Nicholl Fellowships to promising screenwriters. In the mid-1990s, a young Catalan author - who had forsaken the advertising business to write but had just one young-adult novel under his belt - re-located to the West Coast. There, in the Los Angeles he had "idealised" as a jazz- and movie-loving kid, he planned to write for the screen and so buy time for his own fiction. "My mercenary work would be done – anonymously. Of course, it was a very naïve plan, but it sounded good at the time. So I decided to go to LA and try my luck."

Inevitably, the unknown hopeful from Barcelona struggled until he entered a script for the competition and found himself among the finalists. Agents and producers purred that "they loved the script but it was not what they were looking for". So what was this precious manuscript like, which opened many doors but never made it to the screen? "It was a noir story with a supernatural twist" and "Gothic overtones – Sunset Boulevard meets Rebecca". In those days, before The Sixth Sense and other money-spinning spooky spellbinders, "that was considered a problem... They said, you've changed genres."

So Zafón, too far ahead of his time in the pursuit of sinister suspense, continued to toil away in the screenwriting salt mines. He divided his time between Catalonia and California and his career between butchered scripts "read by 10 people in Burbank" and teenage novels back in Spain. "As a working writer, I was conservative and cautious, and I didn't want to jump from a moving train". Then came the bolt of Gothic lightning that would give life to his fifth novel, The Shadow of the Wind.

He already had form in this field. As a wannabe novelist in Barcelona (his parents lived one block away from the riotous stone jungle of Gaudi's unfinished Sagrada Familia church), the teenage Zafó*had penned "a 700-page monstrous Victorian-Gothic thing". Now that fascination with a sprawling fictitious universe of labyrinthine tales and "Russian doll" plots came into focus.

He imagined a single vast city of stories. "When I first started working on The Shadow of the Wind, I was not sure if this was one gigantic book. But I felt it would be absurd to do some kind of monstrous, 3,000-page novel." Instead, Zafó*decided to plan "four books that would be interconnected but independent". The Shadow of the Wind "was the first time that I was writing exactly what I wanted to write". And, it turned out, what the world wanted to read. Since 2001, this first visit to the "Cemetery of Forgotten Books" in Barcelona and the uncanny maze of stories that surround it has sold more than 12 million copies, 1.3 million in the UK alone.

Already a multi-million-selling phenomenon on the Continent, Zafón's second "door" into his mansion of yarns opens in Britain this week. We discuss The Angel's Game (again translated by Lucia Graves, daughter of Robert) in surroundings that suit this lifelong lover of Dickens. Vintage volumes line the shelves of a cosy London hotel library as - less aptly for this dweller in fictional darkness – fierce sun beats down outside. Zafó*himself, stocky and affable, would look and sound like any other laid-back southern Californian of vaguely Hispanic heritage were it not for the extraordinary literary story he has to tell, and the quiet passion of its telling.

The Angel's Game (Weidenfeld & Nicolson, £18.99) revisits some of Zafón's much-loved urban landscapes in old Barcelona. Yet this novel stands alone, a complement or even an antagonist to its forerunner. If the previous book celebrated the ecstasies of reading then this one – no less in love with literature, and no less crammed with archetypal plots – explores the agonies of writing.

Its reader must share the "game" of composition. "I felt that The Shadow of the Wind was a mechanism that did everything for you," he says. "This time I wanted to try something different, at the risk of providing something that would make readers uncomfortable."

In turbulent 1920s Barcelona, the solitary hack writer David Martí*rises (or so he imagines) from a life-sentence of drudgery, spewing out lurid potboilers for book-trade swindlers. Zafó*both satirises and indulges David's "disease of Grand Guignol". The scribbler's own story – a murdered father, an ice-hearted mother, a plutocratic protector, a book-loving Dickensian assistant and a true love imprisoned in a tower – outranks in scandal and sensation anything in his own "Mysteries of Barcelona" or "City of the Damned".

Then an elegant, enigmatic publisher from Paris offers David 100,000 francs to write a book that would found a new religion. This fabricated faith must deploy all the hate-driven plot devices of successful scriptures through the ages: "sooner or later, the world becomes flesh, and the flesh bleeds".

Yes, we have slipped into the infernal territory of the Faustian pact. Zafó*believes that "'Faustian tales are the story of modern man... In many ways, our story is a Faustian bargain. We don't ask to come into this world but we find ourselves here... We're faced with choices and dilemmas that are not necessarily the ones we wanted to face".

Zafó*thrillingly tiptoes along the fine line between paranormal events and psychological delusion that modern pioneers of the uncanny, from the Jameses (Henry and MR) on, have trod. Is this a novel of the occult, or of obsession? A melodrama, or a psychodrama? "Depending on who you are, the story acquires different meanings, different interpretations," suggests the master of the game himself. One virtuoso hyper-Gothic scene snaps at the heels of another, from hell-hounds snarling in the grounds of Gaudi's Parc Guëll to the lover's face glimpsed under the ice beside a snow-bound sanatorium. Meanwhile, the publisher descends from time to time (with a whiff of sulphur), checking up on his investment in a poor soul who has pledged to forge a new creed "full of blood and anger".

However objective, or subjective, we deem David's eerie adventures, no reader could remain in doubt about Zafón's baleful depiction of the literary world. Seething with charlatans, cranks and crooks, his bookish Barcelona captures – for all its fantastical décor – its author's disenchanted vision of the writing life. "I'm not David Martí*and he's not me," he insists. "But the worst aspects of his feeling trapped reflects more my experiences with screenplays". Writing mostly figures as servitude and compulsion. "It's a source of self-destruction and unhappiness, and a lot of negative energy," he adds.

By now, Zafó*himself has found in literature a level of reward and recognition far beyond the fantasies of any David Martín. But he took a very roundabout route to fame. Born in 1964, he grew up, as Franco's moribund dictatorship faded away, in a nation trapped within a Gothic nightmare of its own.

"My memories of Spain as a child were of intense mediocrity," he says. "What I perceived around me was a dulled, frozen world – because essentially Spain had been frozen for 40 years... Then, from the late 1970s and 1980s, there was an enormous modernisation of society. The country started moving at a really fast pace because it needed to catch up."

Yet Zafó*found that this dynamism of renewal "didn't belong to me". The young outsider consorted with foreign giants, from Dickens to Welles. "I was always looking far away. Why? I don't know." He did come to love the rich, sad past of his still-wounded city: "There were places where you could smell the civil war in the air".

As opposed to the "sunshiny image" it enjoys today, "the Barcelona of the 1970s, of my childhood, was a much darker place" - a sombre network of buried memories and secret traumas that, in its flamboyant way, the "extreme stylisation" of his books do record. "Civil war leaves a kind of a path behind that is different even from the horrors of World War Two," he reflects.

"Things that have been inflicted between neighbours – it has a different quality, and you can feel that in Barcelona". As a child, "My parents would never talk about the war. My grandparents would never talk about the war. Nobody who had been there would talk about the war. But it seemed that the walls were talking about it." Perhaps, obliquely, so do his novels - even if its events feature hardly at all. Some critics connect the Gothic style in literature with Freud's "return of the repressed".

For all his devotion to his home city, Zafó*remains a sort of stranger in Spain, without any "tribal attachment". He also seeks to rise above the vicious politics of language in his native Catalonia: "I consider myself a Catalan. I'm bilingual; I can read and write both languages. I write my fiction in Spanish because it's the first language I absorbed." Impatient of small rivalries and low horizons, this child of old Barcelona has now inscribed his stories on a page that the world reads. Yet these literary labyrinths will remain books alone.

The former screenwriter refuses all offers to adapt them by the film industry he served. "They are books about the world of books. It just feels wrong to me to transform them into something else... Nothing can tell a story with the depth and the richness that a novel can – if it's done right".