The haunting of Shirley Jackson: Was the gothic author's life really as bleak as her fiction?

Fifty years ago, one of America's eeriest writers, addicted to drugs and alcohol, died of a heart attack, aged just 48. Her life seemed as paranoia-filled as her novels. Yet a new collection of unpublished work – and the son who found it – tell a different story: of a loving, gregarious mother. So what drove the darkness that left a generation of readers reeling in horror?

Jackson wrote: 'I live in a dank old place with a ghost that stomps around in the attic room we've never gone into' (Masha Manapov)

Twenty years ago, Laurence Jackson Hyman found a shabby box on his porch. There was no sign of who had left it, nor a return address. After hours of suspicious avoidance, he decided to take a peek. Inside, he found a treasure trove of unpublished tales – by his mother. It was, by all accounts, merely the latest mystery in the history of weird-fiction author Shirley Jackson.

Paranoia and uncertainty underpinned both Jackson's writing and her life. She died of a heart attack 50 years ago this week, at the age of 48, addicted to prescription medicine and alcohol – dependences that some have suggested were brought on by an unhappy marriage to a man who had been sleeping with his students.

Though she was a queen of gothic fiction and a hugely controversial figure in her native America, she has curiously been all but ignored on this side of the Atlantic – but the situation could be about to change with the publication of those lost pieces. Some of the manuscripts left on her son's doorstep, easily identifiable to the family by the paper stock and the familiar impressions of Jackson's trusty Royal typewriter, have formed the basis of a new book of largely unseen work.

Her reputation as one of the 20th century's most outstanding "weird" writers is well founded. American students have for decades studied her short story "The Lottery", in which the residents of a small town draw lots to choose which of their number will be ritually stoned to death to ensure a good harvest. If the concept sounds familiar, that's no surprise: it predated the similarly themed pagan film The Wicker Man by 25 years – and Suzanne Collins' bestselling Hunger Games trilogy by half a century.

When the story – Jackson's first published piece – appeared in The New Yorker in 1948, it drew a deluge of hate mail from shocked readers. "I think it was too much for the public to take," says the author Joanne Harris – a fan, who credits Jackson as an influence. "People wrote letters calling it disgusting, saying it upset them. Maybe it was because she was writing in an area that people didn't think she was qualified to; perhaps it was something to do with her being a woman."

Indeed, Jackson was expected to undertake the role of a 1950s housewife even when she was at her peak – ferrying her four children to and from school and doing the laundry, before finding a few moments here and there to bash away at her Royal. It often seemed that her husband, the literary critic Stanley Edgar Hyman, was more respected than Jackson during her lifetime.

Her best-known novel, The Haunting of Hill House, is the quintessential haunted-house story. Published in 1959 and filmed in 1963 as The Haunting, it is a masterpiece of edgy tension and creeping terror, telling the story of a paranormal investigator and his carefully picked guests, who stay in the titular property in a bid to prove the existence of the supernatural.

One intriguing aspect of Hill House is its ambiguity, especially whether the protagonist, Eleanor Vance, really experiences – or just imagines – the horrors that unfold within the mansion: disturbing messages written across hall walls, mysterious knocks on doors, distant, ghostly singing. In one famous scene, Eleanor and fellow investigator Theodora are in bed in the darkness, holding hands for comfort, as crashes and bangs intensify in the night. When the lights come on, Theodora is nowhere near, leaving the already-fragile Eleanor to wonder tremulously aloud: "God! Whose hand was I holding?"

A similar paranoia pervades the short story "The Daemon Lover", in which a woman experiences mounting panic as she waits for a man who never arrives. Elsewhere, Jackson wrote of a girl wrapped up in multiple personalities; a rape that drives a woman to an alienated, yet seemingly self-gratifying madness; and even the end of the natural world.

Amid the furore surrounding her writing, Jackson did nothing to dampen the public disquiet. In fact, she liked to say she was a witch, possibly in response to an Associated Press reporter who once said she wrote "not with a pen but with a broomstick".

It was all in keeping with a woman who kept up to six black cats at home, and who supposedly believed in magic – according to her New York Times obituary, "she had considered herself responsible for an accident to an enemy by having fashioned a wax figure of him that had a broken leg". And as she reveals in a short, mischievous essay among her newly published work, entitled "The Real Me", Jackson eschewed the pretence of being "a trim little housewife", asserting: "I live in a dank old place with a ghost that stomps around in the attic room we've never gone into… the first thing I did when we moved in was to make charms in black crayon on all the door sills and window ledges to keep out demons."

Alongside her gothic work, Jackson chronicled her family life with a Dorothy Parker-esque acerbic wit – and suggestions that not all was well in her own home were not without foundation. Some biographers claimed that her husband slept with students at the all-women's college he taught at in Vermont – indeed, after Jackson's death in 1965, he married one of them.

A scene from The Haunting starring Julie Harris and Richard Johnson (Rex) (Rex Features)

So, were the disquieting undercurrents that ran beneath her stories of small-town America informed by her own, private difficulties? "Perhaps the stories were the way she coped with her life and her marriage," suggests the author Neil Gaiman, another Jackson aficionado. "She seems to have been increasingly miserable, trapped and sad."

Both Gaiman and Harris cite Jackson's novel We Have Always Lived in the Castle as a favourite. The story of two sisters ostracised over suspicions that they murdered their parents is, says Harris, "deeply unsettling". Gaiman adds: "It's a wonderful literary novel that's also a mystery and a perfect study of the inside of someone's head – a strange novel that lives on the border of fantasy without ever crossing it. I love the unease of it."

For Harris, it's that uneasiness which defines Jackson's work. "Calling her work American gothic isn't a satisfactory description for what she does. American gothic is more about the big revelations, the climactic scenes. With Jackson, the strangeness is more trippy. She builds tension that never breaks. Reading her, you're left with a sort of thunderstorm headache."

Yet, for Laurence Jackson Hyman, the eldest of Jackson's children and now 72, his upbringing was typified by anything other than darkness. He gives short shrift to biographies that dwell on her supposed problems – branding one in particular "mean-spirited and factually absurd".

"Both my parents regularly drank alcohol – as did virtually every one of their friends and colleagues," he tells me now. "They gave great parties, and often world-renowned artists, sculptors, actors, teachers, musicians, composers and writers" – among them Dylan Thomas, JD Salinger and Ralph Ellison – "would come to our house to spend time with my parents.

"Much has been made of my mother's use of 'drugs', but she never took anything not prescribed by our family physician. My mother had some health issues in the years before her very early death, and she took medications for those. She did not take any medicines for 'pleasure' and often resented having to take any at all.

"My parents wrote constantly," he adds. "Sometimes they would both be going at their typewriters with a ferocity unmatched in a newsroom at deadline. Shirley would write whenever she could find the time, in the mornings when we were at school, the afternoons, the evenings and sometimes far into the night. We would hear her typewriter pounding away; it was part of the fabric of our household life. She would leave notepads around the house and in the kitchen so she could make notes about plots, stories and characters as she did the housework or cooked."

Laurence and his sister Sarah have carefully edited the previously unpublished works left on his doorstep, together with more manuscripts donated after Jackson's death by her husband to the Library of Congress in Washington. The resulting collection, Let Me Tell You, is an eclectic mix, comprising short stories with the trademark Jackson weirdness, but also those light, comical family pieces, which she wrote in abundance for The New Yorker, among others.

Jackson's husband died in 1970 and her four children are now the curators of her legacy. They won't be commemorating the 50th anniversary of her death next Saturday, but are rather looking ahead to 2016, and the centenary of her birth.

"Shirley was a very loving and attentive mother, and I never felt she even remotely let the darkness of some of her writing affect her family life," muses Laurence. Her legacy, he says, should be her writing: "My mother has 14 books in print right now, plus a couple of children's books, and she wrote literally hundreds of short stories in her lifetime."

Jackson has not exactly slipped into obscurity since her death– there is an annual award in her name given out to weird fiction that summons the spirit of her writing – but as Harris points out, she remains "enormously underrated". Perhaps Let Me Tell You will bring her to a wider audience – and allow Shirley Jackson to claim her rightful position as one of the most important writers, gothic or otherwise, of the 20th century.

'Let Me Tell You' is published by Penguin Classics, priced £20