Scared? That's the spirit

The ghost story is timeless but today we like our spectres served up with a splatter. Kim Newman rattles his chains on behalf of celluloid ghosts past and present

The current manifestation of the amiable ghoul Casper, who, in the film Casper, has dropped his "the Friendly Ghost" tag from the by-line, may seem merely the latest exercise in suburban nostalgia and special effects ingenuity, but it continues a tradition of Hollywood ghostliness far older than the marginally-remembered cartoons and TV series from which the character has been summoned. Ghosts have walked the stage since Shakespeare, but Hamlet's murdered father or Banquo's accusing spectre have always had to be played by actors in blue greasepaint and bloody shrouds. The early cinema discovered special effects by way of stage magic, with the illusionist Georges Melies using primitive double-exposures to enable actual transparent spirits to cavort and cackle. Suddenly, ghosts could be convincingly ghostly.

The spiritualist craze of the late Victorian era, with its ectoplasmic materialisations and table-tapping, co-existed with the very first exhibition of the cinema of the fantastique and with the literary tradition of MR James, whose ghosts tend to be creepy rather than cuddly, and often make their presence felt rather than seen. James, and other ghost story writers, treat their apparitions as his fellow Victorian and Edwardian worthies treated the poor: the spirits are either waifs to be pitied (big-eyed children, wailing victims of unavenged murders) or thorough rotters (executed felons, inhuman fiends, crawling hands). To these archetypes, the cinema added the harmless but active spook, of which Casper is the latest incarnation.

The ghosts of Melies, who feature in the likes of Le Chateau Hante of 1897 and Le Spectre of 1899, tend to be mischievous, often running amok like Slimer in the Ghostbusters series, serving, like bad fairies and devils, as surrogate illusionists, orchestrating the other tricks. It is possible that the early cinema drew on a repertoire of gimmickry that was also used in the 1890s by fraudulent mediums intent on deceiving their clients into the belief that they were in the presence of genuine ghosts.

Silent cinema gave rise to dozens of adaptations of many popular stories featuring ghosts (Hamlet, A Christmas Carol, The Legend of Sleepy Hollow), but somehow few made an impact. Though special effects could produce a decent apparition, the real feel of ghostliness requires more than sight. Certainly, sound allows for a great deal more business: the whistling wind, footsteps in an empty hall, the creak of a door. Often, following James's craze for damp and disgusting ghosts, there is a striving to add other sensations: The Uninvited, a rare serious ghost story from 1944, features an evil spirit who manifests herself as the scent of mimosa. Robert Wise's 1963 film of The Haunting, just out on video in a wonderful widescreen edition, almost catches an unforgettable, but impossible-to- visualise moment from Shirley Jackson's novel The Haunting of Hill House. In the dark, the terrified heroine (Julie Harris) reaches out and holds the hand of her friend (Claire Bloom) only to realise that Bloom is on the other side of the room and the hand she is holding belongs to someone else.

Surprisingly, The Uninvited (from a novel by Dorothy McArdle) and The Haunting are among only a handful of ghost films which set out to be as scary as most ghost stories. Both, significantly, are adapted from literary properties, as are The Innocents (from Henry James's The Turn of the Screw) and Don't Look Now (from Daphne du Maurier). And few of these are content to be just scary stories in the Jamesean tradition: both The Innocents and The Haunting strain to be character studies of their repressed heroines, imposing psychological interpretation on the spooky business with which lovers of the weird might be impatient.

The overwhelming majority of Hollywood ghosts are comical. Every comedian in the business has made at least one haunted house picture, albeit with spectres who usually turn out to be smugglers or spies in disguise: Abbott and Costello (Hold That Ghost), Bob Hope (The Ghost Breakers), Dean Martin and Jerry Lewis (Scared Stiff, a remake of The Ghost Breakers), the Bowery Boys (Spooks Run Wild). There are also self-explanatory offerings like The Mexican Spitfire Sees a Ghost, Henry Aldrich Haunts a House, Hillbillys in a Haunted House and Francis in the Haunted House. These are broad comedies in which a stumblebum under a white sheet is enough to provoke the hero to screaming hysterics. It might be an interesting avenue of analysis to assume the extremely scared black sidekicks that often crop up (like Willie Best in The Ghost Breakers) are reacting not to intimations of the beyond but visual echoes of the Klan.

A little more sophisticated are the double exposure high jinks of Cary Grant and Constance Bennett in Topper, relishing an afterlife complete with cocktails as they drive Topper (Roland Young) out of his mind. This 1937 fable, from Thorne Smith's novel, produced sequels, remakes and a TV series: its Depression era message was that even death could be turned into a playground.

During the Second World War, the afterlife edged closer for most people and there was a rash of sentimental, ghostly romances, with heavenly messengers fixing mistakes to reunite lovers separated by death (Here Comes Mr Jordan, A Matter of Life and Death) or giving a disillusioned hero a lesson in patriotic self-sacrifice (The Remarkable Andrew, A Guy Named Joe). Even Charles Laughton as a cavalier version of Oscar Wilde's The Canterville Ghost is disturbed by the Blitz and shows some unlikely patriotic spirit, for all the world like a spectral Mrs Miniver.

After the war, sentiments became clouded, romances more fetchingly impossible. Those familiar only with the sit-com of The Ghost and Mrs Muir are invariably reduced to floods of tears by the glossy 1947 film, with an impossibly beautiful Gene Tierney retreating from a harsh world into a lifelong relationship with ghostly sea captain Rex Harrison. Also striking and tear-jerking is the 1948 Portrait of Jennie, in which artist Joseph Cotten keeps re- encountering ghost Jennifer Jones as she rushes through a sped-up replay of her tragic life.

One of the odder aspects of the ghostly films of the 1940s is the simplicity of their effects, creating an otherworldy feel through the distracted beauty of Jennifer Jones or the echoing tones of "Stella By Starlight" (the tune that haunts The Uninvited). The big set-pieces of invisible people making mischief are reserved for out-and-out comedies. The more serious a film ghost is, the less likely it is to go in for showy demonstrations of supernatural conjuring.

The post-modern ghost movie has schismed, splitting off into gruesome psychological splatter movies like The Shining and A Nightmare on Elm Street, updated 1940s fables like Ghost and Always, and wild comedies like Ghostbusters and Beetlejuice. Unlike their 1940s predecessors, these movies are eager to display their effects and make a point of spending a lot of money on the visuals. Casper, desperate for love and friendship, makes a spectacle of himself in an attempt to be ingratiating, and the film-makers back him up with all the imagery available to them; but somehow he transforms into a triumph of sugary soulless science. He may captivate Christina Ricci, but we know that like Puff the Magic Dragon he'll be dumped as soon as she discovers sex. Mrs Muir would certainly not spend decades alone in the hope of a post-mortem reunion with this pudgy little sprite.