Man walks in. Man is sick. The end

Films by artists: they inspire awe and dread in equal measure, says Matthew Sweet. But are any of them worth watching?

A soldier, his face muffled by a gas mask, bursts through the window of a spacious apartment, sweeping his rifle from side to side. He levels his gun at a cabinet of family photographs, and blasts away. Ping pong balls ricochet around the room, sending the photographs flying. And out on the street, people are evaporating inside their clothes. A man standing by a tree is suddenly reduced to an empty shirt and slacks. A couple standing in an alleyway becomes twin piles of crumpled clothes. A soldier seated at a desk picks up an R/T unit: "Towers!" he barks. "Open fire!" And the screen becomes a cascade of frames, which flash by with near-subliminal rapidity: chickens milling around their coop; a bullfighter sticking home his swords; a line of cars; a couple necking on the grass; a figure dancing under a bridge; a bespectacled figure who looks suspiciously like William Burroughs.

A soldier, his face muffled by a gas mask, bursts through the window of a spacious apartment, sweeping his rifle from side to side. He levels his gun at a cabinet of family photographs, and blasts away. Ping pong balls ricochet around the room, sending the photographs flying. And out on the street, people are evaporating inside their clothes. A man standing by a tree is suddenly reduced to an empty shirt and slacks. A couple standing in an alleyway becomes twin piles of crumpled clothes. A soldier seated at a desk picks up an R/T unit: "Towers!" he barks. "Open fire!" And the screen becomes a cascade of frames, which flash by with near-subliminal rapidity: chickens milling around their coop; a bullfighter sticking home his swords; a line of cars; a couple necking on the grass; a figure dancing under a bridge; a bespectacled figure who looks suspiciously like William Burroughs.

Films by artists. The phrase doesn't produce the same sense of bowel-glaciating dread as say, "novels by academics," but prejudice against such works runs deep. Art institutions neglected them for decades. Theatrical film-makers tend to mistrust them: "When I was growing up," recalls the British director David Mackenzie, "experimental films were made by Malcolm Le Grice and involved people shaving for 10 hours." Art dealers don't seem to have much enthusiasm for them, either: "I wouldn't say I had a problem with things you have to plug in," ventures the gallerist Timothy Taylor, "but you have to put them into a different category. I want to see the intent, the complication, the struggle, the depth, and I find it easier to see that in work that isn't moving or on television." And the theoretical writings relating to experimental film – often informed by a puritanical distaste for classical narrative – have enhanced the impression that this form of art constitutes a series of humourless endurance tests. ("Though I was critical of the film's denouement," wrote Malcolm Le Grice in 1979, discussing a work by his fellow artist Peter Gidal, "I was particularly impressed by the slow camera movement over the unspectacular surfaces and objects of a room and even more so by the extreme device of an absolute repeat of the whole film.")

"A Century of Artists' Films in Britain", an immense new exhibition of 170 works by 130 practitioners that will occupy Tate Britain for an entire year, aims to alter these perceptions. Its range encompasses the last years of the 19th century and the first years of the 21st. Its programmes include James Williamson's Victorian trick-film The Big Swallow (in which the director moves towards the camera and gulps the film down his throat), Duncan Grant's Abstract Kinetic Collage Painting with Sound (1914), Margaret Tait's 1956 short Rose Street (which the director insists contains a cameo from a barrow boy named Sean Connery) and Tracey Emin's Why I Never Became a Dancer (1994), her vigorous two-fingered salute to the boys who humiliated her at a dancing competition in Margate. As much of this material is very difficult to see, and no show of this size has ever been staged before, it will be the first serious attempt to restore the experimental film to the wider history of British art and British movie-making.

"Until recently nobody had worked out what to do with this kind of stuff," argues David Curtis, curator of the show. "Cinemas didn't programme it, galleries didn't show it. There wasn't a market for it. So many of these films were simply shown by artists to their friends." To some extent, he admits, this was a self-imposed exile. "In the 1960s, a lot of the people who became enthusiastic about film resisted getting into the art market because they liked the idea of film being a mass medium. They thought that by resisting doing limited editions they would be greeted by a huge public. And exactly the opposite happened." Most of this work vanished into oblivion.

"Experimental films fall into an interesting gap between the visual arts and independent film," says Chris Welsby, whose Park Film (1972) and Seven Days (1974) feature in the exhibition. "The work has, for the most part, been met with indifference, incomprehension or hostility from both camps. Sadly the mainstream press have only contributed to the problem, and until recently, any grudging attention given to the work has been dismissive, hostile or just plainly at a loss. Worst of all there has been little critical discussion in the major art magazines or film journals. This lack of critical attention has contributed to the ghettoisation of experimental film. Thankfully the persistence of the film-makers and curators like David Curtis, the work is now being brought to a larger forum." Welsby is also troubled by the absence of experimental films in standard art histories. "A browse round the art section of any of the major book stores will reveal that most anthologies of 20th century art make no mention of experimental film and yet surely moving images are the most quintessentially 20th-century of art forms!"

A comprehensive history of experimental film-making in Britain has yet to be written. A canon remains unestablished. Everything, therefore, is up for grabs. What is an "artist's film" or an "experimental film"? Is it something screened in a gallery, and not at the Odeon? It seems not. Many of the titles claimed by the Tate show – Basil Wright's Song of Ceylon (1934), Humphrey Jennings' Listen to Britain (1942) – were reviewed in the papers alongside the work of Gracie Fields and George Formby. Is it something made by a film-maker who has rejected the commercial film business? Nope. Look down the list of the Tate's screenings and you'll find figures with dozens of mainstream credits. Adrian Brunel made several pictures with Ivor Novello. Anthony Gross and Hector Hoppin were under contract to Alexander Korda. John Maybury – represented at the Tate by Tortures that Laugh (1978), in which shaven-headed figures count on their fingers under a chilling blue light – is about to shoot a film with Mark Wahlberg.

One of Curtis's main aims, he says, is to "fuzzy up the edges between popular and art cinema". The massive volume of material in the Tate show cannot fail to demonstrate the permeable nature of the border between these two traditions. They have always fed upon each other's ideas, even if they have rarely been prepared to concede the fact. Andy Warhol's Sleep (1963) attempted to persuade a cinema audience to watch a man tossing and turning in bed for 321 minutes; Big Brother now repeats the event annually. The principal visual trick of The Matrix (1999), roving around within a freeze-frame, was achieved in 1983 by Tim MacMillan's Water. The Chapman Brothers' 1998 Sacrificial Mutilation and Death in Modern Art by Jake and Dinos Chapman (in which a rubber glove filled with red paint is the principal actor in an re-enactment of Jackson Pollock's fatal car crash), owes much to those Adam and Joe sketches in which the principal roles are filled by soft toys and plastic robots. The visual howlaround of the video feedback loop (the pulses and streams of light that are generated when a camera is pointed at its own monitor screen), were a prime source for 1960s experimentalists – but the Doctor Who title sequence got there first. Perhaps a wider recognition of the common ground between popular and experimental film might be the first step to recovering the history of the latter.

There's no better place to start than with the work described at the top of this piece. Towers Open Fire (1963) was the fruit of collaboration between Anthony Balch and William Burroughs. Everyone knows Burroughs; few know the name of Balch. As well as being a denizen of the Beat Hotel, Balch was a maker and distributor of theatrically-released movies. He imported foreign art films and marketed them under lurid titles such as Don't Deliver Us from Evil and Weirdo Weekend. He shot mass market pictures such as Horror Hospital (1973) – a blood-slaked flick in which a not conspicuously sane scientist (played by Michael Gough) invites a chirpy teenager (played by Robin Askwith) to join his army of lobotomised supermen.

A sample scene from Horror Hospital, just for comparison: a long dining table, presided over by an old woman in false eyelashes and a scarlet frock. Around her, the inmates of the hospital – ashen faced, red-eyed, white overalled – sit in silence at their empty white plates. Suddenly one of the patients begins to scream, clapping her hands over her ears. Two helmeted bikers rush in and drag her away. Dinner resumes, in silence. Horror Hospital has never been included on any list of experimental films. So when the soldier with the ping-pong ball gun jumps through the window in Towers Open Fire, what are we looking at? The debate starts at the Tate.

'A Century of Artists' Film in Britain': Tate Britain, London SW1 (020 7887 8008), 19 May to summer 2004