John Malcolm: Actor and founder of the Traverse

The Edinburgh Festival Fringe, the annual gallimaufry of drama, music and stand-up, today operates with awesome commercial proficiency. More than a few, however, rather wish that the Fringe's spirit still had some of its early, unpredictable air of misrule, together with the genuinely trail-blazing fused by such maverick personalities as Richard Demarco, Jim Haynes or John Malcolm.

All three were pivotal in Fringe theatre and, most crucially, in the creation of Edinburgh's Traverse Theatre, now sleekly contained in glass and marble beside the Usher Hall, and this year celebrating its 40th birthday. The Traverse originally opened in a small space within the converted premises of a Lawnmarket brothel in what was then a decidedly seamier part of the city. Somewhat characteristically Malcolm, always a feisty, on occasion combative personality (fellow Scots often described him as "thrawn"), became less involved with the project once the initial back-breaking process of getting it open was over.

Malcolm was born in Stirling in 1936. His early life was unhappy, shadowed by a formidable father. Initially religiously inclined – he trained as a Methodist lay preacher – only later did he turn to the stage. He trained at Rada and was soon playing good supporting roles, especially enjoying his time in Dublin in Brecht's Saint Joan of the Stockyard.

Less happy was his return to Scotland for a summer season at the Pitlochry Festival Theatre (housed in a tent in those days). Malcolm's less-than-reverent reaction to the costively genteel good taste of Kenneth Ireland's operation ("Stay six days and see six plays") was predictable from an actor who had worked for and venerated both Joan Littlewood at Theatre Workshop and George Devine at the Royal Court and he was abruptly dismissed.

Malcolm's appearance in an off-beat and ambitious 1962 Edinburgh Fringe project, Fionn MacColla's Ane Tryall of Heretiks, was for him a life-altering event. It was staged in a space housing only a small audience in the Paperback Bookshop ran by Jim Haynes near the University, which had become something of a focal point for a group of young people keenly involved in the arts and reacting against the city's grey Presbyterian ethic.

Edinburgh – as Scottish writers from James Hogg to Muriel Spark and Irvine Welsh have amply demonstrated – is nothing if not a bifurcated city, both geographically and spiritually, and the group based around Haynes's bookshop capitalised on the city's riven nature. Haynes, together with the gallery owner Demarco and the publicist John Calder, held their first international Writer's Conference in 1962 in the McEwan Hall and they were considerably spurred on by Malcolm to start realising their vague plans for a permanent avant-garde art centre.

Both Malcolm and Demarco had seen two Cambridge undergraduates, John Cleese and Tim Brooke-Taylor, perform in 1962 in a space in James Court, in the former house of ill-repute in the Lawnmarket, then called the Sphinx Club. Demarco led fund-raising while Malcolm – who had been the first to approach Tom Mitchell, later the Traverse president and owner of the building – worked hard to convert the space, fitting it out with 59 seats which he and Terry Lane, the Traverse's first director, personally ripped out of the recently closed New Palace Cinema.

Joyce McMillan, the leading Scottish theatre critic, has described Malcolm's contribution to the Traverse as that of "in some ways the most important character of all"; certainly the enterprise's air of barely tamed anarchy through much of its early life could be said to stem from him. He quit the Traverse after a characteristically rousing row with Terry Lane during rehearsals for the opening production of Sartre's Huis Clos.

Television occupied Malcolm for a period subsequently; he had a good run in the soap-opera Crossroads alongside appearances in Scottish-set series such as The Borderers and Dr Finlay's Casebook. Between 1964 and 1966 he was a member of the Royal Shakespeare Company, playing mainly supporting parts. He loved the Warwickshire countryside and with his then wife Tamara he converted a former Salvation Army base in Chipping Norton into another new theatre space – named, uncompromisingly, The Theatre – which they ran jointly until the end of their marriage in 1977.

Thereafter Malcolm's work was mostly on screen – Pennies From Heaven and When the Boat Comes In on television and several effectively unsettling studies of various Nazi chiefs in films including The Dirty Dozen – Next Mission (1985) and David Hemmings's The Key to Rebecca (1985)

An especially fine stage opportunity came Malcolm's way when Kenneth Williams cast him as the terrifying Truscott, the police inspector who invades the venal household of Joe Orton's Loot (Lyric, Hammersmith, 1980). There was considerable speculation in theatre circles about the fraught possibilities in the partnership between two such combustible characters as Malcolm and Williams, but rehearsals proved to be stimulating and rewarding, with Malcolm giving a performance of striking distinction.

Edinburgh had changed hugely when Malcolm returned to the city in his later, illness-stricken, years. The Traverse may now be housed in style but something of the spirit and moral force of the first building has survived the move and different directors. Writers such as John Byrne, Gregory Burke or David Greig all, in different ways and to varying degrees, managed something of that pugnacious Scottish conscience wedded to a wider international awareness that stamped Malcolm and the Traverse's original ideas alike.

Alan Strachan

John Malcolm, actor, director and manager; born Stirling 26 March 1936; married (one son, one daughter; marriage dissolved); died Edinburgh 13 June 2008.