Jackie Leven: Musician whose songs were peopled by loners and lost lovers

Leven: In concert he was a dryly charismatic troubadour (DAVID SINCLAIR)

The singer-songwriter Jackie Leven chronicled the inner lives of what his friend and fellow Kingdom of Fife native Ian Rankin described as "disappointed hard men". His lyrics, often set in the raw landscapes he saw as a boy, were populated by hard-drinking loners, their lost lovers and mothers. He found a redeeming sadness in such characters, which were partly self-portraits of a life marked by addiction and exile from human company. Stardom briefly seemed possible with his notorious post-punk group Doll by Doll, but he remained on the margins.

Born in 1950 in the conservative community of Kirkcaldy, Fife, to a London-Irish father and Northumbrian mother, both Romany Gypsies, he was an outsider from birth. His mother played blues records around the house, and by 16 Jackie was playing in local folk clubs, and briefly married. Though a prize-winning pupil at school (where Gordon Brown was in his year), he was the first Scots student expelled for drug-taking. A vendetta by a local gang forced him to flee Fife aged 17. During the late 1960s, he spent months huddled homeless around London's South Bank, and worked as a labourer in Kerry, Berlin and Madrid, where he recorded his first album, Control (1971), under the pseudonym John St Field. He also once mentioned he had married a millionairess.

He met his Doll by Doll bandmates while squatting Dorset farmhouses in the early '70s. "In some respects the best days of our lives, in a wonderful dream of Old England," he recalled. They reconvened in 1975, in a west London squat, Bristol Gardens, a street with "a huge contingent of badly damaged, but nevertheless wonderful fun Scottish guys ... prostitutes and petty thieves". Doll by Doll carried their own "serious unresolved edges". Leven had a near-fatal overdose before they made Remember (1978), a blunt instrument of a debut which approximated their confrontational live act. "I can remember a couple of nights at the Marquee where I would have given anything to be in that audience," he recalled. "At the same time, there were people thinking, 'If I don't get away from this band right now, I might literally go insane.'"

Their second album, Gypsy Heart (1979), was Leven's lyrical breakthrough. Its romantic Celtic soul music explored what would become enduring themes: identity, wanderlust and landscape. "I've never had my own place to live, I've always ended up living at other people's, and that continues to this day," he told me in 2007. "But that record made me feel I had a place in the world, even if it was a very unpleasant place that wasn't going to do me any favours."

Record-company politics and their own unstable natures did for Doll by Doll after two more albums. In 1983, Leven was strangled and had his throat slashed in a mugging. Unable to speak or sing for over a year and dropped by his label, he found solace in heroin. He would sit for weeks in Marylebone station, silent and profoundly depressed.

Recovering in 1985 via acupuncture and "psychic healing", he co-founded the CORE Trust, a charity which offers holistic help for addiction. Through it, he met Princess Diana. Moving to Scotland's West Coast, he soaked up the "fierce warmth" of trawlermen in the bars at night, and began to write again.

In 1994, he returned to music with The Mystery of Love is Greater Than the Mystery of Death, the first of 24 albums for Cooking Vinyl. These formed a prolific, redemptive songbook about a cruel world, often sung and played with deceptive lightness. His own partial healing, and 1990s fascination with the "Iron Man" philosophy of Robert Bly, gave him new perspective on the scarred tough men of great songs such as "Classic Northern Diversions". He settled happily in Hampshire with his long-term partner Deborah Greenwood, but each year he took a sleeping bag, and wandered country lanes again. "My missus says, 'Jackie, you're a vagabond and I accept that'," he said.

In concert he was a charismatic troubadour. Settling at the bar for a pint before the gig, he'd then leaven his songs with dryly hilarious tall tales. Four live albums released in 2009, under the general heading "The Haunted Year", give the fullest picture of his work. A yarn where he announces Sting's death on a Kirkaldy commuter train is inseparable from the songs. His last album, Wayside Shrines and the Code of the Travelling Man, was released this year, during which he continued to tour, until the cancer which killed him made it impossible. "I sometimes think I'm too connected to the pain of other people," he once said, unsure if this was a good thing. "Out of that, and anger about that, I find myself writing." This sympathy for often violent, stoic people who were rarely offered it had, in his finest songs, few parallels.

Nick Hasted

Jackie Leven, musician: born Kirkcaldy 18 June 1950; one son; twice married; died 14 November 2011.