Brian Wilson: Here comes the sun

When Brian Wilson brought 'Smile' to the Royal Festival Hall in 2004, the gig was hailed as the return of a genius. As he prepares for another premiere on the South Bank, The Beach Boys' creative powerhouse talks to James McNair

As a former recluse and agoraphobic who underwent bouts of schizophrenia, Brian Wilson does not sail through interviews. It isn't that he's impolite or deliberately cagey, more that his earlier, psychotropic drug-induced periods of mental illness still cause problems in the people-skills department. Talking to Wilson in 2000, the year after he finally overcame his stage fright and began touring again, one US writer described the singer's responses to his questions as being like "the uninflected bark of an obedient spaniel".

Despite this, nothing was going to keep me from Wilson. Who wouldn't want to meet the 65-year-old visionary behind The Beach Boys' best and most innovative music? Who wouldn't want to meet the tireless perfectionist who visited four different recording studios before emerging triumphant with 1966's "Good Vibrations", still many people's idea of the greatest single ever made?

I meet Wilson at Laguna Beach, California, where he and his second wife Melinda have been holidaying for the last month. There's a spectacular view of the San Pedro Channel. As I sit a little nervously watching hummingbirds fan crimson orchids, Wilson arrives punctually with the blond, bespectacled Melinda and his manager, Jean Sievers.

Wilson is taller than you might expect, a gentle, unassuming presence. He's wearing blue flip-flops, khaki shorts and a many-hued Hawaiian shirt that makes some recompense for his own lack of flamboyancy. His fleeting smile bears testament to good Californian dentistry, and his silver hair is swept back in the Fifties style of his youth. As we watch him being photographed, Wilson claps big hands and shouts, "Go! Go! Go!" like a sports team coach. It's a little glimpse of the motivator he must once have been in the recording studio.

Our meeting has been occasioned by That Lucky Old Sun (a Narrative), the new, Southbank Centre-commissioned song-cycle that Wilson will premiere at the Royal Festival Hall next Monday. "Midnight's Another Day", a preview track, augurs well for the project's success. The song is a piano-led tune that Wilson penned with the famed producer Van Dyke Parks (also the key lyricist on 1967's Smile, the classic "lost" album that Wilson eventually completed in 2004), and boasts choice vocal harmonies and typically Wilson-esque chord modulations. It is the lyrics, though, that are most affecting: "All these voices/ all these memories / made me feel like stone", runs Wilson's careworn vocal, while elsewhere he sings of "[life] chapters missing" and "pages torn", making obvious reference to his own wilderness years.

"That Lucky Old Sun", meanwhile, is a cover version of a standard that Beasley Smith and Haven Gillespie published in 1949. So what drew Wilson to that tune, following in the footsteps of Frank Sinatra and Johnny Cash? "I was sitting in my music room playing my keyboard when I just stumbled across it," he says, immediately breaking into song: "Up in the morning / out on the job / work so hard for my pay / but that lucky old sun got nothing to do / but roll around heaven all day."

And the tune moved you? "Yes, and I felt I had to learn it properly. I knew it was an old Negro spiritual, so I went to Tower Records in Sherman Oaks and bought Louis Armstrong's version. I was blown away, but I thought it needed some more spiritual chords. I didn't want to do a verbatim version like so many other people have done."

Wilson premiered Smile at the RFH in 2004, so it feels natural that he has chosen to do the same with That Lucky Old Sun (a Narrative). He says that "people in Britain and Europe are much more appreciative of the arts," and even goes so far as to cite the love he has felt from British audiences as a factor in the "creative explosion" he experienced in 2006.

"Something just got into me. I wrote 18 songs last summer. When it rains it pours and I put my buckets out and caught everything I could. You want song titles? Well, one's called 'Mexican Girl' and one's called 'Oxygen to the Brain'. One's called 'California Roll' and another one's called 'The Good Kind of Love'. It's crazy lyrics and crazy narration, just little stories about my day. You remember Smile? When you hear the new song cycle it's a teeny bit like that."

Is his writing still influenced by the classical composers? "Oh, yes – I love Bach! I have to tell you that JS Bach was easily the greatest musical innovator in the history of the world. He was so advanced for his time. There's a spiritual depth to his music. You can listen to it and it's like meditation."

Brian Douglas Wilson was born in Hawthorne, California, on 20 June 1942. Together with his younger brothers Dennis and Carl, he formed The Beach Boys in 1961 with their first cousin Mike Love, their schoolfriend Al Jardine completing the first classic line-up two years later.

Initially, their songs employed Chuck Berry's tested "cars and girls" aesthetic, mixing romantic, sun-dappled images of a surfing lifestyle that only Dennis lived, with layered vocal harmonies. "My brother Carl taught me how to play bass," says Wilson. "I'm a self-taught keyboard player, though – I figured out our harmonies at the piano."

It was somewhat ironic that Wilson, deaf in his right ear since infancy, should have become the group's musical leader. But, like his idol Phil Spector, he was recording in mono- rather than stereophonic sound, and thus not especially hampered by his hearing defect. Still, as he began to write more harmonically complex, more sonically adventurous music, the scars allegedly inflicted by his abusive and vicariously ambitious father Murry Wilson were more difficult to shrug off.

Stories of the tensions between Wilson and his father are legion, one even relating how Wilson defecated on Murry's dinner plate as a child. "No, that's absolutely not true," Murry told Rolling Stone in 1971, two years before he died, but other markers of the conflicts he had with Wilson are very much on record. A failed songwriter himself, Murry would eventually trick Wilson into signing over control of The Beach Boys' Sixties song copyrights. A California court later awarded Wilson $25m in compensation.

By 1965, the youth who was listening to Gerswhin at home had also taken LSD and heard The Beatles' Revolver. He announced that Glen Campbell would forthwith be his stand-in on Beach Boys tours, and set about making 1966's Pet Sounds, the brilliantly orchestrated record that's home to the plaintive "Caroline No" and – reportedly – Paul McCartney's favourite song, "God Only Knows".

"I played the tracks to the boys, who had just gotten home from the road," Wilson wrote in his sleeve notes for a 1990 reissue of the album. "They all flipped for the songs." All except Love, that is, who called Pet Sounds "Brian's ego music". The rub was that Love's "sun and fun" approach to lyric writing had been dispensed with in favour of something more poignant, Wilson collaborating instead with songwriter-turned-copywriter Tony Asher. And Love never got over it.

Back in Laguna Beach, Wilson and I contemplate Pet Sounds afresh. Does he think that someone could reinvent pop music in 2007 as spectacularly as he did 41 years ago? "There are only so many notes, but I would say yes, it's possible. It would take another creative renaissance, but I'm not sure how that could happen. There will always be good songs in people that have to come out."

Life appears to have become more stable for Wilson since marrying Melinda in 1995. The couple reportedly met when she sold him a car. With Carnie and Wendy, Wilson's daughters by his first wife Marilyn, all grown up, he and Melinda have adopted two young daughters, Daria Rose and Delanie Rae, and a son, Dylan. "The girls are nine and 10 now," says Wilson. "They're great, but they're so restless and energetic they make me feel like I'm 150 years old. We sing 'Barbara Ann' and 'Surfer Girl' together, which is a lot of fun."

The odd non sequitur aside (Wilson responds to a comment about Californians liking to keep fit with a baffling reference to John Lennon's "Across the Universe"), the most poignant part of our chat comes at the end. "Yes, there are lots of memories for me here," agrees Wilson, taking stock of the view, "and that can be good or bad. My brother Dennis was a surfer," he adds. "I never learnt."

Ah yes, Dennis Wilson. With cruel irony, The Beach Boys' hard living drummer and lone surfer drowned under the influence of alcohol at Marina Del Ray, Los Angeles in 1983 aged 39. Wilson's youngest brother Carl, meanwhile, died of brain and lung cancer in 1998. "When I do 'God Only Knows' today I always think of Carl," says Wilson, his face momentarily cracking in pain. "It's a very sentimental thing, but that's the truth," he adds. "He sang that song so beautifully."

Wilson has been luckier. The LSD- and marijuana-fuelled sessions that saw him equip orchestras with toy fireman's helmets during the making of Smile are the stuff of legend, but in 2004, when Wilson finally completed the album, the sense of closure must have been immense. One suspects that no such closure is on offer when he turns to the story of his own family, however. Something of his very tangible sadness must surely derive from surviving his two younger brothers.

Later, I spot Wilson under a parasol by the pool. He is sitting very still, looking out to sea past a group of excitable children who are unfolding a large, brightly coloured blanket. Nobody seems to recognise him, or, if they do, they leave him in peace. He seems an almost spectral figure; the ghost of pop's glorious past.

Brian Wilson premieres 'That Lucky Old Sun (a Narrative)' at the Royal Festival Hall, London SE1 (0871 663 2500) from 10 to 16 September; he then tours to 24 September (see www.brianwilson.com for details)