He likes playing for people who don't like him. Good job too. The fans have gone; the hat is no more, but Gilbert O'Sullivan carries on regardless

The Gilbert O'Sullivan we see on the cover of his latest album, Every Song Has Its Play, harks effortlessly back to the Gilbert O'Sullivan who first made it big. He is shorn of the old, early Seventies Top of the Pops garb (flat cap, sleeveless pully, hobnail boots and, in the product of one unforgettable photo session, shorts) but still young-looking and more than a little melancholy round the eyes.

The naked woman, on the other hand, whose rear view occupies the left side of the frame, would have to be considered, for the writer of gentle family favourites such as "Alone Again (Naturally)", "Clair" and "What's In A Kiss", something of a departure. "I wanted to make it clear that this was an adult album," O'Sullivan told me, further explaining that, as an ex-art student, he had always maintained an interest in nudes.

We met at the smart Athenaeum Hotel in London. O'Sullivan is apparently a fan of reassuringly expensive afternoon teas, and the venue was his choice, arrived at by a process of elimination which took out Fortnum & Mason (not open late enough) and his favourite Piccadilly lounge (obtrusive pianist on weekdays). O'Sullivan, who had just travelled in by boat and train from his tax haven in Jersey, entered at great speed in a colourful shirt, khaki trousers, green plimsolls and the kind of strap-on eye-shade worn by golf pros. He talks softly for the most part and sometimes with the side of his forefinger across his mouth. He speaks a lot about not giving a damn what people think but appears even in this slightly guarded, as if ready in advance for the gags which must come when you're the composer of "Ooo-wacka-doo-wacka-day". "I see myself in competition with Blur and Oasis," he says. "But everyone else just sees me as this guy with a history."

Well, maybe. But it's quite a history: two albums that stayed in the British charts for well over a year each; three American million-selling singles; and one exhausting piece of career-mangling, five-year litigation, albeit with a happy ending. The irony of O'Sullivan having become since then, in some eyes, a period piece, is that it was always part of his schtick to appear to be standing outside the times. His 1973 smash, "Get Down", for instance, was not an exhortation to dancers, in the hip expression of the day, but a warning to his dog to keep off the furniture.

Twenty-two years later, O'Sullivan is still at it, gigging and recording, though the hits have stopped coming, the vogue for his old-school, Tin Pan Alley pop having passed, and it's just possible that he pauses every now and again and wonders where he now fits in. The news that he was on the bill for this year's Glastonbury Festival, which took place at the weekend, couldn't help but raise the diverting image of O'Sullivan playing "Matrimony" for a field full of dope-sozzled New Agers. "I quite like the idea of performing in front of people that don't like me," he says philosophically, and then refers me to the tour of student dance halls he undertook, against all advice, in the late Seventies. "They started out a bit doubtful, but I won them over. The quality of my songs will get through to people. They are good songs. Lyrically, some of them are interesting: there's stories, a bit of humour. I'm very confident about the music I play, you know."

Gilbert O'Sullivan, whose real name is Raymond, is 48. He was born in Waterford in Ireland and retains the accent, though he says his earliest memory is from Battersea in London, where his father, a butcher, had come looking for work. He recalls, aged seven, flapping about with his sister, robed for their first communion. Shortly after that, the family moved to Swindon, which O'Sullivan regards as his home town. Swindon happily colludes with him in this. His face was painted in the late Seventies on a mural in the town centre along with other key figures in Swindon's history, including the four members of the pop group XTC, Rick Davies of Supertramp and Isambard Kingdom Brunel. "I don't know why people knock Swindon," O'Sullivan says. "It seems to crop up a lot in comedians' routines. I never understood that. I've always liked the place."

In 1967, after art school in Swindon, where he studied graphic design, O'Sullivan left home for London. "My mother was pushing me out of the house. She reckoned I was too comfortable at home."

It would be impossible to overstate the young O'Sullivan's self-confidence at this point. No amount of rejections of his demo tapes deterred him. He knew he was good and he thought it was only a matter of time before objectors woke up. He had a rudimentary ability as a pianist, a recognisable voice and a vast reservoir of pluck. And, to go with them, he soon had a small deal with a music publishing company which managed to sell a couple of his songs to the Tremeloes. But all the while, there was developing in O'Sullivan's mind a vision - of a singer in a pudding-bowl haircut and a flat cap.

"I was very honed in on how I wanted to look. And it was always going to work. I didn't have one evening at home, depressed that it wasn't going to happen." His approach with record companies was, as he put it: "Here I am: I've got an image like nobody else's, I sound like an old man - I must be a dream come true. Go out and sell me! And they're looking at me, thinking, 'Go and grow your hair and put a pair of blue jeans on'."

But he refused. In an age devoted to conspicuous glamour - to all-over lame and blinding sequins and hats with mirrors sewn into them - O'Sullivan saw fit to offer us an image redolent of good old-fashioned penury.

"I'm the first to admit that that image destroyed any credibility I had with my own generation. Image was a dirty word back then. There was a lot of pressure on me to look like James Taylor, to conform. Maybe I would have sold more records. But that was the price I paid and I would do exactly the same again."

He played a shrewd game early on, dallying with small management companies, but avoiding contractual ties because he believed he was saving himself for the big one - for Robert Stigwood, who managed the Bee Gees or Gordon Mills who managed Tom Jones and Engelbert Humperdinck. O'Sullivan got out of one management deal by crying on the doorstep in front of the manager's mother at midnight. "She got hold of her son and said, 'You give him back his contract this minute! How could you do that?' " Thus when Gordon Mills eventually asked him, "Are you signed to anyone?", O'Sullivan could say he wasn't. Charismatic, powerful and inordinately rich, Mills put O'Sullivan in a bungalow on his Surrey estate and left him there to write. Mills liked the songs, which he thought had some of the traditional values of Broadway about them, and he even thought the image might work - though, Sullivan says, Mills didn't really like it, but simply felt it was different enough to stand out. And here began the relationship which would land O'Sullivan firstly in the charts and later in the courts.

"I used to babysit for him. [Hence the song "Clair".] I would go on holiday with them. I could walk in and out of the house as I pleased. I ended up being like a son. I thought it was good to be part of the family. I was so wary of the types of people who worked in the music business, the sharks.Psychologically, this was the perfect set-up for me. I felt that there was no way I would ever be conned in this scenario. It's kind of ironic, when you look back ..."

Mills's other charges, Humperdinck and Jones, were occasionally on the scene. "But they were grown men - they had their cars and wives and lifestyles - and I was a boy. I'd say hello and then shy away. Eventually, I would go and potter round the kitchen with the children."

It began to go wrong with a disastrous and ultimately aborted tour of America in the mid-Seventies, when Mills booked O'Sullivan into giant arenas which he couldn't fill. (O'Sullivan did at least meet his future wife, Aase, on this trip. They were married in 1980.) Soon after, O'Sullivan, admiring Rod Stewart's success with Tom Dowd on the Atlantic Crossing album, began wanting to bring in other producers on his own material: Mills wanted to carry on producing the records himself. Tension grew until again O'Sullivan displayed the unlikely steel at his core. He might have been a mild-mannered, home-loving babysitter, but he was clearly tough as a bullet when it came to his music and getting his own way. After a long period of feeling, as O'Sullivan puts it, "comfortably numb ... getting ever closer to the edge and summoning the courage", he finally decided to walk out on the deal. Mills's business partners promptly informed O'Sullivan that they were keeping his songs and O'Sullivan, for the first time in his career, talked to a lawyer.

On 8 June 1979, O'Sullivan filed a suit against Mills. It took five years for the case to come to court. "I'm sitting there with all these high- powered lawyers. They were all saying, 'Let's hope it's not a Welsh judge, because Gordon Mills is Welsh.' This seemed amazing to me. And then, of course, we got a Welsh judge and I was convinced it was over."

Far from it. It was calculated that, of the pounds 14.5m generated by O'Sullivan's records through the Seventies, he had received only pounds 400,000. Awarded back-dated royalties and the mastertapes of his songs, O'Sullivan removed himself to Jersey. (Gordon Mills died the following year.)

O'Sullivan was better off than he had ever been - and less successful. Weaker souls might have hit the bottle at this point. But O'Sullivan has never been much of a drinker. He carried on writing and recording and sought deals where he could, adamant it would start to come right again. He re-emerged in 1988 with the German-only release, "Frobisher Drive", named after his street in Swindon. "Why Germany?" I ask him. He says, "It was the only place that would have me." Sadly, the label he was signed to was bought out by Warner's, which promptly purged the roster and hosed out O'Sullivan. He now records with the backing of a Japanese label which is happy to return to him, after three years, all the rights in his material, an arrangement to which, as he points out, no big-time label would dream of agreeing, but which consoles him hugely, given what he has been through.

Jersey, he says, "is a good place to raise kids [O'Sullivan has two daughters, Tara and Marie, 10 and 14]. Plus there's the tax. It means you don't have to buy forests. You just pay your 20 per cent. And you can get Radio1 there. I couldn't live without Radio 1. They condemned me to oblivion, but they're what I grew up with."

He writes songs for nine months of the year in the music room in his house. In direct (and very O'Sullivan-esque) contradiction to the traditionally exorbitant rock'n'roll way of things, he had a wall put in to make the room smaller - 10 by 8 feet - like the rooms he has always written in. There's less of the "Ooo-Wacka-Doo"s these days and more of the trenchant stuff, like "Dishonourable Profession", a song about the music industry. "I'm very ambitious," he says, over the jam and cream debris of his tea, "despite the opposition that's up against me, despite the rejection I get constantly. I do hit my fist against the wall occasionally in frustration. But then I have a healthy arrogance. I always knew I was a good songwriter."