'I haven't succeeded at love': A rare audience with rap legend P Diddy

Question: how much does a rap mogul pay for his cardigans? Answer: $2,500 (£1,600). I know this fact because Sean Combs, the hip-hop hyphenate variously known as Puff Daddy, Puffy, P Diddy, and more recently, plain old Diddy, has just instructed his wardrobe man, Dave, to lend me an item of knitwear to cope with the sub-zero temperatures in the Mojave Desert, where he's making the video of the track "Yesterday", from his new album Last Train to Paris. Its price tag flutters in the breeze. "Make sure you give it back," says Dave. "And try not to get it dirty."

Times are hard for many Americans, and here in rural California, where the unemployment rate sits at 10 per cent, $2,500 is a perfectly normal monthly wage. Is it right that a cardigan should cost the same? Maybe not. But His Diddyness clearly does not suffer from bourgeois guilt. Instead, he's a study in rapperly ostentation, decked out in a black leather overcoat, replete with fur trim, a thick gold bracelet, dotted with several dozen diamonds, two medallions, and earrings made from extravagant chunks of what is known in the trade as "ice".

The uniform makes a statement: here is a man of consequence. And Diddy certainly is such a man. At Bad Boy records, the firm he founded 18 years ago, he not only mentored a generation of talent – from Mary J Blige, and Usher to Lil'Kim – but also perfected "commercial" hip-hop, a genre which changed the face of popular music and is now beloved by suburban white kids all over the world. As a Barnum-esque showman, gifted salesman, and all-round celebrity impresario, he later leveraged that musical brand into a sprawling business empire.

In addition to records, Diddy sells clothes (under the label Sean John), fragrances ("Unforgivable" and "I Am King"), and a popular brand of vodka (Ciroc). He stars in movies, and reality TV shows, including I Want to Work for Diddy, an Apprentice knock-off in which he tests would-be employees by making them complete important tasks, like, say, sourcing him a slice of cheesecake, in Manhattan, at 2am. He's a successful singer, DJ, devoted father of six children (five biological), and an urban philanthropist. Forbes recently put his annual earnings at $30m (£19m), and estimated his net worth at 10 times that amount.

"What I sell, to be honest, I sell lifestyle. I sell entertainment. It all falls under one umbrella, one mindset," is how Diddy sums up his line of business. There are, however, many consumer products he wouldn't flog. "I'm not just like some green motherfucker that's just doing things to make money... Everything I sell is associated with part of my lifestyle." Take vodka: "I've never drunk beer, wine, never drunk whisky. Vodka is what I've drunk all my life. So my heart and soul is in it."

Today, his job involves standing in front of cameras, assuming the posture of a very pissed-off man, and mouthing the lyrics to "Yesterday", a break-up song as bleak as the Mojave. At the same time, he must guard against hypothermia and hope that a team of special-effects guys can get their smoke machine to work properly, in gale-force winds. Life isn't always a doozy, even when you're a member of rap's royalty.

Getting to Diddy has proved to be quite a job. In fact, it required the patience of a saint. Our first meeting, at the Beverly Hills Hotel before Christmas, was cancelled three hours after I'd showed up, since he was still in bed (though The Independent's photographer eventually got a short audience). A tentative "date" on New Year's Eve in Las Vegas came and went, without word from his office. A week later, a third meeting was called off, at 24 hours' notice, because according to an apologetic e-mail: "Diddy's schedule has inflated".

Eventually, though, I earn an audience. It happens on the shores of a remote lake, a six-hour round trip from Los Angeles. But at this late stage, you take what you're given. When I arrive on set, I am met by Capricorn, Bad Boy's formidable brand director, who assures me that the big man "knows that he owes you 45 minutes". I make the acquaintance of two PAs, who carry BlackBerries and voluminous handbags, and wave gingerly at his dozen-strong entourage, who seem far happier than protocol dictates, even if one of them seems to be employed purely to hold Diddy's supply of toothpicks.

Finally, Diddy appears. He wears fake gold teeth (a prop for the music video), a spotless goatee beard and a generally solemn demeanour. Out stretches his hand. "You can call me Sean," he says, which is nice, given that everyone else has been using "Mr Combs" or "Sir". Then he walks off, trailing a small crowd of staff in his wake, like a queen bee leading her worker drones. Twenty yards on, he stops, barks the word "Tissue!" (eliciting a Kleenex from PA number one's handbag), wipes his nose, and then strides purposefully off again.

Two hours of thumb-twiddling later, I am at last summoned to Sean's Winnebago, for a proper chat. The chairs are sumptuous, and the air heavy with the scent of Diptyque candles. An Apple computer resides on the coffee table, locked into his Twitter account (3,185,000 people follow @iamdiddy). "Sit there, where I can see you," he decrees. Then he turns off his television set, and fixes me with a pair of large, strangely melancholy eyes, and for a total of exactly 51 minutes, I am cocooned in white leather, enjoying his undivided attention.

We begin by talking about Last Train to Paris, Diddy's sixth studio album. It is credited to Diddy – Dirty Money, a trio Sean has formed with two female vocalists, Dawn Richard and Kalenna Harper, took three years to make and is touted as a "concept" album. In plain English, that means it consists of 18 songs, which in more-or-less chronological order tell the story of a man's ill-fated love affair with a woman he meets in a nightclub. The story is set against the backdrop of the French capital.

On paper, this pitch sounds a touch complex; pretentious, even. But in practice, Last Train to Paris holds together nicely. It's an eclectic record, referencing hip-hop, soul, dance, and Motown, and as befits Sean's lofty status in the world of talent brokerage, features 17 guest vocalists, including Justin Timberlake, Grace Jones and Chris Brown. Released in the US before Christmas, it enjoyed critical acclaim, reached number seven in the charts, and spawned at least one break-out hit, an anthemic track about redemption called Coming Home. The European launch happens this week.

The album's prevailing theme is love, which Sean describes as "something that I live for, and I need, but I haven't been able to be successful at". This seems an appropriate moment to talk about his own romantic life. Over the years, he has publicly fathered five children with three different women. His longest-standing girlfriend was Kim Porter, with whom he had a son and twin daughters before splitting in 2007. They remain friends. But many other lady-friends have been and gone.

Diddy is therefore now a 41-year-old bachelor. Though he looks younger, and naturally provides munificently for his kids, who live mostly in Florida, you sense that his failure to settle down preoccupies him. On a recent episode of The View, the US talk show, host Barbara Walters sent a frisson of unease through the room by asking, with regard to the mothers of his children: "Why didn't you marry any of them?".

I remind him of the incident. He looks a touch irritated. "I don't wish that I got married," he says. "In a perfect world, sure, things would have worked out that way. I would have married one of the women that I've had children with. But it didn't work out that way. Right now, I don't have a desire to get married, or a desire not to get married. So far, it just didn't happen for me. For some people it does happen. For others it just doesn't happen. It's something I leave in God's hands."

Looking back on Sean's career, you might say that the Lord has always provided, in the end. Born in New York, he was an altar boy during childhood, and attended a respectable Catholic school (today he's a non-denominational Christian). His father Melvin, who moved in edgy circles, was shot outside a nightclub during Sean's childhood, meaning he was raised on the straight and narrow (more or less) by his mother, Janice. After leaving school, he won a place at Howard University in Washington.

Though he had no formal training, and cannot read music, Combs always had an "ear" for a tune. "From the age of 12, I would be able to hear records on the radio, and the first time I would hear them, I would say 'I like that record'. Each time, it would go on to become a top 10 hit," he recalls. "The first time I heard a record, if I liked it a lot, I just knew it would be a hit. So I felt like I had an ear."

After scoring an internship at Uptown Records, an influential New York label, shortly after turning 21, Diddy got his big break in the industry by a happy accident. "One day, by chance, I got left in the studio when someone didn't show up, so I produced a record. And it wound up being for Jodeci." The record was a remix, Come and Talk To Me, which wound up selling over two million copies. In 1993, he left Uptown, set up Bad Boy, and started repeating the trick, over and over. Within four years, he was at the helm of one of the most successful record labels in urban music. Then disaster struck.

On 9 March 1997, Sean's closest friend, Biggie Smalls, a former crack dealer and Bad Boy artist, was shot and killed in Los Angeles. The murder, which has never been solved, was thought to be a revenge attack for the killing six months earlier of Tupac Shakur, an artist signed to Bad Boy's arch rivals, the LA label Death Row Records. The deaths marked the culmination of a long-running "beef" between the two East- and West-coast labels, confirming hip-hop's status as perhaps the most dangerous genre in the history of American music.

The loss hit Sean hard. His original success, he says, had been "the equivalent to hitting the lottery". But during his rise he'd been "corrupted" by sudden wealth, fame and status. Though he's always denied any responsibility for (or knowledge of) Shakur's killing, he admits that his behaviour in the run-up to Biggie's murder was less-than clever. "Money fucks you up. You lose yourself. But it was fun. There were negative things I was doing, things that made me big-headed. Then God took everything away from me. I lost my best friend."

Only last year could Sean finally lay Biggie's death to rest. He produced Notorious, an acclaimed biopic about his late friend. "It was helpful in closing that chapter in my life," he says, of the movie. "Losing somebody you love at such a young age, you lose a lot of your spirit. The movie helped me deal with it."

Yet for all the pain, Biggie's killing also marked a turning point. Shaken by the affair, or perhaps realising that greater rewards might lie from dragging hip-hop towards the mainstream, Sean began cleaning up his act. He altered his nickname, from Puff Daddy to the less threatening P Diddy, and later plain old Diddy (he claims, when I ask, to have only later discovered the comedian Ken Dodd, original creator of The Diddymen), and toned down the references to guns, drugs and "hos". Swearing was reduced; mainstream pop songs began being sampled on his artists' tracks. This was the start of "commercial hip-hop".

Ironically, Sean's greatest loss also inspired the track that brought him his greatest success. In May 1997, he blasted into the public consciousness with his first, and still most famous, major hit as a performing artist: a repurposing of the Police track I'll be Missing You, which paid tribute to Biggie. It became the first rap song to go straight to the top of the billboard charts, won a Grammy, and would eventually hit number one in 15 countries.

Riding on its coat-tails to respectability, Diddy then launched a fashion label, the first part of his business empire. "I have always been a big fan of Motown," is how he explains his decision to shift into entrepreneurship. "I was always asking: 'Why didn't Barry Gordy have his own Ed Sullivan show?', or 'Why didn't he do a clothing line for Diana Ross?'. I would still, to this day, buy a Diana Ross dress for my woman. That's where I started getting ideas from. And they just evolved."

Success has since brought its critics, of course. Rivals occasionally grouse that Sean takes credit for writing songs which were actually created in a collegiate environment, with a team of writers and session musicians. While his ability to remix well-known tracks is second to none, and his notorious perfectionism makes him one of the foremost producers in the business, he has detractors. Some say he lacks his own ideas, pegging him as a creative magpie; a gifted salesman, rather than a musical genius.

Once such critic has been 50 Cent, who told a recent interviewer that Diddy is "not an artist", since "an artist would be someone who actually wrote something on a record". He added: "What made hip-hop exciting to me was each person's experience, and how you learn from them ... That's not there if [someone's] just being an executive, like 'I don't write rhymes, I write cheques'."

When I read out the quote, Sean bristles. I ask if he takes criticism to heart. "No, no, no," he replies. "Look, I'm one of the greatest that ever did it. Know what I'm saying? The history is right there. You can just look at it and see who the leader is, by the way people walk, talk, act, dress. It's clear: I'm responsible for a whole generation of people that have empowered themselves, and become businessmen and women and artists."

He also bristles when, discussing his film career, I make the mistake of describing his role in Get Him to the Greek, when he plays a psychopathic music mogul called Sergio, as a "cameo". False, he says. "That's not true. I do not play me. Sergio is not the way I am, and that wasn't what I was doing. You haven't seen me act like that all day." But these days, Diddy tends to settle "beefs" quietly. I apologise, and am forgiven.

This lays bare, I think, a fairly mature sense of perspective. Diddy is growing up. These days, he finds it harder to play the role of hard-partying hip-hop mogul ("If you look at my last two or three years, that's not what I've been about") and has stopped visiting frenetic nightclubs, where, as he puts it, "everyone has ADD".

The recession has also tempered some of his bling-encrusted excess. Where Sean once employed a valet called Fonzworth Bentley, who resembled an Edwardian manservant, and once gave his son a $400,000 (£252,000) Maybach car for his 16th birthday, he now tells me: "We raise our kids that they have to work for everything. It's the way I was brought up." They will enjoy the high life, but learn to appreciate it, too.

"I can't be running around hiding who I am, and if I have been working for the last three-and-a-half years, neglecting my family, and I decide to take them out on a yacht for 10 days, I don't feel like I'm a bad person for doing that," he says. "I don't feel like I'm trying to flaunt my wealth in anybody's face, because that's not the intention. But I definitely think it's important, as I grow, for me to have balance."

Recently, he has admitted to suffering insomnia, saying that he no longer enjoys his ability to function on very few hours of sleep. "It was something that I used to be proud of. But it was a gift and a curse. Being in love with what I do so much, I'm always thinking about it. But after a while, it gets tired and boring, just working all the time. It's hard for me to calm down and go to sleep. I'm excited with what I do, but there's a whole other life, another world out there outside of work."

It's getting dark, and a knock on the door summons Diddy back to the desert, to carry on filming. As I return the cardigan to Dave, I wonder how, after all these years, Diddy still motivates himself. How, with $400m in the bank, and enough money to buy $2,500 items of knitwear, he can be bothered to step out into the freezing night, to posture for the cameras.

"To whom much is given, much is expected," he responds. "I have to step up to that responsibility. To be a shining example of a real person, striving to be the best I can be, the best person, and father." It's a solemn end to what has been a strangely serious conversation. In fact, only later do I realise that one thing has been missing from my encounter with Sean Combs: not once during our time together, or anytime during the day, did rap's greatest mogul actually manage to crack a smile.

'Last Train to Paris' is out this week