George Osborne: Under Brown's skin

He enjoys goading the PM into personal attacks. But, on the eve of party conference, the shadow Chancellor warns Conservatives against complacency. Dominic Lawson meets... George Osborne

(Andy Paradise)

George Osborne's Commons office is a shrine to the seismic political events of a year ago. There is a bottle of beer and a range of coasters stamped with the logo "Bottler Brown", an unmistakable reference to the Prime Minister's decision, following the last Conservative Party conference, not to call a general election. Above the shadow Chancellor's desk, there are famed newspaper front pages mocking Labour's panicked decision to match Osborne's conference commitment to raise the inheritance tax threshold. In fact, the only memento missing from the display is Gordon Brown's scalp.

It's worth remembering that on the eve of the last Conservative conference, the settled view of the political and media classes was that theTories were about to be buried once and for all: even The Spectator's front page had a cartoon of a trussed-up Cameron, under the headline "Now get out of this, Dave". Dave did – largely thanks to his friend George's startling pledge to lift all estates below £1m out of inheritance tax. I invite Osborne to relive that moment.

"Whatever I end up doing in politics, I'm never going to forget last year's conference, when our backs were to the wall and there was a real test of whether we would fall apart under massive political and media pressure. And I think that when people judge politicians, they're not just judging their policies, they're judging their character, and we can point to the way we held ourselves together, came out fighting and changed the story of British politics last autumn. But a year on, we're not rubbing our hands, thinking we've got it sorted. We are more than aware of the electoral mountain we've got to climb to overturn a very large Labour majority and form our own overall majority, which is an even bigger task ... David Cameron and I are the least complacent people you will find in British politics at the moment."

I'm still fascinated to know if Osborne really believed his inheritance tax pledge was a sure-fire winner – or was it just a desperate gamble?

"I'm not going to claim that I knew it would have quite the impact it did – but I was fairly sure it would resonate not just in the hall, but across the country. I had done a little bit of private canvassing. I had spoken to our candidate fighting the Lancashire seat of South Ribble – someone called Lorraine Fullbrook. It's not one of the richest constituencies, by the way, and it's one we've got to win. I asked Lorraine what was the one thing I could do that would really help in South Ribble, and she said, do something on inheritance tax, because there are so many people in my constituency whose houses have risen in value over the past 10 years and who have aspirations to do something for their children, and fear that they will be caught by inheritance tax."

So Lorraine Fullbrook, it turns out, is the unsung hero of the Conservative Party – and Gordon Brown's nemesis. But the clever twist that sealed the deal was the idea of funding the commitment with an entirely new tax on the so-called "non-domiciles" – aka foreign fat cats in the City – as the shadow Chancellor is at pains to tell me.

"I know the attention was on the inheritance tax cut, but we also took the difficult decision which was that we said we are going to ask the non-domiciles in this country to pay something, and many were not paying anything. So who was it in British politics who called time on the non-doms? Not a Labour Chancellor. Not Gordon Brown. But actually the Conservatives – which I think gives the lie to that belief that somehow the Conservatives won't do anything to offend their friends in the City."

Indeed, some commentators on the right argue that George Osborne hasn't got enough contacts with the big players in the financial world, that he's a lightweight whose skill is merely his ability in astute political positioning – hardly appropriate in the current fraught economic circumstances. This, in essence, was the argument put last week by the Daily Mail's highly respected City editor, Alex Brummer.

"Alex was arguing, as far as I could tell, that we aren't talking to enough people, and we're not getting the best advice. Well, I have spoken in the past week to pretty much the chief executive of every major British bank. As to whether we're up to the task, I think people have to judge us on the decisions we've already taken. We came under pressure from all sorts of people, including perhaps Alex Brummer, to commit the Conservative Party to immediate income tax cuts if we were to win the general election. But we resisted that pressure, and that has been entirely vindicated. I would now be in an extremely difficult position as shadow Chancellor, given the deteriorating public finances, if I had committed us to instant income tax cuts."

It is, in fact, a compliment to George Osborne's current status that Gordon Brown spent much more time attacking him than he did David Cameron in his speech at the Labour Party conference – and the Prime Minister took special delight in a claim that Osborne had said of the City's excoriated short- sellers that "it's a function of financial markets that people make loads of money out of the misery of others".

Osborne is seething about this. "It's a complete lie that I opposed the Financial Services Authority's actions against short-sellers. Here you have Gordon Brown, the Prime Minister of this country, misquoting me – and that's a polite way of putting it. What I actually said was that 'nobody takes pleasure from people making money out of the misery of others', but that short-selling is part of the normal functioning of capital markets – and, by the way, Alistair Darling says exactly the same thing.

"What the FSA has stopped is a particularly aggressive, unfunded form of short-selling. I notice that Gordon Brown has taken the credit for this. He said: 'I stopped the short-selling.' The chief executive of the FSA has now issued a statement saying that 'at no point did any civil servant or government minister contact me in advance of the changes'. There is an important principle here. The FSA is an independent regulator. It does not take a call from the Prime Minister and act on his instruction. I know that Gordon Brown seems to have forgotten these niceties, but other people regard them as quite important."

It's fair to say, however, that over the three years of his shadow chancellorship George Osborne has provoked Gordon Brown beyond endurance. At one point he seemed to imply, facetiously, that Brown was on the autistic spectrum. At another he declared the then Chancellor Brown seemed to be "dealing with depression". Is this really the sort of personal attack that politicians should be engaging in?

"It's a question of whether someone has the right character to be Prime Minister, and I've never thought he has. By the way, I've now been joined by half the Labour Party in saying this. There are things I've heard cabinet ministers say ... that are several degrees beyond what I've ever said publicly or even privately about Gordon Brown. He was not a pleasant person when I was shadowing him. I have a much more civil relationship with Alistair Darling, for example, just as I've always had with other Labour ministers I've shadowed. So I think, in the end with Gordon Brown, he reaps what he sows. The thing about being prime minister or the leader of a party is that, in the end, even through the distorted lens of the political stage and the media, the truth outs and people get a sense of who you are."

But you actually enjoy getting under Gordon Brown's skin, don't you?

"Well, it does seem to be the case that I have. When I was watching his conference speech and he said: 'We did fix the roof when the sun was shining', I thought: 'Bullseye!'"

It seems clear that Brown's attitude to Osborne – and, indeed, Cameron – is also freighted with social antagonism. I suggested to Osborne that his background – he is the heir to a baronetcy as well as the Osborne & Little wallpaper business, and is a former member of Oxford University's somewhat decadent Bullingdon Club – seems to gnaw at Brown.

"Isn't that pathetic? Isn't that totally pathetic? The man is the Prime Minister, for Christ's sake. He shouldn't let himself get riled like that. And if it's a question of background, I merely point out that I went to the same school – St Paul's – as Harriet Harman, Labour's Deputy Leader. She's the niece of an earl, by the way. And his Chancellor went to Loretto, which is a private day school. So should he be laying into his Deputy and his Chancellor?"

Nevertheless, will the general public feel that a Chancellor George Osborne understands their pain in a recession, sheltered as he is with inherited wealth? Does he see that as a problem?

"Labour tried this class-war rhetoric in the Crewe by-election. It was tested to destruction. They dressed people up in top hats and tails – it was like a parody of Labour in 1970s – and it fell on the deaf ears of the British public: they were humiliated. And I'll let you into a little secret: Gordon Brown is not going to be hard up for the rest of his life. Alistair Darling is not going to be hard up for the rest of his life. They are not going to be on the breadline, even if we manage to kick them out of their day jobs."

If that does happen, will George Osborne wonder if it could have been him as Prime Minister, rather than David Cameron? After all, Michael Howard as Tory leader made little secret of the fact that he wanted Osborne to follow him, which is why he appointed him at the age of just 31 to the job of shadow Chancellor.

"Not really. I had to make a judgement. When confronted with the question [after Howard's resignation], did I want to run for the leadership, I looked at my friend David Cameron and he was so clearly in a frame of mind where he was up for it, and he had the self-confidence that it was the right time for him. And I thought: I'm nowhere in that space.

"In fact, talking to him made it clear to me that if I was even thinking of running for the Conservative Party leadership I should be thinking and acting like him. And I wasn't. Besides, for me to have been Chancellor of the Exchequer would be more than I could have hoped for when I first thought about going into politics."

As Osborne points out, half the Labour Party seem to think that the only way of stopping David Cameron becoming Prime Minister at the next election is if they ditch Gordon Brown. So does he lie awake at night worrying that Labour might actually summon the will to do that?

"No, I don't. I know there is a consensus in British politics that we want Gordon Brown to stay where he is, and it's certainly the case that we relish the chance of confronting him at a general election. However, if Labour change their leader they are entering the unknown – they would be the first party in peacetime to have three prime ministers in a single parliament. So they are in an extremely difficult place: stick with someone who is clearly unable to lead them properly, or plunge into the unknown and a further round of division. Thankfully, I'm a Conservative, so I don't have to make that choice."