Andrew Grice: Cameron should resist the siren calls urging him to the right

Inside Westminster

David Cameron is being pulled in different directions by his party (Reuters)

When will the real David Cameron stand up?

When he speaks to the Conservative Party's annual conference on 5 October, it seems. In the run-up, he is being pulled in different directions by groups inside his party urging him to use his address to tilt the balance in their favour.

Cameroons want him to reassert his modernising credentials, convinced that the Tories did not win an overall majority because his rebranding of the party did not go far enough. They remind Mr Cameron that too many public sector workers and people from ethnic minorities judged his party the "same old Tories". They warn him not to give ammunition to Labour's campaign, revealed here, to portray him as a Thatcherite wolf in sheep's clothing.

However, many Tory MPs are prodding the Prime Minister in the opposite direction. They welcome his response to the riots, a tough law and order line with a moral edge about "broken society" (although yesterday he nodded to the modernisers by prescribing "tough love" for problem families). These MPs include many of the large class of 2010 who entered the Commons last year; they feel less constrained by party whips. We will hear a lot from them in the coming weeks. They think too many concessions are being made to the Lib Dems and want a more traditional, Thatcherite approach – not least on Europe, which could become a headache for Mr Cameron.

Yesterday he styled himself a "common sense Conservative", a slice of motherhood and apple pie that will not offend the voters.

It was a holding operation for the modernisers and traditionalists who wonder whether he is "one of us". Some close allies insist Mr Cameron is a bit of both. He had a traditional, Conservative upbringing but sees himself as a One Nation Tory.

Labour's plans to portray Mr Cameron as a man of the right are no doubt partly aimed at spooking Tory modernisers. It's an old trick. One of Alastair Campbell's greatest hits was to accuse the Tories of "lurching to the right", a claim which conveniently boosted Labour's attempt to dominate the centre ground.

When Parliament resumes from its summer break on Monday, the Prime Minister faces tricky decisions which are likely to have an important bearing on whether voters consider him a moderniser or traditional right-winger. The future structure of the banks is at the top of his in-tray. Mr Cameron is sympathetic to their pleas to delay for several years the building of firewalls between their high street and riskier investment arms. He may be even more open than the Chancellor George Osborne to the banks' argument that an early shake-up could weaken our already frail economic growth.

"Remember that Cameron is the son of a stockbroker," one insider reminded me. Hardly the image modernisers want to project when the bankers are still public enemy No 1.

A similar danger looms over the 50p top rate of tax on earnings over £150,000 a year. Mr Osborne wants to cut it to 45p; he believes it is raising less revenue than Labour expected and deters entrepreneurs by making the UK more highly taxed than rivals such as France and Germany. The Prime Minister's instincts on the 50p rate are probably the same as his Chancellor's. But cutting the tax bills of those at the top would send a terrible signal to millions of voters struggling to cope with squeezed living standards. No doubt Mr Osborne would sugar the pill, perhaps by taking more people at the bottom out of tax , the Liberal Democrats' key goal.

Nick Clegg's party is also pressing for new taxes on wealth such as property which can't be moved offshore. The Lib Dems believe their long-standing calls for a crackdown on the rich are finally in vogue, with higher taxes for those at the top in France, Spain and the US.

Mr Osborne acknowledged the new mood by upping the rhetoric against tax evaders who salt away their money offshore, branding them "leeches on society". But words will not make abolition of the 50p rate easier to sell. If the public conclude that the Conservatives are looking after their old friends by letting the banks off the hook and cutting the taxes of the rich, Mr Cameron will reinforce Labour's right-wing caricature.

As he makes difficult calls on the banks and tax, he should remember why he heads a coalition rather than a majority government. And think about the poster Labour would love to run at the 2015 election: "Same old Tories."