Anxious Tory backbenchers warn David Cameron: Downing Street drift must stop

Prime Minister, flailing for policies, is likely to appoint yet more special advisers

Long-time Cameron ally and freethinking strategist Steve Hilton. Now clearing desk before sabbatical in the US. Says he will be back but unlikely to return to the same job (PA)

Conservative MPs are clamouring for a shake-up of the Downing Street machine to enable David Cameron to "get a grip" after two months of mistakes and disastrous headlines.

The Prime Minister is likely to appoint more party advisers to the No 10 Policy Unit, after criticism that its civil servants lack political nous. But he is resisting the wholesale reorganisation demanded by his critics.

Two years ago to the day since the Coalition was formed, the mood inside Downing Street is said to be "anxious" and "rattled" after what has been dubbed the "omnishambles" since the March Budget.

As one insider put it: "We hoped it was a blip. The fear now is that it is turning into something much bigger." Another added: "We are stuck in a rut... There is a sense of drift."

One minister admitted: "No 10 is flailing around, looking at the opinion polls for initiatives that might get the Government back on track."

The Leveson Inquiry means the cloud is unlikely to be lifted soon. Cameron aides nervously await his appearance and that of Jeremy Hunt, the embattled Culture Secretary.

Personal rivalries may be fuelling the tension inside No 10. Craig Oliver, a former BBC executive who became No 10's director of communications after Andy Coulson's resignation, is said to have a tense relationship with Gabby Bertin, Mr Cameron's political spokesman since he ran for the Tory leadership in 2005.

Ms Bertin rolled her eyes when Mr Oliver sent out text messages to some journalists saying how well the Prime Minister had performed after his recent appearance on the BBC's Andrew Marr Show.

Critics complain about an absence of strong leadership and a sense of direction. There is talk of a bigger vacuum after the imminent departure of the strategy guru Steve Hilton, who is taking a year's sabbatical and will not be replaced. Andrew Cooper, founder of the Populus polling company, is called director of strategy, but insiders say the person really doing that job is George Osborne, dubbed "the part-time Chancellor" by Labour because he often attends the twice-daily strategy meetings of the Cameron inner circle.

Mr Cooper warned the Chancellor that his plan to cut the 50p top tax rate would undermine the Government's "we're all in it together" mantra.

Mr Osborne dismissed the advice, saying he knew what Mr Cooper thought. But Mr Cooper may have been proved right, especially as the Budget also included the "granny tax," a toxic mix.

Yesterday David Davies, the right-wing Tory MP for Monmouth, apologised to his constituents for "incompetence at the highest levels of government". He warned: "David Cameron needs to change his tack very rapidly, otherwise he's not going to be in position for very long."

Downing Street is accused of muzzling such critics. At a private meeting of Tory MPs on Wednesday evening, loyalists rounded on Nadine Dorries, the outspoken backbencher who branded Mr Cameron and Mr Osborne "arrogant posh boys". Mark Pritchard, secretary of the 1922 Committee of Tory MPs, said yesterday: "As we move into a new parliamentary session, No 10 appears to want to wound or neutralise all who might speak truth to power. This is dangerous and will ultimately be politically counter-productive."

Former No 10 staffers under Labour think Mr Cameron made a huge mistake by abolishing the strategy unit set up by Tony Blair to ensure a "strong centre".

Mr Cameron was influenced by a pledge he made in opposition to cut the number of party political staff – special advisers – which mushroomed under Labour and were associated with its "spin" culture.

It left him heavily dependent on neutral civil servants, who are less good than political animals at sniffing out policies that might cause controversy with the public, the Conservative Party or the Liberal Democrats.

Indeed, it was reported last night that Mr Cameron is planning to rank civil servants in order of ability and sack those who are underperforming, after growing concerns at the "useless" advice they have been providing ministers. "We need a much better early-warning system, to stop things before the damage is done," said one adviser.

Aides say Mr Cameron's instinct on taking power was not to run a "Stalinist regime" like Mr Blair and Mr Brown, with everything micromanaged from No 10. But a rethink is under way. And Mr Cameron's pledge has already been broken: the number of special advisers employed by his Government has risen from 66 in 2010 to 83, more than the 74 under Mr Brown.

The Prime Minister is loyal to his staff. Close allies believe the attacks on the Downing Street machine are part of a battle waged by Tory right-wingers who have never come to terms with sharing power with the Liberal Democrats and are demanding "real Tory" policies.

"The local elections hardly show the voters are calling for a lurch to the right," said one Cameron aide. "People need to realise this is what it is like two years into a government. We are in an economic s***storm and the cuts are starting to bite."

In and out: Insiders at No 10

Gone Steve Hilton – Long-time Cameron ally and free-thinking strategist. Now clearing desk before sabbatical in the US. Says he will be back but unlikely to return to the same job.

Gone Andy Coulson – resigned as News of the World editor after initial phone- hacking scandal. Appointed Tory communications director and then same job at No 10. Quit in January last year.

Gone James O'Shaughnessy – Tory adviser who quit as policy director last year. Seen as crucial link to Tory Party but was not replaced.

Staying Sir Jeremy Heywood – new Cabinet Secretary, still closely involved in running of No 10, where he was previously Permanent Secretary. Critics say he has too much power.

Staying Ed Llewellyn – chief of staff. Former aide to Paddy Ashdown and Chris Patten. A key link man with Liberal Democrats. Critics want tougher chief to bang heads together.

Staying Paul Kirby – Became head of No 10's civil service policy chief last year. Tory critics want more political input.