Joan Smith: Dave has fallen in with the wrong crowd

A rash promise made to secure the top job four years ago is coming back to haunt the Conservative leader

I doubt whether many people would hesitate if they were offered the choice of an evening with Edward McMillan-Scott, until last week a Conservative MEP, or the right-wing Polish politician Michal Kaminski. McMillan-Scott is by all accounts urbane, popular and committed to mainstream conservative ideas. Kaminski is a fan of General Pinochet, believes that gay men should be called "poofs" and has accused migrant workers from the East of taking Polish jobs.

Last week, David Cameron faced a much bigger choice than whom he'd like to have to dinner. After a phone call from the Polish president, Lech Kaczynski, who was apparently furious that the Tory MEP had defeated Kaminski (who used to be Kaczynski's spin-doctor) in an election for vice-president of the European Parliament, Cameron expelled McMillan-Scott from the Conservatives. Even more astonishingly, he forced the leader of his British MEPs, Timothy Kirkhope, to step aside so that Kaminski could become leader of the European Conservatives and Reformists, a new right-wing group at the European Parliament.

To say it was not a happy moment for British Conservatives is an under-statement. Cameron has worked hard to get rid of his party's "nasty" image, rebranding the Tories as caring, compassionate and supportive of a range of lifestyles; he's even apologised for Section 28, a notorious piece of anti-gay legislation and a totem of Conservative support for traditional family values. Poland's Law and Justice Party, of which Kaminski is a leading member, doesn't so much embody those values as offer a caricature of them: founded by the decidedly weird Kaczynski twins, it's homophobic, nationalist and fanatically opposed to abortion. Centre-right parties in Europe keep the twins at arm's length, preferring to talk to mainstream right-wingers such as the former Polish prime minister Jerzy Buzek, who has just been elected President of the European Parliament.

Last week's events in Strasbourg may seem arcane to anyone without a passionate interest in European politics, but they speak volumes about Cameron's political judgement – or lack of it. Centre-right parties such as Nicolas Sarkozy's UMP and Angela Merkel's Christian Democrats are aghast that Cameron has kept his promise to leave their group, the European People's Party, and sit with a bunch of people widely regarded as right-wing fruitcakes. This is EPP leader Wilfried Martens, a former Belgian prime minister, speaking last month: "Cameron's campaign has been to take his party back to the centre in every policy area with one major exception: Europe. I can't understand his tactics. Merkel and Sarkozy will never accept his Euroscepticism."

Cameron's Euroscepticism runs deeper than many realise, but that is only half the story. Last week's spectacular bust-up has its origins in a tense moment during the Tory leadership contest in 2005, when Liam Fox was about to withdraw as a candidate. David Davis, a former Europe minister, was not prepared to match Fox's show-stopping pledge to pull Tory MEPs out of the EPP, but Cameron was. It won him the leadership, but at a cost that has been fully revealed only in the past few days.

Cameron has been supersensitive about the subject for months. As of last week, his MEPs find themselves isolated from mainstream Conservatives in the European Parliament, sitting with fringe parties from Belgium, Latvia and Hungary, as well as Poland's Law and Justice Party. At a time when there is widespread dismay over the arrival of two BNP MEPs, it couldn't be further from the image many British Conservatives want to project. The party's pro-European business spokesman, the former chancellor Ken Clarke, recently said he'd been assured by his colleagues that "they're not going to sit with neo-fascists or cranks or anything of this kind".

Kaminski may not be a fascist but he has certainly admired people who were. Ten years ago, when Augusto Pinochet was languishing in a rented house in Surrey, awaiting extradition for torture and murder, Kaminski turned up to comfort his hero, presenting him with a Catholic medal as a mark of respect. (Baroness Thatcher also visited, but she at least had the excuse that Chile had provided crucial support to the UK during the Falklands war.) Challenged about the visit in the Polish parliament, he declared: "General Pinochet is not somebody we should be discussing, but somebody we should be emulating."

Presumably Kaminski wasn't thinking of the ex-dictator's imaginative ways of dealing with opponents, which tended to involve electrodes and helicopters, but his implacable opposition to abortion. One of Pinochet's final acts was to introduce a total ban on terminations, even if the mother's life was at stake; the law remains in force in Chile to this day. Kaminski's then party wanted an identical ban, reluctantly accepting a compromise that left Poland with one of Europe's most restrictive abortion laws.

Cameron's front bench might be able to live with these elements of Kaminski's past, but other unsavoury anecdotes have emerged in the past few days. On one occasion, he is said to have turned up at Warsaw Central Station and handed out leaflets to foreigners, accusing them of taking jobs from Polish workers. He criticised Aleksander Kwasniewski's presidential victory in 2000, claiming that the former communist had won by courting support from homosexual organisations; he even dismissed "gay" and "homosexual" on TV as euphemisms, arguing that gay people should be referred to as pedaly ("poofs") – pedaly is also street slang for paedophiles. He attacked Kwasniewski's apology for a wartime massacre of Jews in the north-eastern town of Jedwabne, opposing a commemoration and urging local people to reject claims that the killings were carried out by Poles.

In his youth, Kaminski belonged to a right-wing organisation called Polish National Rebirth (NOP), which McMillan-Scott denounced last week as homophobic, racist and anti-Semitic. It is linked to the European National Front, a very unpleasant federation of racist parties, although Kaminski insists the NOP moved to the right after he resigned from it in 1990. Some Polish commentators accept that his politics have changed, observing that his language is more extreme than his actions, but they remain mystified by Cameron's willingness to be linked with one of Poland's most controversial politicians.

The bigger problem for Cameron is that this loose cannon now has a front seat at the European Parliament and speaks on behalf of all 25 Tory MEPs. What he will say is anyone's guess, but Cameron's inability to get himself out of a rash promise does not suggest an agile political intelligence. One solution would have been to allow his MEPs to sit independently for a while and go about making his peace with Sarkozy and Merkel, but the Conservative front bench – including, don't forget, the profoundly Eurosceptic William Hague – has taken the risky option. It is a hugely significant decision: if the Tories return to power next year, the old squabbles between the UK and the rest of Europe may come back with a vengeance, along with unhealed wounds in their own ranks.