Oh dear. We've got a right one here

We ought to resist the temptation to feel sorry for politicians. They chose to go into the business. No one asked them to do it. Certainly no one made them do it. Mr Iain Duncan Smith could now be enjoying a modestly prosperous career in, say, middle management (the path recommended to the young Mr Michael Portillo by his old Cambridge tutor, Mr Maurice Cowling). But nevertheless I feel a twinge of, at least, sympathy for Mr Duncan Smith. The poor man always has a test of some kind before him.

We are all familiar with tests. Classical mythology is full of them. Modern education seems to consist in little else. It provides the nearest parallel to Mr Duncan Smith's present situation. In olden times, tests were something you either passed or failed, and that was that. Today they go on constantly. If it is not one thing, it is the other, Maths one week, English the next.

Likewise with Mr Duncan Smith. For the last two weeks he has been required to satisfy the examiners at Prime Minister's Questions. This he has done, more than adequately, I should say. This week he has another test before him: the reply to the Queen's Speech, the second most stringent test that any Leader of the Opposition has to undergo (the first being the response to the Budget, which for some reason is always made by the Leader of the Opposition rather than by the Shadow Chancellor).

Next year there are also various elections, Scottish, Welsh, local government. Mr Duncan Smith knows when these will come about. What he cannot know is the date of the next by-election and the nature of the terrain.

There is a silly school of political analysis which holds that by-elections "do not matter". If this means they can be a shaky guide to the forthcoming general election, that is true enough, even though they are not wholly without their uses in this respect. If, however, it means they do not affect the politics of the day, that is manifestly untrue. I could devote the rest of this column to providing examples, which would be tedious, so I shall confine myself to a few.

The Orpington by-election of 1962 disordered the Tory government of that time and led indirectly to the unnecessary (as things turned out) resignation of Harold Macmillan in the following year. The loss of Roxburgh to the Liberals in 1965 damaged Alec Home. The Carmarthen by-election of 1966 persuaded the Labour government to take not only Welsh nationalism seriously but the Scottish variety as well. And as Mr Michael Brown – a Tory MP at the time – correctly pointed out in The Independent last week, the loss of Eastbourne to a Liberal Democrat in 1990 was a crucial element in the fall of Margaret Thatcher: as important as the poll tax, and more so than Europe. That was the time when Mrs Thatcher announced at the party conference beforehand that the Lib Dem candidate was "a dead parrot" without having the faintest idea of what her scriptwriters were talking about, for she was not a follower of popular culture.

Mr William Hague was lucky with by-elections. Of the 17 that occurred when the Conservatives were in opposition, only two constituencies changed hands: one in South Antrim and the other in Romsey, where the Liberal Democrats gained the seat. The effect of this was more to consolidate – perhaps to save – Mr Charles Kennedy's leadership of his party than to imperil Mr Hague's, though naturally the Tories were not at all pleased. Mrs Thatcher had been even luckier in opposition. In the whole 1974-79 period the Conservatives sustained no losses and made six gains, while the Liberal Democrats made one gain – David Alton in Liverpool – at the expense of Labour. It will be surprising, on probabilities alone, if Mr Duncan Smith proves equally fortunate. Another Romsey could certainly activate those 25 signatures necessary to requisition a vote of confidence.

The loss of a by-election is largely out of Mr Duncan Smith's control. Not so the imposition of a three-line whip on a matter – adoption by unmarried partners – which should not have been made the subject of a whip of any sort in the first place. The ridiculous statement which followed the catastrophic vote, when Mr Portillo, Mr Kenneth Clarke and others voted against their party, was equally the responsibility of the leader. In the immediate post-war years someone said: "Of course, Herbert [or Herbert Morrison, in some versions Nye, or Aneurin Bevan] is his own worst enemy." To which Ernest Bevin replied: "Not while I'm alive he ain't."

Mr Duncan Smith really is his own worst enemy. Monday's division was clearly on a matter where a free vote should have been allowed. Members of Parliament have traditionally been permitted to possess consciences where drink, sex and Sunday observance are concerned. As society becomes more complicated, and government more intrusive, the role of the still, small voice will inevitably become enhanced, even if with some people, as C R Attlee once observed, it is in danger of turning into a megaphone. The statement that followed was even more extraordinary than its occasion. Part of it went:

"Over the last few weeks a small group of my parliamentary colleagues have decided consciously to undermine my leadership. For a few, last night's vote was not about adoption but an attempt to challenge my mandate to lead the party. We cannot go on in this fashion. We have to pull together or we will hang apart."

My late mother used to advise me never to attribute motives. But this is not just attributing motives. This is barking. As Enoch Powell once remarked of a short-lived Tory politician, John Davies: "We've got a right one here." If Mr Clarke and Mr Portillo did indeed conspire as Mr Duncan Smith charged, firmer action was clearly necessary. But the evidence is that they were making a genuine protest.

Mr Oliver Letwin appeared several times on radio and television to admit that he had been connected with both decisions – on the vote and on its sequel. Mr Letwin is an educated man, which makes him exceptional on the Tory side and, for that matter, on the Labour side as well. The trouble is that he has been educated out of his wits. He was partly responsible for the poll tax. Mr Letwin is also a natural moderniser. Mr Duncan Smith is not. He was not elected as one but as its opposite. That is why his present position is so false. Most MPs would rather have Mr Clarke, as they did last year. He would still win at Westminster, with or without Mr Portillo's support, with Mr David Davis second. Once the contest was thrown open to the party, however, Mr Davis would win. It is doubtful whether the rules allow the submission of only one candidate such as Mr Clarke. We may well be witnessing the strange death of Tory England.