Amol Rajan: The PM has a problem with female voters – and politicians

Twelve years, one month and two days after the infamous photocall of Blair's Babes, Gordon's Girls are deserting him. Now, there is talk instead of the WAGs (Women Against Gordon).

In 24 hours the Prime Minister lost the Home Secretary, Jacqui Smith, and the Communities Secretary, Hazel Blears. On their own, the departures need not have irreparably damaged Mr Brown's reputation for keeping women in senior government positions.

But alongside the departure of the Children's minister, Beverley Hughes (standing down as an MP at the general election), the former health secretary Patricia Hewitt (also leaving Parliament), and Ruth Kelly, who quit the Cabinet in a bizarre announcement during the Labour conference, they amount to a considerable weakening of female influence in Labour's operation.

Given that focus groups consistently tell Mr Brown he has a problem with female voters – that he fails to connect with them – he will have to shore up his female reserves in the reshuffle.

When Tony Blair left the leadership of the Labour Party, he had eight female cabinet ministers. Mr Brown cut that to five full-time and four whose attendance at the Cabinet was optional.

He may have felt he did not need as many women as Mr Blair, given Harriet Harman's surprise win in the race for Labour's deputy leadership. Only Ms Harman, the Chief Secretary to the Treasury, Yvette Cooper and Baroness Royall, the leader of the House of Lords, remain full-time in the Cabinet.

The spate of departures this week will focus attention on which women might fill the hole at the heart of government.

The stamina of Tessa Jowell, the Olympics minister and loyal Blairite, has never been questioned and her solidity in the Olympics brief could earn her another senior position.

The former foreign secretary Margaret Beckett is also respected for her enduring influence and is unlikely if asked to decline an invitation to the top table. As Housing minister, she only attends the Cabinet occasionally.

If the Schools Secretary, Ed Balls, is made Chancellor, Yvette Cooper, his wife and Chief Secretary to the Treasure, will have to move because of a potential conflict of interest. That could provide an opening for Baroness Vadera, the Business minister, who is considered an intellectual heavyweight and survived the controversy over her premature comments about seeing "the green shoots of recovery".

The Public Health minister Dawn Primarolo is known to be close to Mr Brown and may be promoted. But the chances of the junior Work and Pensions minister Kitty Usher could suffer because of the revelation that she spent £20,000 refurbishing her home.