Door still open as a 'massive talent' departs to the back benches

(EPA)

Ed Miliband held out the prospect of a return to frontline politics for his older brother, David, last night, admitting that the quality of Labour's top team had been hit hard by the departure of a "massive talent" to the back benches.

After days of agonising, David Miliband concluded that his presence in the Shadow Cabinet would provide too great a distraction – and a potential source of discord – for the party and backtracked on a pledge to serve under his brother if defeated in the leadership contest.

The fallout from the dramatic battle between the Milibands for the Labour crown, which Ed won by the slimmest of margins, dominated the final full day of the party's conference in Manchester. The younger brother now faces the challenge of rebuilding relations with Blairites, who are aghast that their favourite son has been forced to quit the Labour front bench.

In an early move to heal the wounds, Ed Miliband yesterday forced Nick Brown, a long-standing ally and confidant of Gordon Brown, to step down as Chief Whip.

In a letter to his constituency party in South Shields, David Miliband said he would not stand for election to his brother's Shadow Cabinet, but confirmed he would remain as an MP. He made his announcement as nominations closed for the Shadow Cabinet.

The final blow to his political ambitions came when he was caught on camera denouncing his brother's criticism of the Iraq war, forcing him to conclude that he would be a permanent distraction if he remained in the public eye.

Ed Miliband praised him for reaching a "typically thoughtful, mature and courageous decision" and signalled heavily he wanted him to return to frontline politics. "I am very clear that, as leader of this party, my door is always open for him to serve in the future, either in opposition or back in government," he said. "He is a massive talent for our party and indeed for our country, and I'm certainly not going to hide the fact that he's my brother and I have the highest regard for him, and I think we would have been a stronger team with him in it."

David Miliband, who had been backed in the leadership contest by the majority of the Shadow Cabinet, had been urged to remain in the top team by close allies. But the divisions between the brothers over strategy for tackling the deficit made it unrealistic for him to serve as shadow Chancellor. The recriminations over the Iraq war also meant it would have been hugely difficult to continue as shadow Foreign Secretary.

David Miliband said in his letter that he believed Labour would be more able to make "a fresh start" if he stood down. "This is now Ed's party to lead and he needs to be able to do so as free as possible from distraction," he wrote. "Any new leader needs time and space to set his or her own direction, priorities and policies. I believe this will be harder if there is constant comparison with my comments and position as a member of the Shadow Cabinet.

"This is because of the simple fact that Ed is my brother who has just defeated me for the leadership. I genuinely fear perpetual, distracting and destructive attempts to find division where there is none and splits where they don't exist, all to the detriment of the party's cause." He said he wanted to "recharge my political and intellectual batteries" after 16 years in senior political positions.