Earl of Cardigan: 'Seeing police beat a pregnant woman does poison your view'

An early run-in with authority shaped him. Matthew Bell hears about his latest fight

Frozen out: The earl in the unheated room where his family portraits once hung (Tom Pilston)

The headlines were simple enough. "Earl of Cardigan living on benefits." The pictures showed a well-fed 60-year-old, in Gyles Brandreth V-neck and tie, outside a sprawling mansion. Why, we were invited to splutter, should an Old Etonian heir to a 4,500-acre estate – and father of singer Bo Bruce from The Voice – be eligible for jobseeker's allowance?

For one thing, the earl doesn't live in the big house. Never has. To find David Brudenell-Bruce, you drive a mile into the spooky Savernake Forest near Marlborough in Wiltshire, which has been owned by his family since 1067, then keep going past the beat-up Audi. This is the car his daughter smashed up two years ago, which she found "very therapeutic". His home is a converted stable block in peeling white paint. It actually feels colder inside than out, except next to the Aga, and there's no hot water. To wash, the earl visits the leisure centre in Marlborough.

News of his financial hardship has emerged during his bitter legal battle with the estate's trustees. When Cardigan went to America in 2006, he appointed two trustees to assume responsibility in his absence. On his return, he discovered that family portraits had been removed and sold, and his prized rose beds had been dug up. The trustees, he accused, had locked him out of control of his own estate. Even worse is that one, John Moore, had been a friend for over 20 years.

Moore and Cardigan were together in 1985 when they witnessed one of the worst acts of police violence, the Battle of the Beanfield near Stonehenge. A convoy of travellers was heading there from Savernake Forest when 1,000 officers rounded them up and arrested them. Journalists were blocked from entering the field, but Cardigan described seeing policemen clubbing a pregnant woman, and other appalling acts. His testimony was vital in undermining claims that the travellers had started the violence, and several newspapers attacked him for it. "I remember an editorial Bill Deedes wrote in The Telegraph accusing me of betraying my class. I sued and got a BMW off them."

Cardigan has never been able to forget the police's behaviour that day, and has resisted excessive authority ever since. Which is why he now vows to win back control of the estate. Cardigan says Moore's motive in turning against him is hard to fathom, and imagines it could be down to delusions of grandeur. "He has a brand new Land Rover and wears a green suit, wellies and a Barbour – everything a country squire must have," says Cardigan.

One of many run-ins with Moore happened when Cardigan took his American in-laws to see Tottenham House, the Grade-I listed house that stands empty at the heart of the estate. "He came out and said: 'I'm sorry, Lord Cardigan, this house is under my control, because I am the trustee. I am not going to allow you to come in.' He then turned round and walked into my front door, of my stately home, and shut the door."

Later, Cardigan shows me Tottenham House. It is an exquisite piece of Neoclassical architecture in an advanced state of decay. Water gushes through holes in the ceilings, and in the Grade II* stable block, whole floors have collapsed. Yet Cardigan is powerless to do anything. His family moved out after the war, and for years it was a prep school, Hawtrey's, which he attended. What was it like being at school in your own home? "The headmaster was terrified of appearing to give preferential treatment to the landlord's son, so he beat me extra to compensate."

David's life has been a catalogue of mishaps and misfortune. His wife of 30 years, whom he divorced in 2010, died of pancreatic cancer last year; he is estranged from his children; and a plan to rescue Tottenham House by turning it into a luxury hotel collapsed when the company involved went bust. Suffering from post-traumatic stress, he checked into an American clinic, where he met his current wife, Joanne.

Cardigan's father, the eighth Marquess of Ailesbury, passed the estate to David after he attended agricultural college. In 1980, he married Rosamund Jane, and they had two children. But in 2005, she asked for a divorce, and so began a spiral of disasters. He left for America, where his daughter was being treated in a drug rehab clinic. The stress of losing his wife, children and estate has left him clinically depressed. You could liken the Earl of Cardigan's story to a Shakespearean tragedy. He makes a more contemporary comparison. "You remember that wonderful Eddie Murphy film, Trading Places?" he says. "It's about two men who conduct an experiment: they take a man in the gutter, and a Wall Street trader, and they swap their places. And to do this they completely destroy the trader's life." He picks up a digestive biscuit and crushes it to dust. "Well, that's how I feel. I won't stop fighting until I get back what's rightfully mine."