The one million people working in US security

Out of America: The horrors of 9/11 and fear of a reprise have led to the 'mining' of phone records and emails

Barack Obama with head of US intelligence James Clapper (AFP)

I worry about myself. I find myself quite un-appalled by the news that in the fight against terrorism, the United States government has been secretly collecting unimaginable quantities of phone and internet data, much of it on its own citizens.

Make no mistake, the commotion here has been deafening. The first disclosure, that the National Security Agency (NSA) had been vacuuming phone records of millions of customers of the Verizon telecom giant was bad enough – not least because one assumes the same goes for the other big phone companies. But then came the revelation that the NSA and the FBI had access to the traffic of nine of the largest US internet companies, under a programme called Prism, first authorised under George W Bush, but continued – possibly expanded – by Obama.

This was followed by a rare public appearance by James Clapper, grand supremo of US intelligence. He denounced the leaks as "reprehensible" and playing into the hands of America's enemies. But he confirmed the existence of both programmes and provided details of their legal grounding. He insisted that the internet trawling by the NSA was aimed at foreigners abroad, although one wonders. For me, however, the shock was minimal – and I can't believe I was alone in assuming that this sort of thing has been going on for ages.

The proximate reason has been the explosive growth of the US security state. America's view of the world was transformed by 9/11. How was anyone to know if other horrific plots were being hatched? With the 2001 Patriot Act, under which the data trawling is permitted, a panicked Congress effectively wrote the Bush adminstration a blank cheque. The tools were waiting: the most sophisticated IT industry on earth, and the NSA, the secret and incredibly sophisticated electronic-eavesdropping and code-breaking agency also known as "No Such Agency".

You just had to read a terrific 2010 Washington Post series on the US secret state to grasp what was happening: a system all but out of control, hundreds of companies and new government agencies, employing well nigh a million people (almost double the population of Washington DC), with top-secret security clearances, spending billions and billions of dollars to keep the country safe. By and large, they have. Until the Boston Marathon bombings this April, there hadn't been a terror attack of note in the US since 9/11.

Obama, a scholar in constitutional law, came to power in 2009 having vowed to roll back his predecessor's encroachments on civil liberties. Instead, he's kept in place most of the Bush apparatus – torture and the infamous CIA black camps being the exceptions – and in areas like the use of armed drones and perhaps data mining, appears to have extended it. And no one should be surprised.

Existing practice is hard to change: witness Obama's sincere but futile efforts to close Guantanamo Bay. And once it is granted new powers, any government, even a democratic one, is loath to give them up.

Despite the outcry from civil libertarians, however, there is little doubt the Obama administration has been acting legally, within the powers accorded it not just under the Patriot Act, but by laws going back to 1978. The problem is that those powers are loosely defined, while the work of both the special court that authorises the eavesdropping and wiretapping, and the congressional committee that monitors it, are shrouded in secrecy. Mr Clapper has come at least part way clean.

But if the burgeoning US security state is the immediate reason for Prism, the Verizon data mining and the rest, the great enabler of course has been the internet – whose biggest players, Google, Apple, Microsoft, Facebook and so on, also happen to be American. And like everything else, the internet and the IT revolution are not unmitigated blessings.

They have brought instant, wondrous, mind-enlarging knowledge. They keep you in touch with loved ones anywhere on earth. But the internet has also brought child porn, lethal bomb-making advice, and cyberbullying. As for the privacy now supposedly so violated by government – what about the self-inflicted violations of privacy that occur on Facebook and other social media, not to mention the daily invasions of privacy conducted by companies out to learn our shopping habits?

For me, government snooping comes very low on that list of evils. The same goes for the ubiquitous use of closed-circuit television in Britain. Do no wrong, and no problem – and who knows, if you get mugged CCTV might identify your assailant. It might even save your life.

None of this means there won't be consequences for Obama. He has support from many Democrats as well as Republicans and there's no evidence of specific cases of abuse. But his liberal base is disappointed, to put it mildly. He's on the defensive, distracted when he wants to focus on matters like immigration reform and seeing Obamacare into effect.

And even before last week, he was embroiled in a couple of "Big Brother" controversies, the US tax office targeting conservative political groups and the seizure of journalists' phone logs in pursuit of a leak. He's going to have to explain himself. On Friday he made a start, insisting that in combatting terrorism, trade-offs were unavoidable: "You can't have 100 per cent security and also have 100 per cent privacy and zero inconvenience." But almost certainly, he'll have to do more.

Meanwhile, in the age of the internet and America's security para-state, all privacy is relative. Once upon a time, people read books and wrote letters, and spies talked to each other on park benches. Maybe such ancient habits weren't so bad after all.