IN THE SPRING thaw of 1992 a KGB archivist called Vasili Mitrokhin walked into the British embassy in Riga. Stashed at the bottom of his bag, beneath some sausages, were copies of Soviet intelligence files that he had smuggled out of Russia. Before the year was out MI6, Britain’s foreign-intelligence service, had spirited away Mitrokhin, his family and six large cases packed with KGB records which he had kept hidden in a milk churn and some old trunks under the floor of his dacha.
The pages of “The Mitrokhin Archive”, eventually published in 1999, are steeped in vodka and betrayal. They tell the stories of notorious spies like Kim Philby, a British intelligence officer who defected to Russia in 1963. And they exposed agents like Melita Norwood, who had quietly worked for the KGB for 40 years from her home in south-east London, then shot to fame as a great-granny. Her unrelenting Marxist refusal to shop at Britain’s capitalist supermarkets earned her the headline: “The Spy Who Came in from the Co-op”.
Mitrokhin’s record for the largest-ever haul of intelligence was smashed in 2013 when an American contractor, Edward Snowden, fled from Hawaii to Hong Kong with a secret archive of his own that contained more than 1.5m classified files from America’s National Security Agency (NSA). Mr Snowden uncovered programmes with names like DISHFIRE and OPTIC NERVE under which the NSA and its British counterpart, GCHQ, were alleged to be monitoring phones and computers around the world. Mr Snowden’s accusation was not that foreign agents had infiltrated Western intelligence agencies but that Western agencies were spying on ordinary people, including their own citizens.
To look at Mitrokhin’s meticulous typed-up transcriptions side by side with Mr Snowden’s capacious pen-drives conveys a sense of how deeply and rapidly the business of intelligence has changed. Western intelligence agencies used to inhabit a parallel world where spy battled spy. Their trade was stealing or guarding secrets. Their masters were the men and women in government. Today the intelligence services are part of everyone’s world. Their main task has been to protect society from terrorists and criminals. They are increasingly held to account in the press, parliaments and courts. This special report is about their struggle in the past 15 years to come to terms with this transition. They are not done yet.
Who can spy on the spies?
The intelligence revolution is partly the result of new technology. As recently as 1999, on becoming director of the NSA, Michael Hayden asked to send an e-mail to all staff. He was told: “We can’t actually do that.” The organisation used computers to break codes rather than to surf the web as everyone else did. The NSA’s new facility in Bluffdale, Utah, the first of several, now stores exabytes of data drawn from everyday communications. At Britain’s GCHQ, most code-breaking was done on paper until well into the 1980s. Today, inside its doughnut-shaped building in Cheltenham, south-west England, the hum from banks of computers that stretch away into the half-light is drowned out by the roar of air-conditioning.
The revolution has brought spying closer to ordinary people. After the attacks on America on September 11th 2001, counter-terrorism and counter-insurgency became the focus for the American intelligence agencies, says John Parachini, who heads intelligence policy for RAND, a think-tank. Almost two-thirds of today’s intelligence personnel have been hired since 9/11. As the world has moved online, so the spooks have become involved in monitoring organised crime and paedophiles as well as terrorists. That might mean tracking a drugs syndicate from Latin America to Europe, or working out how criminal gangs launder their money, or following paedophiles on the web.
In Mitrokhin’s days spies sent coded messages using short-wave radios and dead letter boxes. Now the communications of the spooks’ new targets are mixed in with everyone else’s, shuttling between computers and smartphones that are identical to those on your desk and in your pocket. Counter-terrorism, in particular, is pre-emptive. Hence the security services have had to act as hunters of conspiracies rather than gatherers of evidence.
I don’t believe you any more
And the revolution is taking place amid growing popular suspicion of everyone in charge. The days are gone when the word of Congress, the home secretary and the odd judge commanded enough public confidence to see off the accusations of a private individual such as Mr Snowden. Belated official acknowledgment of secret programmes has often been met by public dismay, even after assurances that they have been properly overseen. “It is not enough for the authorities just to say ‘trust us’,” writes Paul Bernal, of Britain’s University of East Anglia. “The public needs to know.”
Privacy advocates complain that the spooks have unprecedented scope to pry into people’s lives. They warn of a burgeoning surveillance state. The spooks retort that, on the contrary, they cannot keep up with terrorists and criminals cloaked by encryption, the dark web and the fact that, as the world builds internet infrastructure, a smaller share of total traffic is routed through accessible Western networks.
At the heart of the debate lies a conflict. The goal of a modern intelligence service, in the formulation of Sir David Omand, a former British intelligence chief, is for citizens to trust the state to manage the threats to their everyday lives. To maintain public safety, the intelligence services must be able to employ secret sources and methods that inevitably involve intrusion. Yet to command that public trust, they must also be transparent and prepared to live by rules that protect individual privacy.
These contradictions cannot be wished away. Privacy is a precondition for intimacy, trust and individuality, says David Anderson, a senior lawyer asked by the British government to review intelligence legislation. It secures rights such as the freedom of assembly and fair trials. The knowledge that an all-seeing state is watching has a chilling effect even if you have done nothing wrong. Perhaps your words will be used against you later, under laws passed by a different government. Perhaps the state will try to crush the dissent that prefigures desirable social change—as America’s FBI tried to destroy Martin Luther King by sending a letter, supposedly from a disillusioned admirer, that accused him of being a “colossal fraud and an evil, vicious one at that”.
But privacy is not an unalloyed good. A society that gives it primacy over security invites paralysing disorder and injustice that would inhibit the very intimacy and freedom of expression which privacy is supposed to promote.
Likewise, although the public needs to know what is being done in its name, some spying techniques lose their potency if they are discovered. Early Enigma decrypts in 1940 from Bletchley Park, Britain’s code-breaking centre, were given the “CX” prefix of MI6 reports so that the Nazis would think they were based on standard human intelligence (known in the jargon as HUMINT). A former CIA employee who is now at RAND tells how, after a successful raid in 1998, journalists learned that the NSA was intercepting calls from the satellite phone of Osama bin Laden, the founder of al-Qaeda. Immediately after the news got out, the phone fell silent.
Since the Snowden revelations, Western security services, and particularly those of America and Britain, have come in for savage criticism. Much of this has focused on the intense years immediately after 9/11. The CIA subjected prisoners to brutal interrogation techniques, including simulated drowning, or waterboarding. For some years the NSA operated a telephone-surveillance programme without judicial oversight. That programme was later judged to be illegal.
This special report will look at those transgressions in greater detail. Yet, even taking them into account, the criticism of American and British intelligence is overblown. Rather than being James Bonds, real-life intelligence officers are bureaucrats. Rather than acting as freewheeling individualists, most set out to live by the rules. It is possible to argue about the merits of intercepting and warehousing data, about access to databases and large-scale hacking, but the idea that controlling masterminds at the NSA and GCHQ are plotting mass surveillance is a myth.
Such criticism is especially unfair when it comes from outside the English-speaking intelligence alliance embracing America, Australia, Britain, Canada and New Zealand, known as the Five Eyes. Few countries say much about their intelligence services or have a clear framework for governing them. Britain’s operated almost entirely in the shadows until the 1990s and acknowledged some of its activities for the first time only in 2015. Yet by the end of this year it will have put its intelligence services under a system of oversight that is a model. And America is more open about its intelligence services than any other country.
The stark contrast is with countries like China and Russia, where the security services answer to nobody except the men at the very top. Russian and Chinese citizens are subject to untrammelled surveillance by their own leaders.
Before looking at Russia and China, and the growing awareness that they will become the Western agencies’ main antagonists again, start with the twin shocks of technology and terrorism. They have turned the world of intelligence on its head.