Full-packet-capture society - and how to avoid it

After my previous post on the need for intelligence reform, this post discusses a concrete policy recommendation - but before that, I will describe what is at stake.

I see our world on a trajectory into a dystopian future that is frightening and undesirable. Technological progress is deeply transforming our societies, and while most of it is for the better, we need to step back occasionally and look at the bigger picture.

Storage costs are on a steep downward curve, similar to CPU costs -­­ except that the end of Kryder's Law (the storage equivalent of Moore's law) is not in sight yet. Kryder's conservative forecast in 2009 estimated that a zetabyte of storage will cost about 2.8 billion USD by 2020. Extrapolating that prices will halve roughly every two years, this means that a zetabyte might be as cheap as 100 million USD sometime between 2030 and 2040.

All human speech ever spoken, sampled at 16 khz audio, is estimated to be roughly 42 zetabytes. This means that by the time I reach retirement age, storage systems that can keep a full audio transcript of everything humanity has said in the last 10 years will be within the reach of many larger nation­ states. Perhaps I will live long enough to get CD quality, too. Impressive, but also terrifying.

A future where every word ever spoken and every action ever taken is recorded somewhere will lead to a collapse of what we understand as freedom in society. There is good reason that both the east German StaSi and the KGB kept vast troves of "kompromat" on anyone that showed political ambitions - such data was useful to discredit people that were politically active or to blackmail them into cooperation.

The trouble with kompromat is, though, that nobody needs to actually use it, or threaten its use, for it to become an effective deterrent to political activity. We can see this in western societies already: It is not uncommon for qualified and capable individuals to decide against standing in elections for fear of having their lives examined under a microscope. When everything you have ever done has been recorded, are you sure that none of it could be used to make you look bad?

What about the famous "three felonies a day" that even well-­meaning and law­-abiding citizens run into?

Clapper's argument that "it isn't collection until you look at it" is disingenuous and dangerous. By this logic, vast files tracking people's lives in pedantic detail are not problematic until that data is retrieved from a filing cabinet and read by a human. Transporting his logic into East Germany of the early 80's, collecting excruciating detail about people's private lives was OK, it was only when the StaSi actively used this data that things went wrong.

The discussion whether phone metadata records should be held by the government or by private entities does not matter. Data should only be held for the period which is necessary to perform a task, and storing data in excess of this period without allowing people to view / edit / remove this data carries the implicit threat that this data may be used to harm you in the future. Involuntary mass retention of data is oppressive. And while checks and balances exist now, we cannot be sure how they hold up over time. Deleting the data is the only prudent choice.

Well-­intentioned people can build highly oppressive systems, and not realize what they are doing. Erich Mielke, who had built the most oppressive security agency in living memory in order to protect "his" country from external and internal foes, famously said "but I love all people" in front of East German Parliament. He did not grasp the extent of the evil he had constructed and presided over.

Nobody wants a full-­packet­-capture society. It is fundamentally at odds with freedom. Arbitrary collection and retention of data on people is a form of oppression.

Policy recommendation: A different form of SIGINT budget

How do we balance the need to protect our countries against terrorism and foreign aggression with the need for privacy and data deletion that is necessary to have functioning democracies and non-­oppressive societies?

This question has been much-discussed in recent months, culminating in a set of recommendations made by the panel of experts that the Obama administration had convened.

I agree with the principles set forth in the above­-mentioned document, and with several of the recommendations. On the other hand, I feel that some of the recommendations focus too narrowly on the "how" of collection, rather than policing the overall end goal of avoiding mass surveillance. Regulations that are overly specific are often a combination of cumbersome-in-practice and easily-side-stepped -- properties you do not want from a law.

My personal recommendation is a different form of SIGINT budget: Aside from the monetary budget that Congress allots to the different intelligence agencies, a surveillance budget would be allotted, of a form similar to:

For the fiscal year 2014, you are allowed to collect and retain data on [X number] citizens and [Y number] non-­citizens for a year. Restrictions on the purposes of collection and retention of data still apply.

This budget could be publicly debated ­ and would make sure that data collection is focused on the areas that truly matter, instead of rewarding SIGINT middle managers that try to improve their career trajectories by showing how much "more data" they can collect. The budget would be accounted for in "storage-hours" to create incentives for early deletion. People can get promoted by showing the ability to do the same work while retaining less data, or retaining the data for briefer periods.

This may look similar in practice to the way cloud providers (like Amazon) charge for storage. The agencies get to store and keep data, but they get charged internally for this, daily or weekly. Retain too much data and your collection system runs out of budget - but you can free up budget by deleting old data. The overall budget is public, so the public can have a clear view of how much data is collected under all programs, instead of the undignified spectacle of "we do not collect this data under this program" non-denials.

The big trouble with sniffing internet traffic is that it is fundamentally addictive. You can see the spiral of escalation in almost every criminal hacking career. It is easy to underestimate that the same addictive property of data collection applies to organisations. Middle managers can shine by showing growth in collection, upper management can speak of "total domain dominance" and similar powerful-­sounding words. Collection becomes an end by itself. By imposing hard limits on the number of people whose lives can be touched through surveillance, we make sure that our efforts are focused on the real problems -- and remain liberty­-preserving.

If, for whatever reason, a SIGINT agencies runs out of "surveillance budget" in a given fiscal year, they can always ask Congress to grant an "emergency loan" - provided, of course, that this remains an exception.

Public budgeting and proper accounting of retained data, implemented in modern democracies, would give citizens a clean and understandable method to evaluate and discuss the extent of governmental data collection for national security, without introducing detailed micro­-management-rules on the "how" of collection. It would provide a clear answer to "how much data are you actually keeping", and create strong incentives for early data deletion. It is not perfect, but it may be the cleanest way of achieving both the security and the privacy that a free society needs.



[Many people helped improve this article by proofreading it and offering helpful suggestions or interesting counterarguments. Aside from anonymous help, I have to thank Dave Aitel, Chris Eng, Vincenzo Iozzo, Felix Lindner, Window Snyder, Ralf-Philipp Weinmann and Peitr Zatko]