Berkeley Center for Law and Technology’s Seventh Annual Privacy Lecture was held on October 6, 2014, on Berkeley Law School’s campus. Moderated by Paul Schwartz, the presentation began with Ross Anderson presenting his recent paper, Privacy versus government surveillance – where network effects meet public choice. The panel continued with responses from Carl Shapiro, James Aquilina, and Anupam Chander before opening the floor to audience questions. This post summarizes each speaker’s presentation.
Ross Anderson: Network Effects and Government Surveillance
To lay the foundation for the discussion, Anderson first introduced two views of money and power. The “Bay Area view” is that money and power are all about network effects, the effect that one user of a good or service has on the value of that product to other people; these network effects help create a platform to which others then add value. Anderson contrasted this with the Washington D.C. view, where power is about having more “tanks and aircraft carriers, which is founded on taxation capacity,” and almost nobody speaks of network effects.
Network effects are characteristic of many IT product and service markets. Network effects tend to lead to dominant-firm markets where the winner takes all. Another common feature of IT product and service markets is high fixed costs and low marginal costs stemming from competition that drove down prices to marginal cost of production. This can make it hard to recover capital investment, unless stopped by patent, brand, or network effects. A third common feature of IT markets is that switching technology platforms is expensive, and companies get “locked-in.” Anderson mentioned the Shapiro-Varian observation: that the net present value of a software company is the total switching costs.
The combination of network effects, low marginal costs, and technical lock-in can make dominant-firm market structures very likely, and explain many security and privacy failures. First, market races lead to the Microsoft philosophy of “ship it Tuesday and get it right by version 3.” In a market race, companies open their systems to appeal to those who can complement it, such as app writers. Once the race is “won,” companies lock down the system in order to extract rents from its use. Thus, in many markets (Anderson gave the examples of mainframes, PCs, routers, phones, and social network systems), security is added after the system’s creation. The design is aligned with the platform’s interests at least as much as the users’.
On the other hand, privacy suffers from at least the same problems as security. For example, privacy suffers from asymmetric information, because users don’t know what gets done with their data. Hyperbolic discounting can also be an issue, as many users don’t consider or care about the long-term effects of disclosure. Researchers observe the paradox that people say they want privacy but act otherwise, as evidenced by the fact that most privacy startups fail. In a nod to Berkeley Law, Anderson mentioned that the first workshop regarding information security economics was held here in 2002; the industry has since grown to over 100 active researchers, who explore the models of what is likely to go wrong and attempt to measure it.
Anderson also introduced the concept of economics of surveillance, claiming that network effects were a driving force behind mass surveillance. In addition, the concentration of the industry into a few large service firms also made the concept of PRISM foreseeable, and the concentration of the telecoms industry into a handful of large operators similarly made TEMPORA foreseeable. (It was also described by several journalists in its earlier form of ‘Echelon’.)
Outside of the information security economics realm, network effects also matter in the defense/intelligence nexus, as, for example, neutral networks like India prefer to join the biggest network (read, the United States). However, network effects entangle us with “bad” states which use the same surveillance platforms, leading to problems such as the debate over exports to Syria. Thus, network effects present both political and civil rights problems, as they pull “good” and “bad” parties together. Compared to medieval warfare technology, which was all run on marginal costs, these days “to kill a foreign dictator you can use a single missile shot from a drone – because it’s backed by trillions of capital investment.” Thus, Anderson claims, “warfare has gone from labor to capital,” and has created complex technical “lock-in” games. Furthermore, each country within the “five eyes” can decide whether to minimize its citizens’ personal data, but only Canada has done so. (Anderson suspects this is because government forms are confidential once completed in Canada, unlike in the US or the UK.) Law enforcement network governance in particular comes in the form of various models from Interpol to mutual legal assistance treaties, and is slow and cautious.
Yet the question remains, is the world dealing with one network or many? Anderson argues that networks tend to merge (i.e. the Internet absorbs everything else). Anderson noted that intelligence resources are already used for rapid solution of exceptional crimes, and raised the examples of the NTAC and the Communications Data Bill in the UK, and PRISM in the US. And what will the day-to-day effect of this kind of world be? Anderson illustrated the impact of this consolidation as such: “is it okay, for example, if we move into a world where every inhabited space has cameras and mics, and the cameras and microphones are sharing information with the cloud, can we presume consent to information sharing? . . . Or does Mommy have to pick up the bear and hit the ‘I Consent’ button before the bear reads the bedtime story to the kids?”
Anderson concluded with examining long-term issues, with implications ranging from international relations to the separation of power and the rule of law. First, he raised that Britain provides access to 30% of the Internet; what effects might this have on the US? Also, if code is law, architecture is police: what are we embedding in the infrastructure and how will it affect our descendants? And above all, we need to solve the governance issue. The Bay Area v. D.C. gap is not just about whether Snowden’s a whistleblower or a traitor; the economic models are almost totally different. Yet, Anderson concluded, economics of security and privacy models pioneered at Berkeley a dozen years ago could apply here, too.
Carl Shapiro: Economics of Network Effects
In response, Carl Shapiro focused on the economic aspects of Anderson’s paper. Specifically, how do we translate the economics of innovation to government surveillance, the government sector, and international relations? This issue is of particular concern for those who work in both national economy and national security. The main issue Shapiro predicts was that we need to understand the networks that are in use today, and answer questions like “How do we think the network economics for the private sector are translated to the public sector, if at all?” and “What is the interface between law enforcement and surveillance operations?” In many ways, the U.S. leads the world with regards to its information capabilities.
Shapiro answered some of the questions with some observations of his own. First, Shapiro claimed that the economist will state that organizations will matter as well as incentives due to the cost of sharing information across organizational boundaries. Second, so far, these organizations have not been effective in terms of sharing their best practices to create consistency. Shapiro jokingly concluded by sharing an alternative title to Anderson’s paper: “Network effects meet public sector disorganization and dysfunction”
James Aquilina: Network Effects, Law Enforcement, and Civil Action
James Aquilina responded next, discussing the law enforcement and civil aspects of surveillance. He self identified as cynical, stating the way he thinks about information and user privacy is ultimately that it’s our choice: we could all choose not to connect to a wifi tower right after landing at the airport, or downloading apps that share our data. He noted that it seems as if people act with impatience and demand for the latest and greatest technology with a blind eye to what the price is to their personal privacy. He added that it is personally frustrating that people rarely discuss the risks associated with more restricted intelligence collection. (Aquilina called this negative impact on the communication between government and ISPs the “tragedy of the Snowden affair.”)
On the civil side, Aquilina noted that merging companies are moving so quickly that they are not thinking of security for what they are building or the impact on their consumers. Furthermore, the costs associated with a breach (whether competitive, advanced persistent threat, or insider threat) is “incredible.” Aquilina noted that the overlap of law enforcement and intelligence was growing by sharing a personal anecdote: when he had started at the US Attorney’s office, tips did not involve computer media; now seizure of cellphones, iPads, and computers used by suspects is normal. This has changed the way routine investigations are being conducted, but the law hasn’t actually kept up with this type of technology. Aquilina concluded by calling for a way to deal with digital evidence: “When you think of the effect of Snowden and that kind of revelation, what is unfortunate is that there is less focus on laws that proscribe criminal activity and an effort to bring them current and the way in which technology is now used as instrumentality of a crime.”
Anupam Chander: Global Due Process
The last presenter was Anupam Chander, who discussed the concept of Global Due Process. First, in response to Aquilina’s thoughts, Chander mentioned that he was not sure he wants cooperation between ISP and government to be without tension, and had concerns with government abuse of information. In particular, he felt like Mr. Anderson’s paper showed that a global information network can become a global spying network, with one problem being that there is no effective legal constraints on U.S. surveillance of non-U.S. persons abroad. In response to the lack of protection in U.S. law for those outside the U.S., foreign governments have begun trying to “unplug” from American internet. Chander noted that he was able to observe this effect in countries like Brazil, Germany, and Russia, who are trying to stop information from leaving the country instead of the usual tactic of preventing information from entering in the first place.
Chander identified a couple of potential solutions to this issue. First, the USA Freedom Act seeks to end some mass surveillance under Section 215 of the PATRIOT Act. It also institutes amicus curiae to FISA Court to “advocate, as appropriate, in support of legal interpretations that advance individual privacy and civil liberties.” Second, all countries should figure out a way to treat all information with global due process in a similar way to create consistency. Until these solutions can be implemented, Chander left the crowd with a self-help tip reminiscent of Aquilina’s comments: data encryption by the users is a strong way to protect your own data.