A Response to the Criticisms of Fool’s Gold: An Illustrated Critique of Differential Privacy
By Jane Bambauer and Krish Muralidhar
Two years ago, we coauthored an article that challenged the popular enthusiasm for Differential Privacy. Differential Privacy is a technique that permits researchers to query personal data without risking the privacy of the data subjects. It gained popularity in the computer science and public policy spheres by offering an alternative to the statistical disclosure control and anonymization techniques that had served as the primary mechanism for managing the tension between research utility and privacy.
The reputation of anonymization and “statistical disclosure control” methods is in a bedraggled state at the moment. Even though there is little evidence that reidentification attacks actually occur at any frequency in real life, demonstration attacks have captured the imagination of the press and of regulators. The founders of Differential Privacy helped erode confidence in SDC and anonymization so that Differential Privacy could shine by comparison. Differential Privacy was fundamentally different from what had come before because its standard guaranteed a certain level of privacy no matter how much special knowledge a data intruder had.
The problem is, Differential Privacy provides no assurance about the quality of the research results. As we showed in our paper, it destroys most of the research value of data. In order to salvage data utility, researchers in Differential Privacy have had to introduce relaxations to the privacy promises. But these relaxations have made Differential Privacy less “cryptographic” and more context-dependent, just like the methods of anonymization that the founders of Differential Privacy had rejected. In other words, Differential Privacy in its pure form is special but not useful, and in its modified form is useful but not special.
The Article concludes with a dilemma. On one hand, we praise some recent efforts to take what is good about differential privacy and modify what is unworkable until a more nuanced and messy—but ulitimately more useful—system of privacy practices are produced. On the other hand, after we deviate in important respects from the edicts of differential privacy, we end up with the same disclosure risk principles that the founders of differential privacy had insisted needed to be scrapped. In the end, differential privacy is a revolution that brought us more or less where we started.
Our article clearly hit a nerve. Cynthia Dwork refused to talk to me at a conference, and a few other computer scientists have written hostile critiques that aim primarily to impugn our intelligence and honesty rather than engaging with our arguments on the merits. Anand Sarwate calls our article “an exercise in careful misreading” and Frank McSherry writes
The authors could take a fucking stats class and stop intentionally misleading their readers.
The not-so-subtle subtext is “don’t listen to these idiots. They are bad people.”
Given this reaction, you would think that the critics have uncovered flaws in our applications and illustrations of Differential Privacy. They have not. Sarwate even admits that we “manage to explain the statistics fairly reasonably in the middle of the paper” and primarily takes issue with our tone and style.
I have little doubt that the condescension and character attacks are a symptom of something good: there has been a necessary adjustment in the public policy debates. Indeed, although our piece has received the occasional angry tweet or blog review, the private reaction has been positive. Emails and personal conversations have quietly confirmed that data managers understand the significant limitations of pure Differential Privacy and have had to stick with other forms of statistical disclosure controls that have fallen out of vogue.
We respond here to the criticisms, which come in four general types: (1) Differential Privacy should destroy utility—it’s working as planned; (2) We exaggerate the damage that DP does to utility; (3) We overlook the evolution in Differential Privacy that has relaxed the privacy standard to permit more data utility; and (4) There are methods other than adding Laplace noise that satisfy pure Differential Privacy. In brief, our responses are: (1) This is a disagreement about policy rather than a technical discrepancy; (2) Not correct, and when we take the suggestions offered by our critics, the noise gets worse; (3) Not correct; we spent an entire Part of our paper on deviations from pure Differential Privacy; and (4) Don’t hold your breath.