Hal Roberts

I’ve been looking at deep packet inspection / targeted advertising company Phorm for the past couple of days and have found a clear and simple case of Phorm violating its own privacy policy in contradiction to Ernst & Young’s audit of the company’s systems.

Phorm has been energetically defending itself against complaints about the privacy risks of their systems. As part of its campaign to legitimize itself, Phorm prominently links to an audit completed by Ernst & Young at the end of last year. I eagerly followed the link the first time I saw it hoping for a report full of technical details about Phorm system, only to find that Ernst & Young’s statement within the audit consists of a single page says only that, in the opinion of Ernst & Young, Phorm follows its own privacy policy. No meaningful explanations of what tests they ran on the system. No technical information about the system at all. And certainly no discussion of whether the privacy policy addresses the larger privacy concerns of the community (as Phorm implies the audit does).

Even assuming that the scope of the audit is sufficient, what’s the point of producing such a document? I understand that the audit is produced and procured to reassure the large institutions (ISPs, government regulators, etc) with whom Phorm has to work, and that it has some weight for those actors. But it shouldn’t. Ernst & Young is paid (a presumably large amount of) money to produce this letter of reassurance by Phorm. It theoretically has to produce a thruthful report so that other such reports will be trusted by the audience of future customers (in other words, so that Ernst & Young can produce the same report for NebuAd and that report will reassure NebuAd’s institutional constituencies). But the report is completely opaque, so all we have to rely on is Ernst & Young’s reputation. For that reputation to be valid, though, there has to be a strong feedback mechanism that discredits Ernst & Young when it produces a faulty report. In practice, what’s that pushback? Is there any history of such audits being disproved to the disparagement of the auditing firm? In the face of only a vague threat of some sort reputation loss, the strong, direct incentive to produce positive reports to generate more business will win every time.

In fact, in a couple of hours of looking at the available technical information I found a significant breach of Phorm’s privacy policy missed by the audit: Phorm’s privacy policy claims that it will not disclose its Phorm IDs to any third parties, but a technical description of the system by Richard Clayton finds that Phorm does indeed share its IDs with web sites in a common usage scenario. Specifically, Phorm’s privacy policy claims that:

We will not disclose any randomly generated ID associated with a cookie to any third party, which means that none of this shared information can be used to identify individual users.

But in Richard Clayton’s excellent description of Phorm’s system, he finds that:

24. If, later on [after browsing from an Phorm enhanced ISP], the www.cnn.com website was to be visited via another ISP that was not using a Phorm system (or if subsequent accesses were made using the “https” protocol) then the cookie [with Phorm’s randomly generated id] would reach www.cnn.com.

25. Phorm believe that by placing their name (webwise) within the cookie they place within
the www.cnn.com domain, no clash – or other bad effects – can occur.

In other words, if you browse cnn.com from home, where your ISP is Phorm enhanced, and then later from Starbucks where it is not Phorm enhanced, cnn.com will be sent a cookie that includes your pseudonymous Phorm id. I assume that Phorm thinks this is not such an important data leak, since it considers its id to be completely anonymous. But that id serves a single, global, unique identifier for your web browsing session. It will, in other words, identify you as the same person to all web sites that you visited first at the Phorm enhanced ISP and then later from a non-Phorm ISP. Regardless of whether this data leak is significant privacy risk, Phorm’s privacy policy clear says that it will never pass its id on to any third party. The second item (#25 above) is critical because it verifies that Phorm knows about the data leak but is concerned only with not polluting the cookie namespace of the hosting site.

I credit Richard Clayton with finding and asking Phorm about this data leak (and especially with writing up his excellent report on the Phorm technology). But the policy violation is not an obscure border case. It will effect every Phorm tracked user who takes his laptop to Starbucks occasionally. I’m not the kind of uber-security-geek that Ernst & Young should be hiring for its audits, but this same question occurred to me while reading Clayton’s report — one of the core questions I was hoping the audit would help answer was what data Phorm was sending to the content publishers who host the Phorm served ads.

How did Ernst & Young not find this problem? I have a hard time accepting that an uber-security-geek would ever miss this sort of problem. Did they not test for this sort of vulnerability, concentrating instead on the process oriented AICPA privacy list? Did they just walk through the code verifying that it intends to do what the privacy policy says it does (yup, there is no ‘ip address’ field in the database. next question …)? Do they have a vulnerability attack process that encourages members of their team to break the audit system? We don’t know because they don’t tell us anything at all useful about how they conducted their audit.

For reference, here’s Ernst & Young’s entire contribution to the audit report:

We have examined Phorm, Inc.’s (“Phorm”) management assertion that during the period of June 1, 2007 through December 15, 2007 it:

• Maintained effective controls over the privacy of personal information collected in its Phorm Service (Service) to provide reasonable assurance that the personal information was collected, used, retained, and disclosed in conformity with its commitments in its privacy policy and with criteria set forth in Generally Accepted Privacy Principles, issued by the American Institute of Certified Public Accountants (AICPA) and the Canadian Institute of Chartered Accountants (CICA), and
• Complied with its commitments in the privacy policy.

This assertion is the responsibility of Phorm’s management. Our responsibility is to express an opinion based on our examination.

Our examination was conducted in accordance with attestation standards established by the AICPA and, accordingly, included (1) obtaining an understanding of Phorm’s controls over the privacy of personal information collected in the Service, (2) testing and evaluating the operating effectiveness of the controls, (3) testing compliance with Phorm’s commitments in its privacy policy, and (4) performing such other procedures as we considered necessary in the circumstances. We believe that our examination provides a reasonable basis for our opinion.

Because of inherent limitations in controls, error or fraud may occur and not be detected. Furthermore, the projection of any conclusions, based on our findings, to future periods is subject to the risk that the validity of such conclusions may be altered because of changes made to the Service or controls, the failure to make needed changes to the Service or controls, or a deterioration in the degree of effectiveness of the controls.

In our opinion, Phorm’s management assertion referred to above is fairly stated, in all material respects, in conformity with Phorm’s privacy policy and with the criteria set forth in the AICPA Generally Accepted Privacy Principles.

The rest of the audit consists of a couple of letters from the Phorm management, a copy of Phorm’s privacy policy, and a listing of the “AICPA Generally Accepted Privacy Principles.”