A controversial push by European Union lawmakers to legally require messaging platforms to scan residents’ private communications for child sexual abuse material (CSAM) may lead to tens of millions of false positives per day, a whole bunch of security and privacy experts warned in an open letter Thursday.
Concern over the EU proposal has been constructing for the reason that Commission proposed the CSAM-scanning plan two years ago — with independent experts, lawmakers across the European Parliament and even the bloc’s own Data Protection Supervisor amongst those sounding the alarm.
The EU proposal wouldn’t only require messaging platforms that receive a CSAM detection order to scan for known CSAM; they might also must use unspecified detection scanning technologies to try to select up unknown CSAM and discover grooming activity because it’s going down — resulting in accusations of lawmakers indulging in magical thinking-levels of technosolutionism.
Critics argue the proposal asks the technologically unimaginable and won’t achieve the stated aim of protecting children from abuse. As an alternative, they are saying, it would wreak havoc on Web security and web users’ privacy by forcing platforms to deploy blanket surveillance of all their users in deploying dangerous, unproven technologies, corresponding to client-side scanning.
Experts say there isn’t a technology able to achieving what the law demands without causing way more harm than good. Yet the EU is ploughing on regardless.
The most recent open letter addresses amendments to the draft CSAM-scanning regulation recently proposed by the European Council which the signatories argue fail to handle fundamental flaws with the plan.
Signatories to the letter — numbering 270 on the time of writing — include a whole bunch of academics, including well-known security experts corresponding to professor Bruce Schneier of Harvard Kennedy School and Dr. Matthew D. Green of Johns Hopkins University, together with a handful of researchers working for tech firms corresponding to IBM, Intel and Microsoft.
An earlier open letter (last July), signed by 465 academics, warned the detection technologies the laws proposal hinges on forcing platforms to adopt are “deeply flawed and vulnerable to attacks”, and would result in a big weakening of the vital protections provided by end-to-end encrypted (E2EE) communications.
Little traction for counter-proposals
Last fall, MEPs within the European Parliament united to keep off with a substantially revised approach — which might limit scanning to individuals and groups who’re already suspected of kid sexual abuse; limit it to known and unknown CSAM, removing the requirement to scan for grooming; and take away any risks to E2EE by limiting it to platforms that usually are not end-to-end-encrypted. However the European Council, the opposite co-legislative body involved in EU lawmaking, has yet to take a position on the matter, and where it lands will influence the ultimate shape of the law.
The most recent amendment on the table was put out by the Belgian Council presidency in March, which is leading discussions on behalf of representatives of EU Member States’ governments. But within the open letter the experts warn this proposal still fails to tackle fundamental flaws baked into the Commission approach, arguing that the revisions still create “unprecedented capabilities for surveillance and control of Web users” and would “undermine… a secure digital future for our society and may have enormous consequences for democratic processes in Europe and beyond.”
Tweaks up for discussion within the amended Council proposal include a suggestion that detection orders could be more targeted by applying risk categorization and risk mitigation measures; and cybersecurity and encryption could be protected by ensuring platforms usually are not obliged to create access to decrypted data and by having detection technologies vetted. However the 270 experts suggest this amounts to fiddling around the sides of a security and privacy disaster.
From a “technical standpoint, to be effective, this recent proposal may also completely undermine communications and systems security”, they warn. While counting on “flawed detection technology” to find out cases of interest to ensure that more targeted detection orders to be sent won’t reduce the chance of the law ushering in a dystopian era of “massive surveillance” of web users’ messages, of their evaluation.
The letter also tackles a proposal by the Council to limit the chance of false positives by defining a “person of interest” as a user who has already shared CSAM or attempted to groom a baby — which it’s envisaged can be done via an automatic assessment; corresponding to waiting for 1 hit for known CSAM or 2 for unknown CSAM/grooming before the user is officially detected as a suspect and reported to the EU Centre, which might handle CSAM reports.
Billions of users, tens of millions of false positives
The experts warn this approach remains to be more likely to result in vast numbers of false alarms.
“The variety of false positives attributable to detection errors is very unlikely to be significantly reduced unless the variety of repetitions is so large that the detection stops being effective. Given the big amount of messages sent in these platforms (within the order of billions), one can expect a really great amount of false alarms (within the order of tens of millions),” they write, stating that the platforms more likely to find yourself slapped with a detection order can have tens of millions and even billions of users, corresponding to Meta-owned WhatsApp.
“On condition that there has not been any public information on the performance of the detectors that may very well be utilized in practice, allow us to imagine we’d have a detector for CSAM and grooming, as stated within the proposal, with only a 0.1% False Positive rate (i.e., one in a thousand times, it incorrectly classifies non-CSAM as CSAM), which is far lower than any currently known detector.
“On condition that WhatsApp users send 140 billion messages per day, even when just one in hundred can be a message tested by such detectors, there can be 1.4 million false positives each day. To get the false positives right down to the a whole bunch, statistically one would must discover at the very least 5 repetitions using different, statistically independent images or detectors. And this is just for WhatsApp — if we consider other messaging platforms, including email, the variety of crucial repetitions would grow significantly to the purpose of not effectively reducing the CSAM sharing capabilities.”
One other Council proposal to limit detection orders to messaging apps deemed “high-risk” is a useless revision, within the signatories’ view, as they argue it’ll likely still “indiscriminately affect an enormous number of individuals”. Here they indicate that only standard features, corresponding to image sharing and text chat, are required for the exchange of CSAM — features which are widely supported by many service providers, meaning a high risk categorization will “undoubtedly impact many services.”
In addition they indicate that adoption of E2EE is increasing, which they suggest will increase the likelihood of services that roll it out being categorized as high risk. “This number may further increase with the interoperability requirements introduced by the Digital Markets Act that can end in messages flowing between low-risk and high-risk services. In consequence, just about all services may very well be classified as high risk,” they argue. (NB: Message interoperability is a core plank of the EU’s DMA.)
A backdoor for the backdoor
As for safeguarding encryption, the letter reiterates the message that security and privacy experts have been repeatedly yelling at lawmakers for years now: “Detection in end-to-end encrypted services by definition undermines encryption protection.”
“The brand new proposal has as one among its goals to ‘protect cyber security and encrypted data, while keeping services using end-to-end encryption inside the scope of detection orders’. As we have now explained before, that is an oxymoron,” they emphasize. “The protection given by end-to-end encryption implies that nobody aside from the intended recipient of a communication should find a way to learn any information in regards to the content of such communication. Enabling detection capabilities, whether for encrypted data or for data before it’s encrypted, violates the very definition of confidentiality provided by end-to-end encryption.”
In recent weeks police chiefs across Europe have penned their very own joint statement — raising concerns in regards to the expansion of E2EE and calling for platforms to design their security systems in corresponding to way that they will still discover criminality and send reports on message content to law enforcement.
The intervention is widely seen as an try to put pressure on lawmakers to pass laws just like the CSAM-scanning regulation.
Police chiefs deny they’re calling for encryption to be backdoored but they haven’t explained exactly which technical solutions they do want platforms to adopt to enable the searched for “lawful access”. Squaring that circle puts a really wonky-shaped ball back in lawmakers’ court.
If the EU continues down the present road — so assuming the Council fails to alter course, as MEPs have urged it to — the results will likely be “catastrophic”, the letter’s signatories go on to warn. “It sets a precedent for filtering the Web, and prevents people from using a few of the few tools available to guard their right to a non-public life within the digital space; it would have a chilling effect, particularly to teenagers who heavily depend on online services for his or her interactions. It should change how digital services are used around the globe and is more likely to negatively affect democracies across the globe.”
An EU source near the Council was unable to supply insight on current discussions between Member States but noted there’s a working party meeting on May 8 where they confirmed the proposal for a regulation to combat child sexual abuse will likely be discussed.
