Apple is delaying its child safety features
Apple says it’s delaying the rollout of Child Sexual Abuse Material (CSAM) detection tools “to make improvements” following pushback from critics. The features include one that analyzes iCloud Photos for known CSAM, which has caused concern among privacy advocates.
“Last month we announced plans for features intended to help protect children from predators who use communication tools to recruit exploit them, limit the spread of Child Sexual Abuse Material,” Apple told 9to5Mac in a statement. “Based on feedback from customers, advocacy groups, researchers others, we have decided to take additional time over the coming months to collect input make improvements before releasing these critically important child safety features.”
Apple planned to roll out the CSAM detection systems as part of upcoming OS updates, namely iOS 15, iPadOS 15, watchOS 8 macOS Monterey. The company is expected to release those in the coming weeks. Apple didn’t go into detail about the improvements it might make. Engadget has contacted the company for comment.
The planned features included one for Messages, which would notify children their parents when Apple detects that sexually explicit photos were being shared in the app using on-device machine learning systems. Such images sent to children would be blurred include warnings. Siri the built-in search functions on iOS macOS will direct point users to appropriate resources when someone asks how to report CSAM or tries to carry out CSAM-related searches.
The iCloud Photos tool is perhaps the most controversial of the CSAM detection features Apple has announced. It plans to use an on-device system to match photos against a database of known CSAM image hashes (a kind of digital fingerprint for such images) maintained by the National Center for Missing Exploited Children other organizations. This analysis is supposed to take place before an image is uploaded to iCloud Photos. Were the system to detect CSAM human reviewers manually confirmed a match, Apple would disable the person’s account send a report to NCMEC.
Apple claimed the approach would provide “privacy benefits over existing techniques since Apple only learns about users’ photos if they have a collection of known CSAM in their iCloud Photos account.” However, privacy advocates have been up in arms about the planned move.
Two Princeton University researchers who say they built a similar system determined that the tech was “dangerous.” They wrote that, “Our system could be easily repurposed for surveillance censorship. The design wasn’t restricted to a specific category of content; a service could simply swap in any content-matching database, the person using that service would be none the wiser.”
All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.