Nearly 600 students. Nearly 90 schools. Those numbers, published by WIRED and Indicator this week, represent the documented scope of AI-generated deepfake nude images targeting minors—a figure that almost certainly undercounts the true scale of a crisis unfolding in school corridors, group chats, and bedrooms worldwide.
That same week, a separate disclosure revealed that Apple had quietly threatened to remove Elon Musk's Grok app from its App Store in January. The target: Grok's failure to curb nonconsensual sexual deepfakes spreading across X. The method: a private ultimatum, documented in correspondence with US senators, demanding developers "create a plan to improve content moderation." The result: almost nothing public changed.
This is the policy vacuum in action. Apple possesses perhaps the most effective enforcement lever in consumer technology—the ability to shut off distribution to hundreds of millions of iPhone users. It used that lever. Once. Privately. Against one app, for one violation, with no public accountability and no follow-through announced. The deepfake crisis did not slow.
Compare this to what platform gatekeeping could look like. Apple's App Store guidelines already prohibit apps that facilitate "objectionable content," including nonconsensual intimate imagery. Enforcement could mean mandatory technical safeguards, watermarking requirements, or干脆清退令. Instead, the company's response to the most high-profile AI abuse scandal of the year was a behind-the-scenes letter and a waiting period.
The WIRED investigation underscores what that quiet approach costs. Researchers found the technology enabling rapid deepfake creation had proliferated faster than schools, platforms, or regulators could respond. Victims—disproportionately teenage girls—found themselves with little recourse beyond requesting individual image removal, a process that requires knowing where images exist in the first place.
Defenders of Apple's approach will note that platform enforcement is genuinely difficult. Determining what constitutes a deepfake, distinguishing satire from abuse, and scaling review across billions of pieces of content presents real technical and legal challenges. They may also argue that private pressure achieves results public shaming cannot, preserving relationships needed to negotiate lasting change.
These are not unreasonable points. They are also precisely the arguments that have allowed tech platforms to avoid accountability for fifteen years. The enforcement lever exists. The documented harm is staggering. The question—whether Apple, Google, and other gatekeepers will wield their gatekeeping authority with the urgency the crisis demands—remains unanswered, left to private negotiations while 600 documented students, and likely thousands more, wait for relief that hasn't arrived.