Create your free Charmloop account — no credit card, no limits on browsing. Start making AI art in minutes.
San Francisco's city attorney has ordered Apple and Google to remove AI nudify apps from their respective app stores, arguing that both companies have profited — likely by millions of dollars in fees — from tools designed to generate non-consensual intimate imagery.\n\n## Key takeaways\n\n- San Francisco's city attorney issued formal orders to Apple and Google demanding the removal of AI nudify apps from the App Store and Google Play.\n- Official estimates suggest both platforms collectively earned millions in fees from these apps.\n- The action targets the platforms as commercial beneficiaries, not just the app developers themselves.\n- The move is the most direct government pressure yet on major app stores to police AI image-generation tools that produce non-consensual intimate imagery.\n- Neither Apple nor Google has publicly confirmed compliance as of the order's release.\n\n## The Platform-as-Profiteer Argument\n\nAccording to Ars Technica, the city attorney's office framed the case squarely around money: Apple and Google don't just host these apps passively — they collect a standard 15–30% cut of in-app purchases and subscription fees. When nudify apps monetize through subscriptions, that revenue flows directly to the platforms. The legal theory is that accepting that cut makes both companies active participants in the harm, not neutral conduits.\n\nThat framing matters for the broader AI-art ecosystem. Regulators are increasingly treating the infrastructure layer — app stores, API providers, cloud hosts — as legally accountable for what runs on top of it. The same logic that applies to nudify apps could, in future actions, extend to any image-generation tool that regulators decide produces harmful content at scale.\n\n## What "Nudify" Apps Actually Do — and Why They're Different\n\nNudify apps are a specific category of AI image tool: they take a clothed photo of a real person and generate a synthetic nude version without that person's knowledge or consent. That distinguishes them sharply from general-purpose AI image generators, where a user constructs an image from a text prompt or uploads reference imagery they control. The non-consensual use of a real person's likeness is the core legal and ethical problem, and it's what separates this category from the broader creative tools that AI-art creators use daily on platforms like Charmloop's generator.\n\nThe distinction is worth keeping clear: this action is not a broadside against AI image generation as a practice. It targets a narrow, consent-violating use case. But the regulatory momentum it builds is real, and any platform that hosts image-generation tools is now watching how Apple and Google respond.\n\n## App Store Policy Has Always Been the Weak Link\n\nBoth Apple and Google have existing policies that nominally prohibit apps generating non-consensual intimate imagery. The city attorney's action is essentially an accusation that those policies were not enforced — that the apps slipped through review, stayed live long enough to generate substantial subscription revenue, and were removed (if at all) only under external pressure.\n\nFor AI creators, this enforcement gap has a practical consequence: the app stores are an unreliable distribution channel for any AI image tool that operates near content-policy edges. Developers building legitimate creative tools have long complained that Apple's review process is inconsistent and opaque. This case adds political pressure that could push Apple and Google toward blanket restrictions on AI image generation apps — the kind of overcorrection that catches legitimate tools in the same net as harmful ones.\n\n## What Comes Next\n\nSan Francisco's city attorney does not have federal jurisdiction, so the order's legal teeth depend on California law and the willingness of Apple and Google to comply rather than litigate. Neither company has publicly confirmed it will remove the flagged apps. If they resist, the case could set a precedent — or expose the limits of municipal authority over global app distribution platforms.\n\nFor creators building workflows around AI image tools, the more immediate signal is legislative: several states and the federal government are advancing bills that would explicitly criminalize non-consensual AI-generated intimate imagery. Those laws, unlike a city attorney's letter, would carry enforceable penalties and could reshape what image-generation models are permitted to do — and what safety filters providers are legally required to build in. Staying current on that regulatory arc is as important as tracking model releases. Our earlier coverage of xAI's lawsuit against Grok users for CSAM generation shows how quickly the legal landscape around AI-generated harmful imagery is moving.
