Browse the models and styles behind stories like this one — free account, instant gallery.
Explore the catalog
Seven out of the top nine image-editing models hosted on Hugging Face will generate nonconsensual nude imagery on request, according to a new report from European nonprofit AI Forensics — a finding that puts the platform's open-model policies under direct scrutiny.
Open-model platforms face growing pressure to moderate image-generation tools used for nonconsensual deepfakes.
Image: The Verge / The Verge AI
The AI Forensics report, covered by The Verge, focused specifically on the most-downloaded image-editing models on Hugging Face — not obscure, buried repositories, but the top of the charts. Researchers found these models would comply with prompts designed to strip clothing from photos of real people, including minors. The platform's response, per the report, has been minimal.
Hugging Face occupies a unique position in the AI ecosystem: it's simultaneously the world's largest open-weight model hub and a critical piece of infrastructure for independent creators who want to run models locally or fine-tune them for specific tasks. That dual role makes content moderation genuinely complicated. Removing a model that's been downloaded hundreds of thousands of times disrupts legitimate use cases alongside harmful ones — but leaving it up means the platform is, in practice, distributing a tool for abuse.
Closed platforms — Midjourney, Adobe Firefly, OpenAI's image tools — enforce content policies at the inference layer. Every generation request passes through a filter. Open-weight models hosted on Hugging Face don't work that way. Once a model is downloaded, the operator controls what it will and won't do. The platform's only real leverage is whether to host the model at all.
That's the gap the AI Forensics report is pointing at. Hugging Face does have content policies that prohibit models designed to generate child sexual abuse material and nonconsensual intimate imagery, but enforcement appears inconsistent. The nonprofit found that the problematic models weren't hidden — they ranked at the top of search results by download count.
For creators who rely on Hugging Face to access diffusion models and fine-tunes, this matters practically. Regulatory backlash against the platform could accelerate restrictions on what models can be hosted publicly, which in turn narrows the pool of open-weight options available for legitimate creative work. The EU's AI Act and various national-level deepfake laws are already moving in this direction; a high-profile report like this one tends to accelerate that timeline.
This isn't the first time Hugging Face has faced criticism over hosted content — earlier controversies touched on model security and unauthorized access — but the AI Forensics findings are more pointed because they document a specific, repeatable harm rather than a theoretical risk.
The practical consequence for creators is worth watching closely. If Hugging Face responds with broader automated filtering or stricter upload requirements, that friction will apply to every model on the platform, not just the harmful ones. Creators who currently pull image-generation models directly from the hub for local workflows could find fewer options available, or face new verification requirements to access certain model classes.
The report also lands at a moment when the open-weight community is already navigating industry pressure around model restrictions. Adding nonconsensual deepfake concerns to that debate gives regulators a concrete harm to point to — which is a different kind of pressure than abstract safety arguments.
