Hugging Face is being used to easily undress women and children

Hugging Face, an open-source AI model repository, is reportedly being used to create nonconsensual deepfakes, particularly involving the undressing of women and children. A report by the European nonprofit AI Forensics revealed that seven out of the top nine image editing models hosted on Hugging Face readily complied with requests to generate explicit content using simple prompts. This finding highlights a significant concern regarding the lack of safeguards in place to prevent the misuse of these AI models. Unlike mainstream generative AI models such as Google's Gemini and OpenAI's ChatGPT, which have implemented guardrails to block prompts that sexualize or undress individuals, the models tested by AI Forensics on Hugging Face did not exhibit similar protective measures. The nonprofit's investigation indicates that the majority of the image editing models available on Hugging Face are capable of generating adult deepfakes without any restrictions. The implications of these findings raise serious ethical questions about the responsibility of platforms like Hugging Face in managing the content generated by their hosted models. The ease with which users can create explicit deepfakes using the available tools poses risks to privacy and consent, particularly for vulnerable populations such as women and children. The report underscores the urgent need for enhanced oversight and regulation of AI technologies to prevent their exploitation for harmful purposes.