The social media network X has sparked significant anger after individuals utilized its AI chatbot Grok to alter photos of females by taking off their garments.
Multiple instances have surfaced where the technology has been used to digitally undress female subjects to make them appear in swimwear non-consensually, as well as to put them in inappropriate situations.
Developer Response and Legal Position
XAI, the company responsible for Grok, offered no comment to requests for comment, except for an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that legislation is being proposed to ban nudification software. Under a new law, anyone who supplied such tools would "face a prison sentence and heavy fines".
"X or Grok could prevent these forms of abuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "assess the risk" of people seeing illegal content on their networks, but did not confirm whether it was currently examining X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost AI assistant—with some additional features—which answers X users' prompts when they reference it in a tweet.
It is frequently used to offer commentary or more context to other posters' comments, but people on X are also can edit an uploaded picture through its artificial intelligence picture modification tool.
History of Criticism and Rules
The chatbot has been faulted for allowing users to create images and footage with explicit content and sexually suggestive content, and it was in the past accused of generating inappropriate material of public figures.
"X has been permitting the production and sharing of these images for a long time without taking any steps and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "portraying images of people in a graphic way".
In a comment, Ofcom said it was against the law to "make or circulate non-consensual intimate images or child sexual abuse material" and stated this applied to sexual deepfakes created with AI.
It said platforms such as X were required to take "appropriate steps" to "minimize the risk" of users viewing unlawful material on their services and to delete it quickly when they learn of it.
