The social media network X has sparked significant outrage after people employed its artificial intelligence assistant Grok to manipulate pictures of women by removing their garments.
Several cases have emerged where the tool has been employed to virtually strip female subjects to make them appear in bikinis without permission, as well as to place them in inappropriate contexts.
Company Response and Regulatory Stance
XAI, the company that created Grok, offered no comment to inquiries, aside from an standard response stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to ban nudification tools. Under a proposed law, anyone who provided such tools would "receive a prison sentence and substantial penalties".
"X or Grok is capable of halting these forms of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said technology companies must "evaluate the potential" of people accessing unlawful material on their platforms, but did not specify whether it was currently investigating X or Grok in concerning AI-generated pictures.
Grok is a no-cost chatbot—with some additional features—which responds to X users' queries when they mention it in a tweet.
It is frequently used to provide reactions or more background to other posters' comments, but people on X are also can edit an posted photo through its AI image editing feature.
Background of Controversy and Rules
The tool has been faulted for enabling users to generate photos and videos with nudity and sexualised material, and it was in the past alleged to have been involved in making graphic content of celebrities.
"X has been enabling the generation and spread of these images for months without taking any action and we have not seen any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying images of individuals in a sexually explicit manner".
In a comment, Ofcom said it was illegal to "create or share deepfakes or illegal content" and clarified this included explicit AI images made with AI.
It said platforms such as X were obligated to take "appropriate measures" to "lessen the risk" of users viewing illegal content on their platforms and to delete it swiftly when they become aware of it.
