Elon Musk's online platform X has caused significant outrage after users used its AI chatbot Grok to change images of females by stripping their garments.
Several cases have surfaced where the tool has been used to remove clothing from women to show them in swimwear without consent, as well as to insert them in sexualized situations.
Company Response and Regulatory Stance
XAI, the firm that created Grok, did not respond to questions, except for an standard message stating "legacy media lies".
A government official stated that legislation is being brought forward to ban image stripping technology. Under a proposed criminal offence, anyone who provided such tools would "face a prison sentence and heavy fines".
"X or Grok could prevent these forms of misuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said platforms must "consider the potential" of people seeing illegal content on their platforms, but stopped short of confirming whether it was at this time investigating X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some premium features—which responds to X users' prompts when they mention it in a post.
It is commonly used to provide reactions or more background to other posters' comments, but people on X are also capable of edit an uploaded image through its AI image editing tool.
History of Criticism and Policy
The system has been criticised for enabling users to produce images and footage with nudity and sexually suggestive content, and it was in the past accused of making inappropriate content of public figures.
"X has been permitting the creation and spread of these pictures for a long time without taking any steps and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "depicting images of individuals in a graphic fashion".
In a comment, Ofcom said it was against the law to "make or distribute non-consensual intimate images or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.
It said networks such as X were mandated to take "suitable measures" to "lessen the risk" of users encountering unlawful content on their services and to remove it promptly when they are notified of it.
