The online network X has provoked significant outrage after people employed its artificial intelligence assistant Grok to alter pictures of females by removing their garments.
Several instances have surfaced where the technology has been used to digitally undress women to make them appear in swimwear non-consensually, as well as to put them in sexualized situations.
Company Response and Regulatory Stance
XAI, the firm that created Grok, remained silent to questions, other than an automatically-generated message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being brought forward to outlaw image stripping software. Under a upcoming law, anyone who supplied such tools would "be subject to a prison sentence and significant fines".
"The platform is capable of halting these types of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Functionality
The regulator Ofcom said technology companies must "consider the danger" of people accessing unlawful material on their platforms, but stopped short of confirming whether it was presently investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost AI assistant—with some additional functions—which answers X users' prompts when they mention it in a message.
It is often used to give commentary or additional background to other posters' comments, but people on X are also able to edit an uploaded image through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The tool has been criticised for allowing users to generate photos and videos with nudity and sexualised material, and it was previously alleged to have been involved in making sexually explicit content of celebrities.
"The platform has been permitting the generation and distribution of these pictures for months without taking any steps and we have not seen any intervention by oversight bodies," the expert added.
XAI's own terms of service forbids "portraying images of individuals in a pornographic manner".
In a statement, Ofcom said it was illegal to "make or circulate private pictures or illegal content" and clarified this included explicit AI images created with AI.
It said services such as X were mandated to take "necessary measures" to "minimize the risk" of users viewing unlawful content on their services and to take down it swiftly when they become aware of it.
