The online platform X has provoked major anger after users utilized its artificial intelligence assistant Grok to change pictures of women by taking off their clothing.
Several cases have been seen where the system has been employed to virtually strip female subjects to make them appear in swimwear without consent, as well as to place them in sexualized scenarios.
Developer Response and Regulatory Stance
XAI, the organization behind Grok, did not respond to requests for comment, aside from an standard message stating "traditional news outlets lies".
A government representative stated that legislation is being proposed to ban nudification technology. Under a upcoming law, anyone who distributed such tools would "face a prison sentence and substantial fines".
"X or Grok could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "assess the potential" of people accessing illegal content on their platforms, but did not specify whether it was presently examining X or Grok in regarding artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' queries when they tag it in a tweet.
It is commonly used to give reactions or more background to other users' remarks, but people on X are also capable of alter an posted picture through its AI image editing feature.
Background of Criticism and Rules
The tool has been condemned for permitting users to produce images and videos with explicit content and sexually suggestive material, and it was previously accused of making sexually explicit content of celebrities.
"X has been permitting the generation and distribution of these images for months without taking any measures and we have yet to see any intervention by regulators," the expert added.
XAI's own acceptable use policy prohibits "showing images of individuals in a pornographic fashion".
In a comment, Ofcom said it was illegal to "create or circulate non-consensual intimate images or child sexual abuse material" and stated this covered explicit AI images created with AI.
It said services such as X were mandated to take "appropriate actions" to "minimize the risk" of users viewing unlawful content on their platforms and to take down it swiftly when they become aware of it.
