Elon Musk's social media network X has caused widespread backlash after users used its AI chatbot Grok to manipulate photos of women by stripping their attire.
Multiple examples have emerged where the tool has been used to digitally undress female subjects to render them in swimwear without consent, as well as to insert them in sexualized scenarios.
Developer Reaction and Regulatory Stance
XAI, the company that created Grok, remained silent to requests for comment, except for an automatically-generated message stating "legacy media lies".
A government official stated that new laws is being introduced to ban nudification tools. Under a proposed criminal offence, anyone who supplied such technology would "receive a jail term and substantial fines".
"The platform could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "evaluate the risk" of people viewing unlawful material on their platforms, but did not specify whether it was currently looking into X or Grok in relation to artificial intelligence pictures.
Grok is a free AI assistant—with some additional features—which reacts to X users' requests when they mention it in a message.
It is frequently used to provide reactions or additional background to other users' comments, but people on X are also can alter an posted picture through its AI image editing tool.
Background of Criticism and Rules
The system has been faulted for enabling users to produce photos and clips with explicit content and sexually suggestive content, and it was earlier accused of creating graphic content of celebrities.
"X has been enabling the generation and spread of these pictures for months without taking any action and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "showing likenesses of individuals in a pornographic way".
In a comment, Ofcom said it was against the law to "make or circulate non-consensual intimate images or child sexual abuse material" and stated this covered sexual deepfakes created with AI.
It said services such as X were mandated to take "suitable steps" to "lessen the risk" of users coming across unlawful content on their services and to delete it quickly when they learn of it.
