The social media network X has sparked significant backlash after individuals used its artificial intelligence assistant Grok to change photos of women by stripping their clothing.
Numerous examples have emerged where the technology has been employed to remove clothing from female subjects to make them appear in swimwear non-consensually, as well as to place them in sexually explicit contexts.
Company Response and Regulatory Stance
XAI, the organization behind Grok, offered no comment to inquiries, aside from an automatically-generated response stating "traditional news outlets lies".
A government official stated that legislation is being brought forward to ban nudification technology. Under a proposed law, anyone who supplied such technology would "be subject to a prison sentence and significant penalties".
"X or Grok has the ability to stop these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said technology companies must "assess the danger" of people viewing illegal content on their networks, but did not confirm whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost chatbot—with some additional features—which responds to X users' requests when they reference it in a post.
It is often used to give commentary or further background to other users' remarks, but people on X are also can edit an posted photo through its artificial intelligence picture modification feature.
Background of Controversy and Policy
The chatbot has been criticised for enabling users to produce images and footage with nudity and sexualised material, and it was earlier alleged to have been involved in making sexually explicit material of public figures.
"The platform has been allowing the generation and sharing of these images for a long time without taking any measures and we have not witnessed any action by oversight bodies," the expert added.
XAI's own usage rules prohibits "portraying images of people in a graphic fashion".
In a comment, Ofcom said it was illegal to "make or distribute private pictures or child sexual abuse material" and stated this included sexual deepfakes generated with AI.
It said networks such as X were obligated to take "necessary measures" to "reduce the danger" of users coming across illegal material on their platforms and to take down it swiftly when they are notified of it.
