The online network X has provoked significant backlash after users utilized its AI chatbot Grok to manipulate pictures of women by taking off their clothing.
Numerous instances have emerged where the technology has been used to virtually strip female subjects to render them in swimwear non-consensually, as well as to insert them in inappropriate scenarios.
Developer Response and Regulatory Position
XAI, the organization responsible for Grok, did not respond to requests for comment, except for an standard message stating "legacy media lies".
A government representative stated that new laws is being introduced to prohibit image stripping software. Under a upcoming criminal offence, anyone who supplied such technology would "receive a jail term and substantial penalties".
"The platform could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said technology companies must "assess the danger" of people seeing unlawful material on their services, but did not confirm whether it was presently looking into X or Grok in regarding AI-generated pictures.
Grok is a no-cost artificial intelligence—with some premium features—which responds to X users' requests when they tag it in a post.
It is frequently used to offer commentary or additional context to other users' remarks, but people on X are also capable of edit an uploaded photo through its AI image editing function.
Background of Controversy and Rules
The system has been condemned for enabling users to produce photos and footage with nudity and sexualised content, and it was in the past accused of generating inappropriate content of public figures.
"X has been enabling the production and spread of these images for months without taking any measures and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own usage rules prohibits "portraying likenesses of individuals in a pornographic manner".
In a comment, Ofcom said it was against the law to "generate or share non-consensual intimate images or illegal content" and stated this included explicit AI images generated with AI.
It said networks such as X were mandated to take "necessary steps" to "lessen the risk" of users encountering illegal material on their platforms and to take down it quickly when they are notified of it.
