Elon Musk's social media network X has caused major outrage after users utilized its AI chatbot Grok to alter photos of females by removing their attire.
Several examples have been seen where the technology has been used to remove clothing from women to render them in swimwear non-consensually, as well as to put them in inappropriate scenarios.
Company Reaction and Legal Position
XAI, the firm behind Grok, offered no comment to inquiries, except for an automatically-generated message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being brought forward to prohibit nudification tools. Under a upcoming law, anyone who supplied such tools would "be subject to a prison sentence and significant fines".
"The platform could prevent these forms of abuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said technology companies must "consider the risk" of people accessing unlawful material on their networks, but did not confirm whether it was at this time looking into X or Grok in concerning artificial intelligence images.
Grok is a no-cost artificial intelligence—with some additional features—which reacts to X users' queries when they tag it in a tweet.
It is commonly used to give commentary or additional background to other posters' comments, but people on X are also can alter an uploaded photo through its AI image editing function.
Background of Controversy and Rules
The tool has been faulted for allowing users to generate photos and clips with explicit content and sexually suggestive content, and it was previously alleged to have been involved in making inappropriate material of celebrities.
"X has been allowing the production and spread of these pictures for months without taking any action and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "showing likenesses of individuals in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "generate or share deepfakes or illegal content" and clarified this included sexual deepfakes made with AI.
It said services such as X were required to take "appropriate measures" to "reduce the danger" of users encountering unlawful content on their services and to delete it swiftly when they learn of it.
