🔗 Share this article

The online network X has caused significant backlash after individuals utilized its AI chatbot Grok to manipulate pictures of females by removing their clothing.

Multiple instances have emerged where the technology has been used to digitally undress female subjects to render them in bikinis non-consensually, as well as to place them in inappropriate situations.

Company Reaction and Regulatory Position

XAI, the company that created Grok, remained silent to inquiries, aside from an standard response stating "legacy media lies".

A government representative stated that new laws is being proposed to prohibit nudification tools. Under a upcoming criminal offence, anyone who distributed such tools would "face a jail term and significant penalties".

"X or Grok has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."

Oversight Body Stance and Tool Capabilities

The communications watchdog Ofcom said technology companies must "consider the potential" of people seeing unlawful material on their services, but stopped short of confirming whether it was presently investigating X or Grok in concerning AI-generated images.

Grok is a free chatbot—with some premium functions—which reacts to X users' prompts when they tag it in a message.

It is often used to offer reactions or more context to other users' comments, but people on X are also can alter an uploaded image through its AI image editing feature.

History of Controversy and Policy

The chatbot has been criticised for permitting users to produce pictures and clips with explicit content and sexually suggestive content, and it was previously alleged to have been involved in generating sexually explicit material of celebrities.

"X has been allowing the generation and spread of these images for a long time without taking any steps and we have yet to see any intervention by oversight bodies," Professor McGlynn added.

XAI's own terms of service bans "showing likenesses of people in a graphic fashion".

In a comment, Ofcom said it was against the law to "generate or share private pictures or illegal content" and confirmed this covered sexual deepfakes generated with AI.

It said services such as X were required to take "appropriate steps" to "lessen the risk" of users viewing unlawful material on their services and to remove it quickly when they are notified of it.