Elon Musk’s social media platform X has announced sweeping restrictions on its AI chatbot Grok following worldwide outrage over its role in generating sexualised images of women and children, a controversy that has triggered investigations, app bans, and regulatory probes across multiple continents.
The measures were unveiled on Wednesday as pressure mounted on xAI, the company behind Grok, after authorities in the United States and Europe raised alarm over the chatbot’s misuse. California’s Attorney General had earlier confirmed the launch of a formal investigation into xAI, citing the spread of non-consensual and sexually explicit images allegedly created and circulated through the platform.
In a statement, X said it would geoblock the ability of users to create or edit images of real people in bikinis, underwear, or similar attire in countries where such content is illegal. The platform added that technological safeguards had been deployed to stop Grok from editing images of real individuals in revealing clothing, stressing that the restriction applies to all users, including paid subscribers.
As part of what it described as an extra layer of protection, X disclosed that image creation and photo-editing features through the Grok account would now be limited strictly to paid users. The company said the changes were aimed at preventing further abuse and restoring confidence in its safety controls.
The European Commission, acting as the European Union’s digital watchdog, welcomed the announcement cautiously, saying it had taken note of the additional steps taken by X to stop Grok from producing sexualised images involving women and children. EU officials said the new safeguards would be closely examined to ensure they genuinely protect users across member states.
Read Also:
- Ofcom investigates Elon Musk’s X over Grok AI sexual deepfakes
- Grok disables image generation for free users after deepfake backlash
- Presidency explains Grok watermark on Tinubu–Kagame Paris photograph
Scrutiny intensified after Grok’s so-called “Spicy Mode” feature allowed users to generate sexualised deepfake images using simple prompts. The feature sparked outrage among regulators and rights groups, who warned that the technology was being weaponised to harass individuals online.
California Attorney General Rob Bonta described the situation as shocking, saying his office had received a flood of reports detailing non-consensual, sexually explicit material produced and shared through xAI’s systems. He stressed that the state has zero tolerance for AI-driven creation or distribution of intimate images without consent, particularly where minors are involved. The investigation, he said, would determine whether xAI breached state laws after the material was allegedly used to target and harass individuals online.
California Governor Gavin Newsom also weighed in, condemning what he called xAI’s vile decision to allow such content to spread and urging regulators to hold the company accountable.
The controversy has sparked international action. Indonesia became the first country to block access to Grok entirely, with Malaysia following shortly after. Malaysian authorities later said X’s initial steps to curb indecent image generation were incomplete, adding that the temporary ban would only be lifted if the platform fully disabled the harmful features.
In India, authorities said X had removed thousands of posts and hundreds of accounts following official complaints, while Britain’s media regulator Ofcom confirmed it had opened an investigation into whether X breached UK laws governing harmful online content.
Adding to the pressure, a coalition of 28 civil society groups submitted open letters to Apple and Google, urging them to remove Grok and X from their app stores amid the surge in sexualised AI-generated images.
The scale of the issue was underscored by a recent analysis of more than 20,000 Grok-generated images by Paris-based non-profit AI Forensics, which found that over half depicted individuals in minimal attire, mostly women, with a small percentage appearing to involve minors.
As regulators close in and public scrutiny intensifies, X now faces a defining test over whether its latest restrictions will be enough to rein in Grok and prevent further misuse of AI technology on its platform.
