Indonesia has become the first country in the world to block Elon Musk's Grok chatbot over the risk of fake, AI-generated pornographic content. The country's communication and digital affairs minister said on Saturday that "the practice of non-consensual sexual deepfakes" is a "serious violation of human rights, dignity, and the security of citizens in the digital space". The move comes a day after Grok limited image generation and editing features on Musk's social media platform X to paying subscribers as it sought to tamp down mounting criticism over the deepfakes. Musk has been threatened with fines as several countries are pushing back publicly against Grok, which allowed users to alter online images to remove the subjects' clothes. The billionaire has said anyone using Grok to create illegal content would face the same consequences as uploading such material directly. But European officials and tech campaigners slammed this week's move to limit the AI tool's features to paying subscribers on X, saying it failed to address their concerns.

A phone screen displaying the Grok app and logo is seen on 7 January 2026. A phone screen displaying the Grok app and logo is seen on 7 January 2026. Indonesia blocks Musk's Grok chatbot due to risk of pornographic content Indonesia temporarily blocked Elon Musk's Grok chatbot on Saturday due to the risk of AI-generated pornographic content, becoming the first country to deny access to the AI tool. The move comes after governments, researchers and regulators from Europe to Asia have condemned and some have opened inquiries into sexualised content on the app. Grok AI: is it legal to produce or post undressed images of people without their consent?

Text-to-image (T2I) models have been widely applied in generating high-fidelity images across various domains. However, these models may also be abused to produce Not-Safe-for-Work (NSFW) content via jailbreak attacks. Existing jailbreak methods primarily manipulate the textual prompt, leaving potential vulnerabilities in image-based inputs largely unexplored. Moreover, text-based methods face challenges in bypassing the model's safety filters. In response to these limitations, we propose the Multimodal Prompt Decoupling Attack (MPDA), which utilizes image modality to separate the harmful semantic components of the original unsafe prompt. MPDA follows three core steps: firstly, a large language model (LLM) decouples unsafe prompts into pseudo-safe prompts and harmful prompts. The former are seemingly harmless sub-prompts that can bypass filters, while the latter are sub-prompts with unsafe semantics that trigger filters. Subsequently, the LLM rewrites the harmful prompts into natural adversarial prompts to bypass safety filters, which guide the T2I model to modify the base image into an NSFW output. Finally, to ensure semantic consistency between the generated NSFW images and the original unsafe prompts, the visual language model generates image captions, providing a new pathway to guide the LLM in iterative rewriting and refining the generated content.

Pornographic content occurring in human-machine interaction dialogues can cause severe side effects for users in open-domain dialogue systems. However, research on detecting pornographic language within human-machine interaction dialogues is an important subject that is rarely studied. To advance in this direction, we introduce CensorChat, a dialogue monitoring dataset aimed at detecting whether the dialogue session contains pornographic content. To this end, we collect real-life human-machine interaction dialogues in the wild and break them down into single utterances and single-turn dialogues, with the last utterance spoken by the chatbot. We propose utilizing knowledge distillation of large language models to annotate the dataset. Specifically, first, the raw dataset is annotated by four open-source large language models, with the majority vote determining the label. Second, we use ChatGPT to update the empty label from the first step. Third, to ensure the quality of the validation and test sets, we utilize GPT-4 for label calibration. If the current label does not match the one generated by GPT-4, we employ a self-criticism strategy to verify its correctness. Finally, to facilitate the detection of pornographic text, we develop a series of text classifiers using a pseudo-labeled dataset. Detailed data analysis demonstrates that leveraging knowledge distillation techniques with large language models provides a practical and cost-efficient method for developing pornographic text detectors.

A new chatbot, similar to ChatGPT, is able to turn text into celebrity voices, creating "deepfakes" in the style of Morgan Freedman, Jordan Peterson, Donald Trump and many more. NoiseGPT can even be trained by users to imitate their own voice, or that of their friends, family members or work colleagues. Imagine getting a happy birthday voice-message from your favourite US president, or a voice from beyond the grave in the form of John Lennon or Elvis sharing some personal information with you, that only your closest relatives know about. This is the selling point of the newest chatbot application to be released following the much-hyped launch of Microsoft-backed (MSFT) ChatGPT artificial intelligence content generator in November 2022. NoiseGPT's chief operational officer Frankie Peartree told Yahoo Finance UK: "We are training the AI to mimic around 25 celebrity voices at the moment, and will soon have 100 plus celebrity voices to offer."

San Francisco (CNN)Deepfake videos are quickly becoming a problem, but there has been much debate about just how big the problem really is. One company is now trying to put a number on it. There are at least 14,678 deepfake videos -- and counting -- on the internet, according to a recent tally by a startup that builds technology to spot this kind of AI-manipulated content. And nearly all of them are porn. The number of deepfake videos is 84% higher than it was last December when Amsterdam-based Deeptrace found 7,964 deepfake videos during its first online count.

Tumblr has been banned in Indonesia for providing people with access to pornographic content. The communications ministry said the block – which means nobody in the country can access the blogging platform – came after complaints about hundreds of accounts that provided pornography. The country has stepped up efforts to police the kind of content available online. It has been trying to stem a rise in hoax stories and hate speech, and anti-pornography laws pushed by Islamic parties in the country have led tens of thousands of adult websites to be taken offline. The I.F.O. is fuelled by eight electric engines, which is able to push the flying object to an estimated top speed of about 120mph.