Cyberattacks Using ChatGPT Disrupted by OpenAI: Chinese, North Korean, and Russian Hackers Targeted
Cyberattacks Using ChatGPT Disrupted by OpenAI: Chinese, North Korean, and Russian Hackers Targeted
Three activity clusters were terminated by OpenAI on Tuesday for abusing its ChatGPT artificial intelligence (AI) tool to aid in the creation of malware.
Among them is a Russian-speaking threat actor who allegedly utilized the chatbot to assist in the creation and improvement of a remote access trojan (RAT), a credential stealer designed to avoid detection. Additionally, the operator prototyped and debugged technical elements that facilitate credential theft and post-exploitation using multiple ChatGPT accounts.
“These accounts appear to be affiliated with Russian-speaking criminal groups, as we observed them posting evidence of their activities in a Telegram channel dedicated to those actors,” stated OpenAI.
Although the threat actor’s direct requests for malicious material were denied by the AI company’s large language models (LLMs), they circumvented the restriction by building-block code that was subsequently put together to generate the workflows.
Code for obfuscation, clipboard monitoring, and simple tools to exfiltrate data using a Telegram bot were among the outputs generated. It is important to note that none of these outputs by themselves is necessarily harmful.
“The threat actor made a mix of high‑ and lower-sophistication requests: many prompts required deep Windows-platform knowledge and iterative debugging, while others automated commodity tasks (such as mass password generation and scripted job applications),” said OpenAI.
The operator iterated on the same code throughout talks and employed a limited number of ChatGPT accounts, which is consistent with continuous development as opposed to sporadic testing.
The second cluster of activity came from North Korea and overlapped with a campaign described by Trellix in August 2025 that used spear-phishing emails to spread Xeno RAT to diplomatic missions in South Korea.
The actors, according to OpenAI, worked on specific projects like creating macOS Finder extensions, setting up Windows Server VPNs, or changing Chrome extensions to their Safari counterparts. The cluster also used ChatGPT for malware and command-and-control (C2) creation.
Furthermore, it has been discovered that the threat actors utilize the AI chatbot to create phishing emails, test out cloud services and GitHub features, and investigate methods to enable password theft, DLL loading, in-memory execution, and Windows API hooking.
According to OpenAI, a cluster tracked by Proofpoint under the name UNK_DropPitch (also known as UTA0388), a Chinese hacker group known for phishing campaigns targeting major investment firms with an emphasis on the Taiwanese semiconductor industry, and a backdoor known as HealthKick (also known as GOVERSHELL) shared overlaps with the third set of banned accounts.
In addition to helping with tooling to speed up routine tasks like remote execution and traffic protection using HTTPS, the accounts used the tool to create content for phishing campaigns in English, Chinese, and Japanese. They also looked for information about installing open-source tools like fscan and nuclei. The threat actor was characterized by OpenAI as “technically competent but unsophisticated.”
In addition to these three nefarious online actions, the business also blocked accounts that were being used for influence and fraud operations.
- ChatGPT is being abused by networks most likely based in Nigeria, Myanmar, and Cambodia in an effort to defraud people online. These networks created content for social media to promote investment schemes, translated messages, and wrote messages using AI.
- ChatGPT is reportedly being used by people connected to Chinese government organizations to help with data analysis from Chinese or Western social media platforms and surveillance on individuals, especially members of ethnic minorities like Uyghurs. Users did not use the AI chatbot to execute the promotional materials they requested from the tool.
- A threat actor with Russian roots that was connected to Stop News and was probably operated by a marketing firm that created videos and material for social media platforms using its AI models, among others. The information that was produced was critical of Russia’s presence in Africa as well as the roles played by the United States and France. Additionally, it created content in English that promoted narratives against Ukraine.
- Using its models, a Chinese-led clandestine influence operation known as “Nine—emdash Line” produced social media posts criticizing Philippine President Ferdinand Marcos, Vietnam’s purported environmental impact in the South China Sea, and political figures and activists associated with Hong Kong’s pro-democracy movement.
In two instances, suspected Chinese accounts requested that ChatGPT locate the funders of an X account that was critical of the Chinese government and the organizers of a petition in Mongolia. According to OpenAI, their models did not produce any sensitive data; instead, they solely produced responses that were publicly accessible.
“A fresh application for this [China-linked influence network] was requested for advice on social media growth strategies, like how to start a TikTok challenge and get others to post content about the #MyImmigrantStory hashtag (a widely used hashtag of long standing whose popularity the operation likely strove to leverage),” said OpenAI.
“They asked our model to ideate and then generate a transcript for a TikTok post, in addition to providing recommendations for background music and pictures to accompany the post.”
OpenAI reaffirmed that its tools were utilized to add incremental efficiency to the threat actors’ current workflows and gave them new capabilities that they could not have otherwise acquired from a variety of publicly available web resources.
However, one of the report’s most intriguing conclusions is that threat actors are attempting to modify their strategies in order to eliminate any indications that the content was produced using artificial intelligence.
“One of the scam networks [from Cambodia] we interrupted asked our model to eliminate the em-dashes (long dash, –) from their output, or appears to have removed the em-dashes directly before publication,” the business stated. “For months, em-dashes have been the subject of online discussion as a potential sign of AI usage: this case indicates that the threat actors were mindful of that discussion.”
The OpenAI findings coincide with rival Anthropic’s release of Petri (short for “Parallel Exploration Tool for Risky Interactions”), an open-source auditing tool designed to speed up AI safety research and gain a better understanding of model behavior in a number of areas, including self-perseverance, cooperation with harmful requests, deception, sycophancy, and encouragement of user delusion.
“Petri deploys an automated agent to test a target AI system through various multi-turn conversations that involve simulated users and tools,” Anthropic stated.
“A list of seed instructions aimed at the situations and behaviors the researchers wish to examine is given to Petri. Each seed instruction is then processed in parallel by Petri. An auditor agent plans and engages in a tool usage loop with the target model for every seed instruction. To enable researchers to swiftly explore and filter for the most intriguing transcripts, a judge ultimately assigns points to each of the generated transcripts based on a variety of criteria.
About the Author:
Yogesh Naager is a content marketer who specializes in the cybersecurity and B2B space. Besides writing for the News4Hackers blogs, he also writes for brands including Craw Security, Bytecode Security, and NASSCOM.
Read More:
EBS Zero-day exploited in Clop Data Theft Attack Patched by Oracle
About Author
English