ChatGPT Exploited in Chinese Influence Operation: Cybersecurity Risks and Implications

Post Views: 126

Chinese Law Enforcement Used ChatGPT for Influence Operations

A recent investigation by OpenAI has revealed that Chinese law enforcement leveraged the company’s ChatGPT chatbot to conduct influence operations against Japanese Prime Minister Sanae Takaichi and other critics of the Chinese Communist Party (CCP).

The Campaign

The campaign, which took place in October, involved using ChatGPT to generate negative comments about Takaichi following her remarks on alleged human rights abuses in Inner Mongolia.

According to OpenAI’s findings, the Chinese police-linked user behind the campaign initially attempted to use ChatGPT to publish and amplify the negative comments. However, when the chatbot rejected these requests, the user turned to other large language models (LLMs) to achieve their goals.

OpenAI’s analysis of the generated status reports revealed that the attacker had attempted to garner support from Japanese influencers and had deployed multiple hashtags, none of which gained significant traction. The investigation also uncovered evidence of other campaigns aimed at exerting psychological pressure on online critics of the CCP.

Investigator’s Comments

The lead investigator on the case, Ben Nimmo of OpenAI’s Intelligence and Investigations Team, noted that this type of behavior is characteristic of modern, transnational repression tactics employed by the CCP.

“This is not just about digital trolling or online harassment,” Nimmo said. “It’s about using every available means to silence critics and exert pressure on them from all angles.”

Implications

The incident highlights the potential risks and challenges associated with the use of LLMs in influence operations and underscores the need for developers and users to be aware of these risks and take steps to mitigate them. As the use of AI and machine learning technologies continues to evolve and expand, it is likely that we will see more instances of these types of campaigns and other malicious activities.