Artificial intelligence (AI) can be manipulated into conducting cyberattacks and scams, according to new research from IBM (NYSE:IBM).
Researchers with the Armonk-headquartered company warned that they”™ve discovered simple workarounds that can enable large language models (LLMs) ”” including ChatGPT ”” to write malicious code and give poor security advice. These findings, which were first published in Axios, detailed how easily someone with minimal computer skills could create malware and phishing emails using LLMs.
“What we learned was that English has essentially become a ”˜programming language”™ for malware,” said Chenta Lee, chief architect of threat intelligence at IBM, a blog posting. “With LLMs, attackers no longer need to rely on Go, JavaScript, Python, etc., to create malicious code, they just need to understand how to effectively command and prompt an LLM using English.
appear legitimate.”
Lee pointed out how researchers were able to “hypnotize” LLMs to deliver responses that were incorrect and potentially dangerous.
“We were able to get LLMs to leak confidential financial information of other users, create vulnerable code, create malicious code, and offer weak security recommendations,” Lee recalled, adding that “in the classic sense, data poisoning would require that a threat actor inject malicious data into the LLM in order to manipulate and control it, but our experiment shows that it”™s possible to control an LLM, getting it to provide bad guidance to users, without data manipulation being a requirement. This makes it all the easier for attackers to exploit this emerging attack surface.”
