Can LLMs be used maliciously in cyberattacks?

Yes, large language models (LLMs) can be exploited maliciously in cyberattacks. Their ability to generate human-like text, automate tasks, and process large amounts of data makes them a potential tool for attackers. While LLMs themselves are neutral, their misuse depends on how malicious actors apply their capabilities. For example, attackers can leverage LLMs to craft convincing phishing emails, impersonate trusted entities, or even generate code for malicious purposes. These risks stem from the models’ flexibility and accessibility, which, while beneficial for legitimate uses, can also be repurposed for harm.

One concrete example is the use of LLMs to automate phishing campaigns. Attackers can generate highly personalized scam emails or messages at scale by feeding the model details scraped from social media or leaked databases. Unlike traditional phishing templates, LLM-generated content can mimic writing styles, adapt to regional dialects, and bypass basic spam filters. Similarly, LLMs can be used to create fake customer support chatbots that trick users into sharing sensitive information like passwords or credit card details. Another risk involves code generation: attackers could prompt LLMs to produce malware snippets, exploit code, or scripts for brute-force attacks. While some models include safeguards to block overtly malicious requests, attackers often bypass these by rephrasing prompts or breaking tasks into innocuous-seeming steps.

Developers should also consider indirect risks. For instance, LLMs trained on public code repositories might inadvertently reveal API keys, passwords, or vulnerabilities present in training data. Attackers could exploit this by querying models for sensitive information embedded in code examples. Additionally, LLMs can assist in reverse-engineering software or generating obfuscated code to evade detection by security tools. To mitigate these risks, developers integrating LLMs into applications should implement strict input validation, monitor outputs for suspicious patterns, and avoid exposing model access to untrusted users. Understanding these threats helps in designing safeguards, such as ethical use policies and technical controls, to limit adversarial exploitation.