WormGPT - There is a Dark side to the AI revolution
September 2024
We first heard it muttered years ago...
"When the bad guys start using AI, that's when we are in real trouble!"
It's a fair but generic observation to anyone that has an understanding of how artificial Intelligence works. Afterall AI, and more specifically generative AI is employed to streamline, expediate, and augment any number of tasks - surely the bad guys could use it too, but how serious is the threat? Beyond the fear mongering tactics a lot of cybersecurity vendors use, there is actually a credible amount of damage that can be done by an increasing number of hacking tools that have been augmented with AI.
Some industry commentators would argue that AI has not revolutionized the black hatted side of the equation as much as our defensive capability....or has it? It really depends on where your attention is drawn.
According to one Cyber powerhouse, Ransomware rates were trending down in the previous 12 months, payload based attacks need to be shrouded with living off the land tactics to work and it is seemingly harder for the bad guys to execute on their regular shenanigans, if you ask any Incident Responder worth their salt they are increasingly seeing Ransomware as the goodbye kiss from an attacker who has probably had environment access for a while before walking off into the digital sunset. Great news! The good guys are winning! (just don't mention the fact a vast majority of breaches still occur with compromised credentials Phished from a user).
Logically the drop in Ransomware some are reporting may mean scanning engines have been greatly bolstered by AI and are doing their job. It is 'strange' these days to rely on a signature as a primary method of detection for Malware anyway, and those good old AV test scores have been replaced by Endpoint Detection and Response metrics around the Mitre Attack Framework. We also see solutions such Halcyon really differentiate their technology from your usual EDR/XDR agents, and cripple Ransomware more directly. We love tech like Halcyon, it offers a layered security backstop to the Endpoint in a similar philosophy to Sabiki Email Security does to your mailbox. Shout out to the Halcyon crew!
But let's not high-five each other just yet, there are obviously some pretty smart adversarial cookies out there and they have been methodically integrating elements of AI into their tool set for some time now. While AI seems to be tipping the scales in favor of the good guys when it comes to scan engines and detections of payloads, the bad guys have focused their efforts in employing AI in other stages of an attack.
So what are a few of the areas where the scales have tipped the other way in favor of the bad guys?
Reconnaissance
The first would most definitely have to be reconnaissance. One reconnaissance method that is widely used because of its low sophistication is something called Google Hacking, it is amazing what you can find exposed out there on the Internet using a few basic Google search operators, from vulnerability assessment reports inadvertently sitting on the open web to full employee lists and all manner of information an adversary would need to plan an attack. (Head down that rabbit hole here https://en.wikipedia.org/wiki/Google_hacking).
AI has supercharged methods such as these, but we wont focus on that because our key interest, being an Email Security platform is another area where AI has energized the bad guys and puts users squarely into the crosshairs.
Social Engineering
We wrote an article not too long ago around an investigation uncovering a Phishing campaign targeted at major airline Qantas' frequent flier customers. You can check out the article here but the main point is that attackers were attempting to recruit native English speakers on the dark web to help finalize the last piece of the puzzle in coming up with a colloquially correct message that would entice users to click a link.
We had a good old chuckle at the attackers and were surprised they had not heard of ChatGPT, most definitely the last piece of the puzzle they could have used to potentially launch the campaign. The scary thing about this is that all of a sudden a seemingly innocent AI tool can easily and quickly be used to break down the language barrier for nefarious purposes. With a simple translation from the hacker's native language, then a simple pass through ChatGPT to 'clean up' the message, they could have had a near-perfectly written email in a matter of seconds. We expect the days of poorly worded and obvious BEC/Phishing attacks to be numbered, from here on out it is going to be even harder for your users to spot something nasty.
Fortunately for the good guys, there are guard rails in place to make sure you cant just directly manipulate ChatGPT into being completely evil for you..
Can you imagine if they actually put some effort into training a model to do bad stuff, that would be crazy...
News flash, they have.
Say hello to WormGPT, the new world of AI without the guardrails.
For ~$100 a month, anyone can access what may turn out to be the most potent AI tools of our generation. Calling upon a model that has been trained on data sets from the far reaches of the dark web, decades of devious code, malware samples, language patterns that tickle the very nerve centres of the brain that need to be tickled. With a few strokes of the keyboard a brilliantly crafted email can be generated to trick your finance team into paying that invoice and cut through your 'Gartner Magic Quadrant leader' of an email security solution like a hot knife through butter.
Yes, we know, this has been around a little while so it may be old news to some of you but what really grabbed our attention recently besides some of WormGPT's BETA features (which are even more mind blowing) is the fact it isn't a tool you need to delve into the depths of the dark web to get your hands on anymore.
With WormGPT now able to memorize context of conversations and with unlimited character support, it is entirely possible to have chat bots engaged in complex email chains at scale with multiple victims and then completing the full circle of an attack. Without going off on a tangent around the capability it has around generating malicious code instantly, the platform is turning into a one-stop-shop where an attacker can not only craft a convincing email, but also the plugins, the payload, the Web Server configuration to Phish those credentials and get their hands on your data at speed and at scale or get those funds transferred.
For security professionals that have been in the game since Anti-Virus was shipped on floppy disks, having hands on a platform like WormGPT is almost an out-of-body experience, and it wasn't just us. On a recent trip throughout the region, we caught up with Simon Perry from Secure366 (ex-Carbon Black, ex-RSA).
Besides agreeing that tools like WormGPT don’t just lower the barrier to entry for any aspiring hacker, they blow it away, Simon notes:
"My concern is that as platforms such as WormGPT become ubiquitously imbedded into BECaaS platforms and the emails generated become Teflon coated with significantly better customization ability and dramatically less grammatical and spelling error giveaways that users, and current-generation anti-spam filters have been trained to look for. As always, defenders must adapt."
Addressing WormGPT for what it is, AI without the guardrails, Simon continued..
“GPT type AI models have democratized the skill of writing reasonably good quality and believable text; witness the inroads ChatGPT and other LLM AI models have made into the production of pulp journalism and marketing copy.
Similar models, minus the behavioral guardrails that control what they can be used for, are being used maliciously. A particularly attractive use-case for criminals is the rapid generation of targeted, highly credible spam and phishing emails.”
So there you have it, the bad guys HAVE started to use AI, but how much trouble are 'we' actually in?
It only fortifies out belief at team Sabiki that the control our platform gives to administrators over their own AI model can only be a benefit when combatting tools such as WormGPT.
Having a Dynamic AI engine that can be actively and passively trained on your specific mail flow may just be the ace up your sleeve when coming up against other advanced models, purpose built to be layered behind Microsoft's embedded Email security capability, why not try it out free on your Microsoft 365 mail tenant today?
*WormGPT Img Sources https://www.opensourceforu.com/
Developed by Email Security Professionals and Data scientists with decades of experience to make life easier for customers and MSPs alike, Sabiki Email Security is a cloud-native 'built-for Microsoft 365' SaaS solution that protects your organization from Phishing, Spam and targeted scams using the power of a dynamic AI feedback loop engine. Powered by a 'Dynamic' Machine Learning engine in combination with next-generation contextual and behavioral analysis capabilities, Sabiki Email Security provides an incredible level of granularity in engine customization with seamless onboarding and operation.