January 22, 2024 Joe Honey

Generative AI on the Dark Web: The Case of the XSSBot

Threat Intelligence Engineer Joe Honey examines XSSBot, an account on the dark web hacking forum XSS powered by generative ai.

The Rise and Fall of the XSSBot

ChatGPT, AI, and Machine Learning are rarely out of the news these days, and in fact seem to be seeping into every facet of our lives. This trend has even carried over into the dark web, as we recently observed with the cybercrime forum XSS’s “XSSBot”.

XSS is a primarily Russian-language focused dark web forum that specializes in hacking topics. It’s a long running corner of the dark web that is well respected in the cybercriminal ecosystem, with more than 20 years of posts and 6000+ members creating a community where almost anything hacking-focused is discussed – including the trading of tips, techniques, or illegally obtained goods. (As an interesting aside, one of the few things that the operators of XSS do not allow is discussion of ransomware).

In March 2023 we observed some interesting posts from a new account on the forum called “XSSBot”. After some investigation, it became clear that this was a bot account that uses ChatGPT to power its responses.

An overview of XSSBot’s activity in our dark web investigation platform, Cerberus.

The XSSBot account was primarily active from March 2023 to October 2023. It entered the forum, answered questions and solved problems posed by users for that seven month period, and then disappeared again – all with quite limited fanfare. No posts or announcements were observed by the admins or creators of the bot either announcing its launch or its exit. After October 2023, we assume (but cannot confirm) that there was a technical change somewhere – either in the coding of the bot or its connection to the ChatGPT service it relies on – as only posts by the bot saying “sorry I couldn’t contact the ChatGPT think tank” were seen.

After October 2023 the XSSBot only replied with this response.

How the XSSBot was used

As with most things powered by AI or Machine Learning, we saw a range of questions being posed to the XSSBot – from harmless to malicious, sensible to amusing, and most things in between. Each question was structured as a new thread on the forum, with the actor posing an initial question and the XSSBot replying. The actor also had the opportunity to ask follow up questions or tweak prompts by replying to the bot. We observed a mix of responses – while some actors were content with the original answer provided by the bot, others evidenced clear attempts to refine the answer the bot gave, or to get it to “override” its programming and/or ethical blocks.

Example questions posed to the bot included:

Asking it to create code. Python was the most popular but we also observed questions around JavaScript, Java, Perl and C#; as well as specific questions about Android and iOS operating systems.
Asking what is known about a particular piece of malware.
Asking the bot to deobfuscate some code.
Asking XSSBot to write a rap describing the XSS forum.

And many more.

The first verse and chorus of the rap generated by the XSS bot in response to a forum user’s request. In English, this reads:

(Verse 1)
XSS shadow forum, they talk about freedom here,
Where everyone can say what they really think.
But be careful, my friend, Software edges are at play here,
XSS – cyber attacks, injected into every branch.

Phishing, injections, that’s what we specialize in,
Hackers and programmers, virtual war on guard of insecurity.
They explore, analyze every vulnerability,
They find a weak point, and then the battle begins.

(Chorus)
XSS Shadow Forum, a world where defenses crumble,
Here the heroes of cyberspace fight for freedom.
They explore, they hack, they look for new ways,
XSS Shadow Forum, where legends are born.

How the XSSbot Was Built

There is remarkably little discussion about where the XSSBot came from or whether it will become a regular feature of the forum.

The earliest verifiable mention of XSSBot is from a competition hosted on the forum in June 2020 calling for articles – essentially an R&D competition with prizes on offer for the most interesting or useful submissions. Among the topics discussed was a post by a threat actor called “Jeffs” who, in collaboration with other threat actors (specifically “X-shar”, with claimed contributions from “kelvinthechamp”) created “XssBot – A modular resident bot with a super admin panel”. Further posts by the Jeffs and X-shar reveal a GitHub account (https://github.com/XShar/XssBot) that contained the relevant code and was open to review – both for ideas and advice as well as reporting and remediation of vulnerabilities.

In a potentially competing history of XSSBot, there was a post from an actor called “gilderexpert” in May 2023 that states the bot had been around on the forum since 2005 (long before ChatGPT was released in 2022). However, based on the content of the post and the lack of earlier mentions or activity by a bot-type account we do not assess this as credible.

Jeffs and X-shar detailed in their competition entry the technical workings of their code, except exactly what powers the answers given by the bot. Based on the later error messages, circumstantial evidence such as the timeline of the bots activity, as well as ChatGPT’s established lack of knowledge of events post 2021 – we believe XSSBot used ChatGPT to provide the answers.

Furthermore, in posts where threat actors asked the XSSBot to provide illegal or immoral content, it refused and can also be observed dispensing moral warnings on the legality or ethicality of the question asked. This again reinforces the assumption that the account is powered by a commercial service like ChatGPT as a bot created directly by a threat actor would likely be more comfortable providing malicious or illegal content.

The article published by Jeffs and X-shar makes some key points on the bots workings:

Modular Bot Design – allowing the addition of new modules dynamically to the bot for varied functionalities.
Secure Communication – custom encryption protocol between the bot and server.
Client-Side Operations – bot checks system language and registers with the server, performing assigned tasks.
Server and Admin Panel Development – Uses Golang and Vue.js for managing bot tasks and users.
Encryption in Server-Bot Communication – Ensures secure data transfer between the bot and the server.

Generative AI and the Dark Web

Outside of the XSSBot, ChatGPT and similar generative AI has been a topic of interest on the dark web since its release in 2022. We’ve observed many posts talking about the technology – from debates on its usage and potential, to discussion on how it can be abused to support criminal activity. There have also been attempts to subvert the technology and bypass the ethical restrictions the owners have placed on it. Models such as EvilGPT and WormGPT have been generated for use cases such as creating malware faster or drafting of convincing phishing emails.

While unproven technology on the dark web can seem like a nebulous threat, it is something that must be taken seriously. As AI language models become more advanced they will allow attackers to iterate faster and create more convincing copy – all of which makes life harder for cyber defenders.