Clear, Deep, and Dark Web: Beyond the Iceberg

The clear-deep-dark web iceburg

Here at Searchlight Security, we think understanding these core concepts is crucial to getting the most value out of dark web intelligence. That’s why we’ve prepared a quick guide on the differences between these categories of the internet, how they interact with each other, and busting some common misconceptions.

Perhaps the most famous analogy used to explain the clear-deep-dark web spectrum is an iceberg. In this example, the internet is represented by a large iceberg with only a small area above the surface. This exposed portion signifies the clear web, or what most people think of when they imagine “the internet”.

The clear web includes any website that can be indexed by standard search engines, such as Google, and thus is easily accessible via a standard browser by most internet users. It is estimated to make up 4% of the entire internet. Beneath this is the deep web – thought to comprise over 90% of the total internet – and beneath that is the dark web, estimated to account for roughly 5%. While this analogy is a quick and simple way to communicate the basic structures and ratios, it leaves out some important details and can lead to several common misconceptions being held about the dark web.

Common misconception #1: the deep and dark web are the same thing

One of the most widespread misconceptions when it comes to the deep web and dark web is conflating the two concepts, due to their shared position “under the water’s surface”. This can lead to both terms being used interchangeably, which is confusing and technically incorrect. Though the names are similar, the deep web and dark web describe two very different parts of the internet.

The deep web refers to all sites that are not indexed by search engines but may still be accessible via standard web browsers. As a result, these sites are difficult to find without a direct link or IP address, and often additional authentication. Despite its obscure position online, the deep web is not necessarily nefarious; the vast majority of sites occupying this space host things like medical databases, internal login portals, academic journals, legal documents, and financial records. Although it’s purposely hidden, this is typically for privacy, security, or copyright reasons.

The dark web, on the other hand, is purposely hidden for different reasons, usually to avoid scrutiny by law enforcement, governments or other entities, like internet service providers (ISPs) and advertising agencies. The dark web refers to sites that are not indexed by standard search engines and are not accessible via standard web browsers, instead requiring specialized software such as Tor (read our blog about how Tor works here). Again, not all dark web sites, called hidden services or onions on Tor, would be considered immoral. In repressive regimes, access to the dark web provides an important outlet for citizens to criticize their government and to engage in whistleblowing activities, such as sending documents securely to journalists.

However, it is unfortunately the case that the majority of hidden services host harmful content, ranging from hacking forums discussing techniques and tools like exploits and malware, to marketplaces selling drugs, stolen personal information, and counterfeit goods, to sites facilitating the distribution and consumption of CSEA (child sexual exploitation and abuse) content. The way these dark web sites are accessed – and their purposes for being hidden – are evidently different from the fairly innocuous content typically found on the deep web, hence the importance of the distinction.

Common misconception #2: cybercrime only happens on the dark web

The second often-held assumption is that cybercriminal behavior only takes place in the murky depths of the deep and dark web, safe from search engine snooping. As covered in the previous section, cybercriminal activity is more within the purview of the dark web than the deep web. However, this does not mean that cybercrime doesn’t exist “above-water” on the clear web; in fact, some of the most active criminal communities online maintain a presence there.

It may seem counterintuitive for illegal actors to operate so openly, but hosting a stolen data marketplace or hacking forum on the clear web rather than the dark web offers a number of benefits, most notably, greater user reach and accessibility. One of the main challenges encountered by operators of dark web markets and forums is attracting and sustaining a user base, due to their sites being difficult to locate and search for by design. A change of onion address – the direct access link to a hidden service – without sufficient advertising on popular dark web platforms could render a site practically impossible to find again. Conversely, clear web cybercrime sites are easily searchable by name, meaning they can retain users even if their domain changes.

Other motivations for hosting a cybercriminal marketplace or forum on the clear web include easier start-up and maintenance, and greater ability to protect the site from attacks by using anti-DDoS services. Although this strategy carries obvious additional risks, most saliently exposure to and seizure by law enforcement, it is surprising how long some clear web cybercrime sites remain active for. Longevity is particularly high if a site’s hosting servers are located in a jurisdiction that is either ignorant or permissive of the effects of cybercrime. 

Of course, this tactic is unlikely to work for especially egregious conduct, such as selling large quantities of heroin or sharing CSEA content. But many online shops offering stolen credit card information, login credentials, and other PII exist and thrive on the clear web, alongside forums dedicated to the sharing of leaked databases, hacking knowledge and tools, and techniques to commit financial fraud, among other topics.