The Deep Web, what exactly is that?

Let’s talk Deep Web, again, because there is a lot of misunderstanding about this phenomenon.

First of all, the Deep Web is not the same as Tor and it is certainly not by definition criminal or terrorist. The Deep Web is simply the part of the Internet or in this case, the part of the world wide web that is not indexed by any search engine. That’s it.

In other words, the part of the Internet that search engines cannot see.

The reality is, that the Internet only covers a small part of all available information. The Internet consists of many different services, of which HTTP is just one. And sources show that about 99% of that world wide web is in fact deep web. You will not find it using a search engine. Disapointing is it not?

The deep web is NOT hidden. The deep web is NOT invisible. The deep web is NOT the same as the dark web. The web does not have a colour.

There are many many reasons why search engines cannot see into the deep web. Chris Sherman and Gary Price have written an excellent book about the deep web in which they list many reasons why search engines cannot see certain websites, or don’t want to see certain websites, or why certain websites do not wish to be seen and how.

There are many examples of the deep web. Think about all your paid subscriptions on the web. Think about dynamic data such as weather data or stock markets. These change so quickly that a search engine crawler simply cannot keep up. That’s why search engines have separate services to cover this kind of information. Think about websites that are so obscure that no one links to them.

There is a few lovely ‘underground’ networks that are hidden from plain sight and are part of the deep web domain. Tor is just one of them but there are many more. Tor by the way, is not by definition a criminal or terrorist network. It was intended to protect writers from threats by allowing them to publish almost anomously. Needless to say that criminals quickly jumped onto the bandwagon. In my humble opinion, investigative journalism sometimes for a very good reason have to hide their identity. Unfortunately, criminals do that too. Thus, Tor is in my humble opinion about 98% serious crime, the remainder decent advocates of free speech.

It will all be discussed at the forthcoming RIS OSINT Pathfinder XIX training programme. You will learn what the benefit is of the deep web in the overall intelligence effort, and learn how to run Tor and other hidden networks.




Leave a Comment