The dynamic web is growing so fast that one billion pages are added or changed every day, creating a massive problem for both search engines and security systems. Simon Kearney reports.
Cisco Systems estimates that the exponential growth of the dynamic web, or web 2.0, since 2005 has been responsible for the number of Internet web pages growing to 45 billion. Remarkable when you think that there are only 1 billion PCs in the world and 6 billion humans. What’s more remarkable is that of these 45 billion pages, 32 billion are uncategorised, creating what Cisco calls, the “Dark Web”. The company’s head of security product marketing Ambika Gadre believes the unclassified part of the Internet will be the vast majority – 95 per cent – by 2015.
Cisco Systems, other Internet security companies and search giants are grappling with the possibility that crawling the Web looking for content to classify and signpost is increasingly difficult with the growing number of pages. The dynamic web is making many blacklist filters constantly out of date due to the use of what are called anonymising proxies which mask the true content of a site.
As a result the security industry has been adopting a reputation approach, similar to credit scoring, that operates in real time to make judgments about the source of web pages before flagging them as a potential threat.
Tom Gillis, Cisco’s head of security management, says this approach is now fast enough for the user to be unaware that the filtering software is working in the background.
» This story continues on page 2. Please click here to read
please sign in to rate this article
45706