Every day Google always lists new web pages to be included in its search index, or what is commonly referred to as crawling. But of course not all new web pages on the internet are completely safe, for example sites that are spam, used for phishing, contain malware, and others.
Google Search Liaison, Danny Sullivan said Google already has a system to protect users from the threat of search results that lead to dangerous websites.
"We have a system designed to try to prevent websites and content like that from appearing in the top search results on Search," Danny said in a virtual discussion about Google Search, Wednesday (8/9/2021).
Sullivan said currently Google's search index contains hundreds of billions of web pages or the equivalent of 100 million GB. If printed into a book, these hundreds of billions of pages could stretch to the Moon up to 12 times.
To prevent malicious web pages from appearing on the first page of Search, Google deploys an automated spam detection system that works much like a spam filter in email. Based on Google data, in 2020 alone they found 40 spam web pages every day.
Once detected automatically, the spam page will be manually reviewed by the spam removal team at Google. This team will see if the page violates the Webmaster Guidelines and contact the website creator to resolve the issue.
In addition to detecting web pages that are spam, Google also has a system that can warn website owners if their site is hacked by hackers. This system will display a warning in the search results indicating that the website has been hacked.
"This is quite difficult if it is an important website. But the website itself that we know has been hacked, and maybe the owner is not aware of it," explained Sullivan.
"We will sometimes say in search results, 'Warning, this website may have been hacked', as a way to warn website owners and others that, yes this website is important, but you should know that there may be some issues."