Designing for Search Engines
But, you might ask, how do crawler-based search engines go about determining relevancy, when confronted with hundreds of millions of web pages to sort through?
They use algorithms, i.e. a set of rules that govern their spiders’ crawling techniques, indexing...