Search Engine use special software called Spiders to crawl the website and find new links and websites. Search Engine Spiders also know as robots, crawlers, agents etc. Search Engine robots are different from one Search Engine to another. Some of the Major Search Engines and their agents are as follows:
Search Engines
|
Agent/Robot Name |
Google
|
Googlebot/2.1 |
Yahoo
|
Yahoo! Slurp |
Bing
|
bingbot/2.0 |
Alexa
|
ia_archiver/8.9 |
No comments:
Post a Comment