今天看了一下网站的访问日志,看来原来很有很多搜索引擎的机器来上来抓东西,粗粗看了一下又一下这些
“Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07
YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/;
“Baiduspider+(+http://www.baidu.com/search/spider.htm)”
Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)”
“Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 2 subscribers; feed-id=7763861123115737748)”
Googlebot/2.1; +http://www.google.com/bot.html
Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html