注意看下面的日志,百度蜘蛛模拟人正常访问轨迹,可是IP出卖了它,正常人不可能访问一个页面,还切换IP的,而且是相邻网段。

查IP地址是 北京联通,RTBAsia非人类访问量甄别服务,真人概率4%。

猜测是鉴别内容真实度,可能和星火计划有关。

  • 61.135.169.8 - - [24/Mar/2016:11:34:54 +0800] "GET /a25633.html HTTP/1.1" 200 5612 "http://www.baidu.com/s?wd=www" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" -
  • 61.135.169.8 - - [24/Mar/2016:11:34:54 +0800] "GET /style/css/play.css HTTP/1.1" 200 5603 "http://www.XXXX.com/a25633.html" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" -
  • 61.135.165.9 - - [24/Mar/2016:11:34:54 +0800] "GET /style/js/jquery-1.4.2.min.js HTTP/1.1" 200 36885 "http://www.XXXX.com/a25633.html" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" -
  • 61.135.169.9 - - [24/Mar/2016:11:34:54 +0800] "GET /style/js.js HTTP/1.1" 200 1599 "http://www.XXXX.com/a25633.html" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" -
  • 61.135.169.10 - - [24/Mar/2016:11:34:54 +0800] "GET /style/js/ajax.js HTTP/1.1" 200 2477 "http://www.XXXX.com/a25633.html" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" -
  • 61.135.165.8 - - [24/Mar/2016:11:34:54 +0800] "GET /ck2.2/ckplayer.js?v6 HTTP/1.1" 200 4406 "http://www.XXXX.com/a25633.html" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" -
  • 复制代码