CrawlTrack, webmaster dashboard.
Web analytic and SEO

CrawlProtect, your website safety.
Protection against hacking, spam and content theft

Two php/MySQL scripts, free and easy to install

The tools you need to manage and keep control of your site.

CrawlTrack, webmaster dashboard.
Web analytic and SEO

CrawlProtect, your website safety.
Protection against hacking, spam and content theft

Two php/MySQL scripts, free and easy to install

The tools you need to manage and keep control of your site.

CrawlTrack

      Presentation

      Demonstration

      Download

      Documentation

CrawlProtect

      Presentation

      Download

      Documentation

Community

      Support forum

      CrawlTrack blog

      Thanks

They like it

hubpages.com
The lastest release of Crawltrack now tracks indexed pages within Yahoo and MSN. And also it tracks the keywords and entry pages to your site from the search engines. Very cool. And did I mention that it is free?

mark8t.com
So, while Google Analytics (or scripts of that kind), do a wonderful job of tracking human visitors, they cannot give you any information on spider (or User-Agent) behavior. Truthfully, to measure, or make the most of your SEO efforts, you need to know when a spider is coming in, what pages they're requesting, and how often there coming back (if ever). With this in mind, we suggest you try out the free Robot Tracker CrawlTrack.

phpfusion-mods.net
It is quite easy to protect your phpFusion installation against any hacking attempt. For this you use a free script called Crawltrack which you can find at crawltrack.net. It comes with a detailed instruction on how to set it up for use with many GNU scripts including phpFusion.

forums.digitalpoint.com
Remember, a Web Analytics that is based on Javascript cannot track bots, so you will either need to implement this solution or get a bot tracker. Our recommendation for a software focused solely on bots is CrawlTrack, very impressive so far!

searchmaestro.co.uk
One way to monitor the search engines crawlers is to install a small, but powerful not to mention free program that is designed to track only the crawlers and search engine spiders. The software is called CrawlTrack and is available in a wide range of languages. It is deigned for specifically for use with PHP but it is a valuable piece of software to monitor and track the activity of search engines on your website.

branica.com
Learn how to use Crawltrack to track search engine spiders. It is a great piece of free software.

secure.agaresmedia.com
This is a useful little script for tracking crawlers, the other plus is it'll help block out hackersk.

Identified search engines crawlers

Google, Bing, Yahoo or Baidu are constantly enthusiasm to search the web for new pages to add to their index. You can used CrawlTrack to find out when the crawlers went on your site, but if you want more information on IP addresses and user-agents used by the crawlers you will find them here. All visitors on your site (human or crawler) send a query to your site wincluding two different parameterd. It's the IP address from where is send the query and the User-agent (in human case, it's the browser Internet Explorer or Firefox for example who supply that User-agent). These two informations will help you to idenfied these famous crawlers.

Google

Google use different crawler with a specific mission for each of them in theory. In practical, they often help each other.

GoogleBot

The better known, his mission is to collect datas for the Google index.

User-agent:
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

IP addresses used:
66.249.65.xx
66.249.68.xx
66.249.71.xx

Google-Adsences

He is used to know the content of the pages that carry advertisements adsence to allow the display of advertisements tailored to page content.

User-agent:
Mediapartners-Google

IP addresses used:
66.249.65.xx
66.249.68.xx
66.249.71.xx

Googlebot-Image

As its name suggests it is used to find images that will enhance the image indexes of Google.

User-agent:
Googlebot-Image/1.0

IP addresses used:
66.249.71.xx

Bing

Bing the Microsoft search engine has changed its name several times, it was formerly known as MSN search. It was for a short time Live search before taking the present name of Bing in 2009. The crawler used has not changed his name for a long time, it was always MSN Bot. But since October 2010, a new bot name Bingbot will progressively replace it.

Bingbot

User-agent:
Mozilla/5.0 (compatible; bingbot/2.0 +http://www.bing.com/bingbot.htm)

Adresses IP utilisées:
65.52.108.xx
207.46.16.xx
207.46.195.xx
207.46.199.xx
207.46.204.xx

MSN Bot

The good old MSN Bot has continued to serve to enrich the index of Bing

User-agent:
msnbot/1.0 (+http://search.msn.com/msnbot.htm)
msnbot/1.1 (+http://search.msn.com/msnbot.htm)

IP addresses used:
65.55.104.xx
65.55.208.xx
65.55.209.xx

Yahoo

Slurp Inktomi (Yahoo)

It's the main crawler used by Yahoo.

User-agent:
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)

IP addresses used:
67.195.37.xx
67.195.110.xx
67.195.111.xx
67.195.112.xx
67.195.113.xx
67.195.114.xx
67.195.115.xx
72.30.65.xx
72.30.78.xx
72.30.79.xx
72.30.81.xx
72.30.142.xx
72.30.161.xx
74.6.8.xx
74.6.17.xx
74.6.18.xx
74.6.22.xx

Baidu

Baidu, it's the chinese search engine. If you are looking for chinese visitors, it's the place to be.

Baiduspider

It's the crawler used by Baidu to crawl the web.

User-agent:
Baiduspider+(+http://www.baidu.com/search/spider.htm)

IP addresses used:
123.125.66.xx
220.181.7.xx
61.135.162.xx