CrawlProtect documentation

1) Unzip the downloaded file.

2) Upload on your website the crawlprotect folder (you can change his name, but you should not do it once the htaccess file is created), your CrawlProtect url will be: www.yoursite.com/crawlprotect/.

3) Go to your CrawlProtect url and then choose your language and follow the instruction to proceed to the installation.

4) Once logued, the first thing to do is to go to the administration page to create your htaccess file. Choose the parameters according your site informations and create your own htaccess file. Don't worry, if you had already a htaccess file in place, CrawlProtect will grab it's content and include it in the new file.

5) Then you have to add the proposed php tag on your site pages, you will find here some details indications on how to place it (it's like the CrawlTrack tag).

6) Then you can change your folders and files chmod level directly from the CrawlProtect interface. You just need to check that there is no disfunctionment of your site with these new values.

7) That's all, CrawlProtect is protecting your website. To test if everything is OK, try to visit an url like this : http://www.yoursite.com/index.php?site=http://www.test.com (replace yoursite.com with your website name). You should be blocked.

1) Unzip the downloaded file.

2) Upload on your website the new files in your CrawlProtect folder taking care that your ftp transfert software is set-up to overwrite the existings files (be carefull also at the chmod level of the files, check that the transfert has been successful).

3) Go to your CrawlProtect url, the upgrade process will start automatically.

4) Once logued, the first thing to do is to go to the administration page to recreate your htaccess file in order to take profit of the new features

5) That's all, CrawlProtect is updated and is protecting your website.

1) With your ftp software, copy the htaccess created by CrawlProtect.

2) Change to 604 the CHMOD of the htaccess created by CrawlProtect

3) Suppress the htaccess created by CrawlProtect

4) Go on CrawlProtect admin page and click on "Suppress the CrawlProtect htaccess" et validate the suppression. Your site is now back.

5) Contact CrawlProtect support joining a copy of the CrawlProtect htaccess giving issue.

1) Have a look at CrawlProtect logs (on the script homepage).

2) Take note of the blocage causes listed.

3) Waiting that the documentation is completed, contact CrawlProtect support giving the url of your site and the detail list of the blocages causes.

The process to list your folders and files use a lot of ressources, even more if your site is counting a lot of different folders and files. If you get a blanck page it's usually because the script has run longer than the maximum autorized time and has been kill by the server. You have different solutions to solve that issue.

1) You have access to your php parameters (it's the case for a dedicated server but not for a share hosting). You can then increase the value of max_execution_time in your php.ini until you find the right value.

2) If you cannot modify your php parameters, you can on the CrawlProtect admin page limit the number of folders and files to audit. You can audit only .htaccess, index, header, footer and config files, or only folders and files which are not with a high security chmod level or which have not been recently modified, ot limit to 2 levels of the folders tree. It's by testing different option that you will find the best setting for your site.

CrawlProtect to check his tag presence display your site homepage in an invisible Iframe, and it look like that you have a script to avoid the display of your site inside an Iframe. You have two solutions to solved that issue:

1) Remove that script (just for the time of CrawlProtect installation and tag presence check).

2) Modified CrawlProtect by replacing the header.php file (include folder) with that one. The tag presence check will need you to visit your site once that one is in place (to create the validation cookie).