I cleaned out the site IP blacklist over the weekend.

This removed the many thousands of IP addresses that had been blocked since I initially implemented some anti-spam safeguards several years ago.   Lets see what builds up from here.

There are currently three ways to have your IP address automatically banned:

1. Comment Spam   I'm currently checking inbound comments against a blacklist of banned keywords (Mostly various penis pills and prescription drugs).  
2. Forum Spamming  This site runs on PhpNuke and does not have the forum module enabled.  This doesn't seem to stop the multitude of automated robots that attempt to post things there.  Accessing any of the forum modules or forum admin modules (none of which are linked from the rest of the website) will get the offending IP banned.
3. Bad Robots  There are several places on the site where I've included a small invisible link to a dummy webpage.  This link is marked with all of the appropriate markers to keep legitimate robots away from it.   Any robot that does index it is immediately banned.

The links take you to the current lists of banned IPs. 

Update:  A few thousand blocked requests since yesterday.  There seems to be a single libwww-perl based botnet doing the forum spamming.  The comment spam also seems to be coming from a single source and is following its standard pattern of picking out a single page and spamming it exclusivlty for a day or two.

I added 1 additional safeguard.  There is now a mod_rewrite check to block requests that contain "http:" in the URL or query string.  This blocks a http injection attack that is attempting to spam for a russian chat website.

Update 2:  Now also blocking POST requests with no referrer via mod_rewrite.  You'd think the spambots would at least populate the referrers, but many don't.

Update 3:  Replaced the comment keyword checking with a call to Akismet.  That seems to be working quite well.

Update 4:  Akismet has been successful in blocking 100% of the spam with no false positives thus far.  I've gotten comment spam from 500 unique IP addresses in the past week.  The forum spammers seem to be isolated to a dozen IP which were all picked up and blocked in a day or two after implementing the above check.

Update 5:  A new botnet started getting through Akismet so I added some new hidden form fields validation to the comment page.  One I'm expecting to be blank and the other I'm expecting to have a specific value.  Still have Akismet in there if it passes these 1st 2 checks, but so far this has been 100% effective and nothing has made it to the Akismet layer.

This entry was posted on Saturday, April 19th, 2008 at 1:41 pm and is filed under Website. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.