Those of you who have been on our community for a while might have seen a huge increase in spam posts on the /talk forums over the year. After a bit of thinking I’m assuming we ended up in some black-seo spam software’s database. We’ve apparently solved the issue by integrating support for Akismet and StopForumSpam.com in the forums. So far this has worked great, but I’ll talk more about this below.

This whole thing got me interested into understanding how spam software works, who pays for that kind of stuff, etc… What I discovered after just a few hours of research is, to be honest, a bit scary. If you own a blog or a forum, hopefully this post will be useful for you.

Captcha and anti-bot questions are useless

First of all, it is important to understand that spam software has cracked all known Captcha techniques on most forum platforms. This is particularly true for popular forum systems such as phpBB, and this includes all “are you human” types of questions. The people at phpBB are basically in denial of this, and still recommend to use the Q&A plugin to stop bots from registering, but I’ve seen with experience that this doesn’t work.

There are two types of anti-bot questions: questions that google can answer (“What is the color of the sky?”), and questions it can’t (“type the 4 numbers in this sequence: ab4d56g7s”). phpBB’s recommendation is, of course, to go with the latter, since spam software already queries google to crack the “too easy” questions. But what I’ve learned by visiting some public knowledge websites for wannabee spammers (they call themselves “internet marketers”) is that advanced spamming software defeated most of this already (I won’t name any of those, don’t want to give free advertising to such a tool). My guess currently is that some of that software provides huge databases with answers to the subscription questions for hundreds of thousands of websites (I can’t be sure if this is some collaborative work by several spammers who share the work and benefits of that database, or if they pay for the service, or if it’s a trick involving fake *** websites to get random people to break the questions for you for free,…). And from what I could see with my own experimentation, these lists are probably updated extremely regularly, so that even changing your Q&A question every day doesn’t help if your forum’s become a target.

phpBB default tools are not adapted to efficiently prevent bot registration

The typical tools provided by phpBB to get rid of bot accounts are not adapted to the expertise level of modern spam tools. Ban by email doesn’t work as nowadays it is extremely easy for these tools to register many accounts on sites such as gmail or hotmail. Gmail itself allows you to virtually create as many email addresses as you can by just adding “.” characters wherever you want in your email address. So if you own a forum, you will see lots of spam email addresses looking like “jas.on.wit.ten.ab.c0.0.1@gmail.com”. It is basically impossible to stop that with the default tools in phpBB unless you entirely ban registrations from gmail.com.

Banning usernames is also, obviously, useless, since spammers come up with random ones. Finally, IP ban has proven to be majorly ineffective since the spamming software uses an army of proxies, with many of them being either infected computers, or computers in China and Russia where hosting is cheap and providers don’t seem to worry too much if you’re running some illegal spam business.

The conclusion is, phpBB is perfectly ineffective for stopping spam at the moment. WordPress doesn’t have the same issue (or not at the same scale) because it integrates with Akismet, a database that has a constantly updated list of spam keywords and urls. This blog doesn’t require any form of registration, and yet has close to no spam.

Akismet and StopForumSpam to the rescue

I think phpBB should have Akismet integration by default, but it is not the case so we had to install a MOD to handle this (AntiSpam ACP). This worked great, and started stopping the spam posts, moving them into the moderation queue. But that was also too much work, as nobody wants to manually review hundreds of spam posts per hour and ban the offending bots (we have almost 1 bot registration attempt per minute as I type this).

So in addition to Akismet, we enabled support for StopforumSpam.com at registration (from the same mod). Basically, any user trying to register with an IP, a username, or an email that was recently flagged as a spammer on that collaborative database, will be rejected from our site. This, in combination with Akismet, as far as I can tell, has stopped 99% of the bot traffic on our community, and our moderators can finally start to breathe (I still have to add that MOD on the wagic forums which are in a terrible state right now…). You’ll note that it’s nothing more than what we have with phpBB (IP/username/email bans), except this time, the ban is proactive and relies on a collaborative database, which allows us to automatically ban the bot before it even registers.

Winter is coming

Does it mean the war with spam is over? Absolutely not. First of all, the spam tools I’ve found are constantly evolving to take into account the latest antispam techniques. I really want to say that every single forum owner on this planet should add StopForumspam support to their registration system, but then I also know this would make it the next major target to defeat for spam software.

But don’t be fooled, they are already working on it. On the spammers forums I visited, people are regularly discussing ways to trick services like Akismet or StopForumSpam, simply by getting their IPs and or urls “unflagged” using various tricks. For example one of their techniques for Akismet is to create dozens of bogus wordpress blogs, post their spam comments there, and mark is as “Ham” (false alarm spam) until Akismet moves them away from the spam list.

Spammers also leverage the StopForumSpam website, by adding the list of top StopForumSpam contributors (here) to their own “blacklist” in order to not make it to the spammers database(by never accessing the known StopForumSpam contributor websites). Some of them even use the StopForumSpam list of spammers IPs/emails/usernames as a way to reverse-find unsecure forums where spammers are active, and where their own spam will go through.

In parallel, as I mentioned above, Spam software “companies” are already working on systems that will automate all of this, to guarantee Akismet and StopForumSpam won’t be as effective in the near future.

And even if techniques like StopForumSpam stay effective, even if it reduces the overall amount of spam on your own forum, the money to be made by those techniques is so huge, that it just means the clever spammers will get even richer, with the decrease of competition for them (which somehow doesn’t make me feel good). Some of these people already laugh at systems like StopForumSpam, claiming it’s easy to defeat, and since I can myself see many ways this could happen, I’m inclined to believe them.

But I digress, we solved our current issues with spam on /talk, so enjoy it while it last 🙂