The battle against spammers
Those of you who have been on our community for a while might have seen a huge increase in spam posts on the /talk forums over the year. After a bit of thinking I’m assuming we ended up in some black-seo spam software’s database. We’ve apparently solved the issue by integrating support for Akismet and StopForumSpam.com in the forums. So far this has worked great, but I’ll talk more about this below.
This whole thing got me interested into understanding how spam software works, who pays for that kind of stuff, etc… What I discovered after just a few hours of research is, to be honest, a bit scary. If you own a blog or a forum, hopefully this post will be useful for you.
Captcha and anti-bot questions are useless
First of all, it is important to understand that spam software has cracked all known Captcha techniques on most forum platforms. This is particularly true for popular forum systems such as phpBB, and this includes all “are you human” types of questions. The people at phpBB are basically in denial of this, and still recommend to use the Q&A plugin to stop bots from registering, but I’ve seen with experience that this doesn’t work.
There are two types of anti-bot questions: questions that google can answer (“What is the color of the sky?”), and questions it can’t (“type the 4 numbers in this sequence: ab4d56g7s”). phpBB’s recommendation is, of course, to go with the latter, since spam software already queries google to crack the “too easy” questions. But what I’ve learned by visiting some public knowledge websites for wannabee spammers (they call themselves “internet marketers”) is that advanced spamming software defeated most of this already (I won’t name any of those, don’t want to give free advertising to such a tool). My guess currently is that some of that software provides huge databases with answers to the subscription questions for hundreds of thousands of websites (I can’t be sure if this is some collaborative work by several spammers who share the work and benefits of that database, or if they pay for the service, or if it’s a trick involving fake *** websites to get random people to break the questions for you for free,…). And from what I could see with my own experimentation, these lists are probably updated extremely regularly, so that even changing your Q&A question every day doesn’t help if your forum’s become a target.
phpBB default tools are not adapted to efficiently prevent bot registration
The typical tools provided by phpBB to get rid of bot accounts are not adapted to the expertise level of modern spam tools. Ban by email doesn’t work as nowadays it is extremely easy for these tools to register many accounts on sites such as gmail or hotmail. Gmail itself allows you to virtually create as many email addresses as you can by just adding “.” characters wherever you want in your email address. So if you own a forum, you will see lots of spam email addresses looking like “jas.on.wit.ten.ab.c0.0.1@gmail.com”. It is basically impossible to stop that with the default tools in phpBB unless you entirely ban registrations from gmail.com.
Banning usernames is also, obviously, useless, since spammers come up with random ones. Finally, IP ban has proven to be majorly ineffective since the spamming software uses an army of proxies, with many of them being either infected computers, or computers in China and Russia where hosting is cheap and providers don’t seem to worry too much if you’re running some illegal spam business.
The conclusion is, phpBB is perfectly ineffective for stopping spam at the moment. WordPress doesn’t have the same issue (or not at the same scale) because it integrates with Akismet, a database that has a constantly updated list of spam keywords and urls. This blog doesn’t require any form of registration, and yet has close to no spam.
Akismet and StopForumSpam to the rescue
I think phpBB should have Akismet integration by default, but it is not the case so we had to install a MOD to handle this (AntiSpam ACP). This worked great, and started stopping the spam posts, moving them into the moderation queue. But that was also too much work, as nobody wants to manually review hundreds of spam posts per hour and ban the offending bots (we have almost 1 bot registration attempt per minute as I type this).
So in addition to Akismet, we enabled support for StopforumSpam.com at registration (from the same mod). Basically, any user trying to register with an IP, a username, or an email that was recently flagged as a spammer on that collaborative database, will be rejected from our site. This, in combination with Akismet, as far as I can tell, has stopped 99% of the bot traffic on our community, and our moderators can finally start to breathe (I still have to add that MOD on the wagic forums which are in a terrible state right now…). You’ll note that it’s nothing more than what we have with phpBB (IP/username/email bans), except this time, the ban is proactive and relies on a collaborative database, which allows us to automatically ban the bot before it even registers.
Winter is coming
Does it mean the war with spam is over? Absolutely not. First of all, the spam tools I’ve found are constantly evolving to take into account the latest antispam techniques. I really want to say that every single forum owner on this planet should add StopForumspam support to their registration system, but then I also know this would make it the next major target to defeat for spam software.
But don’t be fooled, they are already working on it. On the spammers forums I visited, people are regularly discussing ways to trick services like Akismet or StopForumSpam, simply by getting their IPs and or urls “unflagged” using various tricks. For example one of their techniques for Akismet is to create dozens of bogus wordpress blogs, post their spam comments there, and mark is as “Ham” (false alarm spam) until Akismet moves them away from the spam list.
Spammers also leverage the StopForumSpam website, by adding the list of top StopForumSpam contributors (here) to their own “blacklist” in order to not make it to the spammers database(by never accessing the known StopForumSpam contributor websites). Some of them even use the StopForumSpam list of spammers IPs/emails/usernames as a way to reverse-find unsecure forums where spammers are active, and where their own spam will go through.
In parallel, as I mentioned above, Spam software “companies” are already working on systems that will automate all of this, to guarantee Akismet and StopForumSpam won’t be as effective in the near future.
And even if techniques like StopForumSpam stay effective, even if it reduces the overall amount of spam on your own forum, the money to be made by those techniques is so huge, that it just means the clever spammers will get even richer, with the decrease of competition for them (which somehow doesn’t make me feel good). Some of these people already laugh at systems like StopForumSpam, claiming it’s easy to defeat, and since I can myself see many ways this could happen, I’m inclined to believe them.
But I digress, we solved our current issues with spam on /talk, so enjoy it while it last ๐
You forgot to put an Insert more tag :p
Thanks!
thanks
im tired of random post about NFL jerseys or UGS BOOTS, etc…
Well, hopefully now they’re all gone, but don’t hesitate to let the mods know if you find more.
will do ๐
its across all forums sadly..some bots are also selective on where they post..most of the spam on the forum *if you click on my name* is in the psp section of the forum spam still gets into other sections but not as bad as the psp section and its mostly NFL jerseys heh..but also sometimes gibberish, ugs boots, or just a simple hello and good bye
or what really annoys me is when spam spams in a necro topic.. lol
Please help us putting it in the wagic forums
Hey guys you wouldnt believe how easy it is to make money buying/selling stock! Using the tools i found on…
Haha just kidding! Spam is really annoying and if people actually fall for such tricks they deserve to lose their Money.
Thug Life
You sir, deserve an epic BANHAMMER delivered to your ball sack. <_<
less ***, more exploits
nah..
Sadly its the internet, someone will always find a way around. Thanks for cleaning the forums though ๐
Yes. Spam drives me INSANE!! lol
Thanks wololo for cleaning up /talk as well as taking the time to post the very informative article. ๐
Let’s not forget to thank the moderators, they are the ones who have suffered the most from the spam situation ๐
heh, fighting spam is like sony fighting this community… ironic it is, but what can you do? They find a loophole, shut em down… wait for the next opening and do it again. Thankyou for dealing with the headache on both sides wololo!
well, thankyou to all the mods you have inlisted as well.
Good news and good work wololo.
Thanks wololo, this was an interesting post!
Its not as bad as you think it is, Google is getting smarter, people are getting smarter (arguable). So the target is a lot smaller == less money.
Its like hackers and “protectors” you can’t 100% protect, but if nobody cares, nobody breaks your console.
The *** idea, is actually pretty genius, its even cheaper then Indian low-payed jobs.
Now if only they had a way to battle trolls..Your blog is rather full of them, Wololo ๐ก
er..didn’t mean the angry face, meant : x ** as in ‘hushed face’
“Winter is coming.”
Reminded me of the Game of Thrones TV Series, and is the common phrase of folks from the House of Starks in Winterfell.
*patiently waiting next year’s season ๐ *
Thank you Wololo for sharing this information.
It is the Game Of Thrones…
I’m waiting for the next book in the series, but at the rate G RR Martin is going the tv show will catch up to the books
wololo = sony behing the scene
I use http://areyouahuman.com on my site and it seems to greatly deter spambots
Are you a spambot?
you have spammers in your forums because phpbb is like a 15 years old forums software.
I manage a mail server for ~1000 users, and I use various well-known anti-spam services block lists as “weights” in mail rules. I can’t take their lists for The One Truth (this creates false positives since clueless admins easily end up in those lists temporarily), but they way in as one of several factors. This catches well over 90%. Still, that means 100-1000 spams that would get through every day in my case, which is of course still a disaster.
The key I have found is to come up with a couple of your own scripts/rules/factors that take care of the rest. And then, DON’T SHARE THEM! They are easily defeated if studied, and could probably be defeated by analysis (bruteforce trial-and-error), but since a minor target like our mail server is of little interested, this has worked wonders. I’m proud to say currently about 99.98% of the spam is rejected, and AFAIK not a single false positive after years of use. Every now and then I tweak them a tiny bit when something manages to slip by the checks by pure luck several times, but this is really minor work.
It’s doable, not even that hard, and your users will love it ๐
(In a way it’s “security by obscurity”, but as far as spam goes this is definitely good enough.)
For remotejoy on psv… Do not try psplinkusb or do … But try psplink and have it setup for nethost … Try it on a psp first to get it going then launch it via vhbl or cef and on pc use nethost and pspsh instead of usbhostfs … Anyway then load remotejoy prx and maybe loadvsh … Have phun.
This site is just weak and *** that’s all. It’s like a cardboard box filled with styrofoam pretty much garbage.
How about you find a more interesting yet stable website?
I’m sure wololo would love to have your expertise on the matter. Why don’t you send him a pm with your ideas?
*** Sandwich, I’ve been managing this site by myself for the past 4 years despite having a pretty busy personal and professional life, I started from nothing and 4 years later this site is *the* reference in psp and vita security.
Think what you want, but I’m pretty sure I don’t need your opinion on what makes an interesting website ๐
i happen to think this website and the content is very interesting and helpful ๐
its a lot of work to manage a site a lot of work to code them too heh
I can revamp this whole website into something extraordinary adding awesome features and what not. So whadda say? I been designing websites since the age of 14.
I say being a designer and having useful content are not the same, so you are kinda changing the subject. What type of features do you have in mind?
Im not sure if he meant the actual building blocks of the site themselves or the sites contents. O_o.
why do i feel this is the same story when it come to adds and my add blocker???
Yeah, it’s probably a similar situation, although I would say ad networks that try to bypass ad blockers are bad ones… I’m expecting google adsense for example to respect people’s decision when it comes to blocking ads. but I might be wrong
New way of finding spambots ๐ , just do like this:
Wrait thou trhird lutter ov Sssprinkel
Correct Answer : R , because Sprinkle ๐
And yes, this is just a joke, but perpharps it works ?
No one besides me bought any asbestos jerseys?
Nah, I’m more into viagra myself
Being a member board mod from RPGamer, I can say that spammers are sneaky little cockroaches; we’ve been fighting a war against them for about 2 years now. Their fav trick now (because they can’t post threads(You have to make so many posts to make a thread if you are new) and most of them are blocked by a board plugin that uses the two things you mention) is to make spam profiles. They put spammy things in sigs and member messages (we use Vbull btw) and in their profiles.
Luckily it’s only 1 a week now. Only thing we’ve been able to do is clear out the spam and ban. I suggest you keep your eyes open for that kinda bull in case it shows up.
great site btw
here my runtvtime.sh script i would like to see a php imageslider from
./runtvtime.sh
#!/bin/sh
echo
tvtime &
sleep 4
echo
arecord -D hw:1,0 -c 2 -r 32000 -f S16_LE -t wav | aplay – &
this is on ubuntu
allso try remotejoy-sdl and watch tvtime pic in pic on as many clients as u wish
Oh please
How do i create a php script from an application virtual frame buffer ?
thanks anyway ๐
hello,
let me rformulate
how do i export a pc app or any app standard IOs android pc mac linux win osx towards a psp and just one app (being tvtime xor xbmc xor mythtv or yavdr or any ?.? ? or the whole desktop ?
ifso are there then allso desktop apps for psp ?
i know of one way between colinux or any ubuntu and xming-w32 x server for windows.
by doing export DISPLAY:=ip:port of xming or any x server
then launch the app with tvtime&
i mean to ask simply for psp and ps3 … on ubuntu ps3mediaserver tvtime bindings with audio ๐
bye nd thanks
People in 2nd world and 3rd World countries(also in Ch.ina) actually have jobs solving massive amount of Captcha. They get for as low as $0.25 for an entire week of captcha solving. And in their economy its a huge number. So I don’t see anyway around that except a Captcha solving limit(And if there was I myself would be locked out of that website).
O wololo,now you know how far the internet goes.did you know that the google is a company that is nonprofit,but google is a illegal company because it uses spam to get its oun moneys and nobodies noticed that,they also serch any information about you until they know all about you,and the informations that they collect about you are almost as naked if anyone search it is “like” facebook but with more personal life and hakers and tie-fs
ps:life is a wich