Spammers

Just had a spammer come through and post a bunch of crap. If anybody notices anything that looks fishy, you can always report it quickly by hitting the exclamation point icon up by the "Quote" button in the post itself. I'll get a notification and then I can take whatever action is appropriate. I have zero tolerance for spam. If you ever want to post something that you think might cross the line, shoot me a message and we can talk about it. Posting a link to your eBay listing where you are selling your old Symbolics machine would not be spam, IMO, but I guess spam is in the eye of the beholder.

Looks like spammers (like this one) are hitting the board - there's probably a captcha in the registration page; I can't remember... but it's probably still worthwhile for a human spammer to fill in a captcha once, then unleash a bot with the new account.
How about adding a captcha when submitting a post, for the first (say) 10 posts a user makes? That might make the job of running a spambot annoying enough to stop spammers.

Deleted. I'll see what I can't come up with. The registration already requires a valid email address, etc. I'm trying not to require a captcha on every post. IMO, that's too punitive. Let me see if I can't ratchet it up a notch, though.

Hmm, two separate spambots posting to the same thread

Yea, I try to whack these guys every couple days. Sorry about the lag. Sometime I'm caught up with other things and don't get a chance to do it. I have turned on the more aggressive forum registration options to try to mitigate it, but a few are still getting through.

Don't think captcha is effective enough. comp.lang.lisp uses captcha for every single post, but there is still a lot of spam there. But there there is no moderator as well, so there is the problem. I think many spammers have bots that can fill a captcha.

gugamilare wrote:Don't think captcha is effective enough. comp.lang.lisp uses captcha for every single post, but there is still a lot of spam there. But there there is no moderator as well, so there is the problem. I think many spammers have bots that can fill a captcha.

I don't know how you access cll (google?), but nntp access does not use captchas.

On a related note, I would be willing to participate as a 2nd-tier mod: somebody who could hide flagged comments until findinglisp had time to unhide/delete them. Better yet, I've seen some other forums autohide comments when a certain threshold is met; the text of such comments is replaced by something like "this comment has been flagged for admin review".

nuntius wrote:I don't know how you access cll (google?), but nntp access does not use captchas.

Yes, I use google, that's explained now.

nuntius wrote:On a related note, I would be willing to participate as a 2nd-tier mod: somebody who could hide flagged comments until findinglisp had time to unhide/delete them. Better yet, I've seen some other forums autohide comments when a certain threshold is met; the text of such comments is replaced by something like "this comment has been flagged for admin review".

This is a good idea.

I don't have much experience filtering spam from forums, but I've found it pretty easy to filter spam in my news reader just by checking for keywords (such as handbags, wristwatches, and the like). In this case, "wow gold" seems like it would catch all the spam so far. Would keyword filtering be possible in this case?

Paul Donnelly wrote:I don't have much experience filtering spam from forums, but I've found it pretty easy to filter spam in my news reader just by checking for keywords (such as handbags, wristwatches, and the like). In this case, "wow gold" seems like it would catch all the spam so far. Would keyword filtering be possible in this case?

There should be some library in PHP that does spam checking, since it is pretty simple. This is Paul Graham's idea.
The idea is not to test "keywords" themselves - someone could make a joke about "Wow! Gold! Look!" - not to mention you own post - but the rest of the post could tell that it is not a spam. The idea is to create a (hash)table of words with some score for each word. The score of a word is calculated more or less like this:

(defun score-of-word (word)
  (/ (number-of-spams-using word) (number-of-total-posts))

Off course, these numbers would be stored in the table themselves, this is just a scratch. Then you take some statistical average (or normal average) of the scores of each word in a post. Based on that score you can tell if the post is a spam or not. The simplest version of this idea wouldn't take 50 lines and would already be very efficient - at least according to Paul Graham.

I think I saw this in Paul Graham's book "ANSI Common Lisp" or "On Lisp", but I am not sure which. Anyway, for reference:

http://www.paulgraham.com/spam.html
http://www.koders.com/lisp/fid7F8E2D70F ... mtp+server

Let me look at some of the options. Generally, I try to login and look at the forum every 24 or so hours during the week, but often don't get to it over the weekend. I delete all spam and spammer accounts when I find it.

In looking at things behind the scenes, it's clear that there are a lot of bots registering users on the board. They don't generally get through the whole process, however. Right now, you need both a captcha and a valid email address to activate the account. Lots of accounts are getting created, which suggests that captcha isn't very effective and the bots are able to get past it. The requirement of a valid email address is what stops most of them.

I'll try to determine if there are any additional phpBB mods that will increase the hurdle further. If anybody has any suggestions, whether specifically phpBB-related or not, please feel free to put them on the table.

Okay, I just enabled another standard option that should reduce the amount of spam you guys see. Specifically, there is an option where if a user has less than N posts, any new post is held for moderation. After N posts, things are posted immediately, without moderation. I set N to 2. That should prevent you from having to see most of the spam. I'll continue to research some of the other options and mods to see if there are better overall solutions to actually eliminate whether the bots can register. After some quick Googling, it looks like the next version of phpBB includes a better (more difficult) captcha module that should at least slow things down until the spammers figure out how to break it. There are some other things that I'll try, too.

Okay, I just added a custom field to the registration profile. Basically, this is a little hack that makes the fields required to be filled out upon registration differ from the standard phpBB registration screen, which should help trip up bots. This means a spammer has to:

• Answer the new non-standard antispambot question
• Defeat the CAPTCHA
• Register with a valid email and use the validation link
• Generate at least a valid number of initial postings before automatic moderation turns off

I'm hopeful that these last couple of antispam features will keep most of the spamming out.

gugamilare wrote:not to mention you own post

Yes, that crossed my mind.

findinglisp wrote:

• Answer the new non-standard antispambot question
• Generate at least a valid number of initial postings before automatic moderation turns off

These two things are already paying dividends. There were no spambot registrations in the system this morning when I came in. Typically, I get 20+ per day, so having zero overnight was a good sign that the new spambot question is helping. Second, there was spam waiting for moderator approval from a spambot that registered a few days ago, but you guys didn't have to see it. Nice! This might be the trick, at least for a little while.

Excellent :) Good job.

Thanks.

My only concern with this is that the automatic moderation of newbie posts could lead to poor findinglisp being overwhelmed with a backlog of awaiting-moderation posts by both legitimate newbies and spammers, but it sounds like the traffic isn't high enough for that yet.

Yup, these changes have put a good dent on the spam. The bots can't easily register anymore (like zero bots registered after I changed that), and the initial moderator speedbump has kept a few bots that registered before the changes from getting too far before I catch them. In a few weeks, I figure I will have flushed out the bots and we'll be pretty-well limited to real people here. Ahhhhh... I love it when a plan comes together.

As for the 1-post hurdle, I think it's reasonable and shouldn't hold up anybody that much. Again, I'll try to look every 24-hours. Eventually, it makes sense to choose a couple of additional moderators. If somebody wants to put their hat in the ring for that, I'm open to it. Let me know and I'll consider a promotion for you.