The spam! The spam!

Question

I understand that the spam this site receives daily is currently handled by users flagging it and that works and it's great. But I can't help but feel that there should be some automated way of handling what one can clearly tell is spam by just the title alone.

What I would like to see (pie-in-the-sky) is some mechanism for detecting obvious spam and slowing down its submission to the site and then having it hidden from the main page after a short time or even deleted entirely. Whether it is keyword-based or uses a Bayesian filter or whatnot would be decided by whoever decides to implement it.

Of course there is the caveat that machine detection isn't perfect, and that there will probably be an "arms race" between spammers and the detection system, but I do feel that it is possible to make spamming expensive enough while not unduly affecting legitimate users of the site.

Answer 1

It's already a work-in-progress. Stack Exchange is using a bot called SmokeDetector (affectionately known as Smokey) which goes through some fairly complex rules to detect spam. You can visit it here:

http://chat.stackexchange.com/rooms/11540/charcoal-hq

It currently merely detects, and (if requested) sends a message to anyone who subscribes. Subscribed members can then flag the spam for moderator attention, or moderators can delete it immediately.

It does get things wrong from time to time, so that is why it doesn't automatically delete spam. For example, on the English Language part of SE recently, someone asked about expletives (eg. f--k) which were picked up by Smokey as as "offensive text detected in body of post". However in this case it was a legitimate post enquiring about whether certain expletives were appropriate in writing a novel.

Of course, there are other posts using words like that which are just for dating-site spam.

Training Smokey is an ongoing effort by a fairly dedicated team of coders. :)

Probably the closest you could get to automating things would be for a number of users, who are frequently logged on, to subscribe to the notifications. Then they could quickly flag as spam such posts.

---

As I write this, I got a couple of notifications from Smokey. One was an hour ago and one 55 minutes ago (I was eating dinner at the time). In that time the community had already flagged it, and it was automatically deleted due to the number of flags. So the current system works reasonably well.

---

Shortly afterwards I got a notification from Smokey. After only 22 seconds, a post had two downvotes. After 1 minute it had 4 downvotes. I had to hurry to delete it before the system got to it first. :)

---

IMPORTANT

Don't downvote spam: flag it as spam - that activates the Stack Exchange automated system.

Answer 2

Arduino indeed gets more spam than usual lately: it is currently the 5th most-spammed site on the SE network, according to Metasmoke data. However, it still gets only about 1/3 of what Drupal site gets, and Drupal has been dealing with this plague for several years by now. There isn't an end in sight.

Automatic blocks are in place. Tim Post points out that the amount of spam blocked is of orders of magnitude more than we see on the sites. Roughly speaking: users flag about 100 spam posts per day, while SE blocks about 10000 spam attempts per day.

The most recent proposal for user-driven content-based blocks is in A centrallised (multi-site) community-controlled globbing blacklist. It also got a reply by Tim Post (who is responsible for many of the anti-spam systems that SE deployed), which suggests that consideration of additional measures is ongoing.

Answer 3

Spam is a good thing and should be cherished :)

It means we are being noticed and deemed worthy of spamming.

The more spam we get the bigger and more important a site we must be.

I think the "come out of Beta" decision shouldn't be made on how many questions / answers you get, but on how much spam you attract ;)

_{(By the way: for those without my warped sense of humour, that's a joke :P )}