Comment Spam discussed at Northern Voice

by on February 10, 2006

Maryam and I are at the Northern Voice conference. Today is “MooseCamp” and Matt Mullenweg (the founder of WordPress) is leading a discussion about comment spam. But, coming into the session he mentioned that he already has 90,000 Wordpress.com blogs (including mine). That’s very cool, but I heard yesterday from someone who works with MySpace that they are seeing 220,000 new MySpaces opening up EVERY DAY. Whew!

I also asked when his “pro” features are gonna come up (like the ability to change your design) and he said they are coming soon. He said that his service got popular faster than he expected so that he’s behind.

Regarding the spam, by the way, he says he’s seeing a whole new kind of spam. Social hacking spam. Spam that gets the blogger to think it’s a real comment. For instance, some of the comment spam that Wordpress is blocking today from getting on my blog is this one: “Do you provide a blog feed subscription for this blog so I can get it via email?”

He says this is actually fooling quite a few of the bloggers on his system (they mark it as “not spam” even though it is). Why? Cause it isn’t obvious spam.

There’s a whole war going on over getting onto bloggers’ comments. It makes me wonder if Russell Beattie doesn’t have the right idea by getting rid of comments altogether.

Matt designed a whole company to stop spam named Akismet. Matt is freaking brilliant. I love how his system blocks spam.

  • Why not use simple graphic word verification, like even Blogger does now? Maybe there is some better scheme coming down the pike, but it does seem awfully effective, for now. It doesn't add that big a burden on us users and it does seem to be *extremely* effective at blocking spam-bots. Has anybody had any experience where automated spammers get past graphic word verficiation schemes?

    If you don't know what I'm talking about, just go and try to comment on one of my Blogger blogs. Even better, send your spam-bot and have *it* try. (No fair trying to pretend that you're a spam-bot.)

    -- Jack Krupansky
  • Jack: Matt explained why: he has blind friends and even he can't read them a lot.
  • Askimet is hands down the best system for spam comments out there. I'm wondering though, what else is out there for non wordpress blogs?
  • Checking URLs in comments against a URL blacklist like SURBL* would solve this.

    * http://www.surbl.org/
  • Captchas aren't accessible even often for people that have 20/20 vision, I don't know how many times I have gotten Ticketmaster's captchas wrong and they also add a barrier to people commenting, which might make them not want to bother.

    Blacklists always seem to end up with sites listed that shouldn't be listed and the owner of the site has to take it up with the blacklist maintainer to get taken off the blacklist and often they aren't aware they are on such a list for a long period of time.

    I haven't made up my mind about Akismet yet, I don't like the fact that every comment has to be filtered through a remote service so I'd like to see more of Akismet integrated locally in a blog install and then for only fuzzies to be sent through to Akismet for further classification.

    That is why I used to like SpamKarma2 as it wasn't a remote service and it was pretty damn effective as well, I disabled it when an incompatibility was introduced in WP 2.0 development, I've been meaning to look into whether there is a 2.0 compatibile version or not yet.
  • Adding a text entry verification on my blog has reduced comment spam to ZERO - I was seeing hundreds a day once my site had been picked up by a few bots.

    I wrote a simple plugin to my wordpress blog that goes...

    "enter the nth word from the following sentence" - where nth changes every hit.

    However all my spam is now coming through trackbacks and pings. I moderate them to filter out. I do notice that I get peak hits: 50/60 spam trackbacks one day and then nothing for weeks!
  • /pd
    removing comments is a bad thing...then open conversations cannot occur. Not everyone blogs .. there are those who just dont blog, but rather furiously troll around and post legit comments and ideas..

    Blog Platforms need to have the methods to counter spam and also the naturally the idea of "blogspam" in a new legal way also needs to be crafted..
  • We had alot of Xbox.com users open up MSN Spaces (I think Jan 26th or 27th), the day the Xbox360 Live Integration with MSN Spaces launched.

    I feel like MSN Spaces get alot of hits a day.
  • I like telling kids at the talks I do that when I started blogging, back in 2000, comments didn't exist (well... barely existed). I was one of the first in my blogging circle to install a commenting script on my blog, and I remember how excited we all were about it. A new kind of user feedback, different from e-mails and referring links!

    Now, many people can't imagine a blog without comments. Well, we could have conversations from blog to blog without comments. We can use Technorati, PubSub, Bloglines and the like to see who is answering our blog posts. But that would require each blogger to be very active in checking out if what he/she wrote started a conversation somewhere.

    What I noticed too is that comments really speed up the conversations. And we also start talking about stuff or adding reactions which at first don't seem important enough to deserve a blog post. And sometimes these little things lead to bigger things.

    So, let's forget about "removing commenting" as a solution to comment spam. We can be smarter than that.
  • I turned the comments off on my blog a long time ago. I figure so many people have blogs these days, they can express love/hate for my post their blog and trackback to me. That way they still show up under the post, but they have to own their own comments.
  • MGB
    Comments on or off?

    Well... you can either talk with people or talk at them.

    Or to put it another way...

    People with a whole lot to say and yet leaves little room or time for anyone elses opinion, usually aren't worth listening to in the first place.
  • Mark: would you have posted that in your blog if Robert had comments turned off? :-)
  • http://poetslife.blogspot.com/

    I just wrote a review of your book on my blog. You can find it at the above address. As the book deserves, it is a very positive highlight of your work. Hope I got it right. If you see any errors, please let me know and I will fix them.
  • Riva
    If comments didn't have clickable links in the first place then the whole problem would be dramatically reduced already. Something anyone can just click on is going to be one big invitation for spammers to abuse, while it doesn't add any value to the blog.

    I can imagine that Robert is interested to see the sites or blogs of others who regularly post comments, but as a reader the only thing I care about are his posts, at a distant second would to review other people's thoughts based on their comments but in no way do I honestly care or ever bother to see what a commenter's link leads to. Disabling comment links for anyone but a blog's owner seems to be a reasonable middle ground.

    I also know that it also helps search engine rankings to be linked from as many sites as possible, but again I fail to see how this adds any value. If the only reason someone pitches in is because it earns them another link, then their contribution will be of little value to begin with.

    In my opinion the solution isn't to make it increasingly more difficult to comment (captchas, email verification, signing up, etc), but to simply remove any incentive for spammers to abuse them.
  • What about to point to other resources though? Commenting is about sharing information with the author and part of that information often involves a link to another resource so removing links would prevent that. Now if you allowed links but kept them as plain text it would still allow the references to be mentioned but at the same time the spammers would still to exploit it anyway.

    Heck even without links I think spammers would still spam comments, a lot of the spam comments I get already link to sites that don't even exist, which leads me to wonder why they even bother but it happens...
  • Jaseone, they register the domains that get through the filters later.
  • Christopher Coulter
    Hahhh, it can't scale. Russ has the right to do with whatever he wants with his blog, as do you, no Blog Police State rule about running comments. Just with you, you know, publishing a whole BOOK on 'conversations' and telling the world the value of "comments", if you were to take that tact, well you'd be a hypocrite. Which is why, me and other detractors, that see beyond the Bloggie and Redmond Reality Distortion Zone are but 'necessary evils'. But I find most bloggers that pretend preach "conversations" don't actually want that, they want fan-clubs. Least you aren't an agressive 'spam' moderator (credit where credit due), I am finding on some blogs, spam and critics are but one and the same. Funny that. ;)
  • Akismet works well to combat comment spam on blogs. But, so do algorithms that are trivial. For example, the following spam-catching algorithm is probably better than 98% accuate:

    if ( (number of links in comment >= 2) or (comment includes "casino" or "roulette") ) then it's SPAM
  • Riva: that's what nofollow is about...
  • Having no comments is the easy way out. My blog doesn't get enough traffic to warrant any kind of spam blocking measures. I have the ability to remove posts in case I get a spam or three.

    On my professional sites though, they all have commenting and get tens of thousands of unique visitors a day? Thus far they have all gotten 0 spam. I require users to be logged in to comment and it is a 100% effective solution. Does it prevent conversations? Well, the frequent articles the receive over 200 comments would disagree with that notion.
  • The "No Comments" suggestion is not what blogging is about. The essential part of blogging is 2 way conversation, as several people have said, this idea needs to go away.
    Spam is a fact of life, as long as businesses can get a result from force feed advertising then we will see it in our snail mail through our web based logging.
    Technology gave us this ability and it will give us the solution.
  • Scoble says... "Matt explained why: he has blind friends and even he can’t read them a lot."

    I'm sorry, but the word "lame", "lame", "lame", ... keeps echoing in my head.

    For the blind: just have a link next to the graphic labeled "Speak the code word". Granted, that won't help those who are blind *and* deaf, but they can't use text-to-speech aids anyway, right?

    And, does Matt *really* have blind friends who are reading blogs?

    I've only had two occasions where I misread the graphic words.

    lame, lame, lame,...

    Can anybody come up with a better reason?

    Has anybody Ask[ed] Dave Taylor if his simple math test blocks all spam?

    Face it, this is one area where even Blogger is *superior* to Wordpress. [Sorry, I just had to say it!]

    -- Jack Krupansky
  • Don
    I have found the combination of bad_behavior (also available for other than wordPress) together with Akismet (seems to work fine for me in WP 2.0.1) to be a good combination.

    Spam won't be necessary if we will all just plug stuff on our blogs in exchange for free stuff anyway ... oh wait that was yesterdays topic here :-)
  • Scoble says… “Matt explained why: he has blind friends and even he can’t read them a lot.”

    I’m sorry, but the word “lame”, “lame”, “lame”, … keeps echoing in my head.


    Ever heard of something called Section 508?

    For the blind: just have a link next to the graphic labeled “Speak the code word”. Granted, that won’t help those who are blind *and* deaf, but they can’t use text-to-speech aids anyway, right?


    Uhm for text-to-speech to work there would have to be some text and well that would kind of defeat the point of having a captcha...

    Face it, this is one area where even Blogger is *superior* to Wordpress. [Sorry, I just had to say it


    Well at least I am able to post this comment here first go unlike on your blogger site where it took me several times for it to load the comments page without throwing an error.

    Captchas are old, VERY OLD technology and are just annoying more than anything else.
  • Jack, even if you have captcha, there are weblog communication methods such as Trackback and Pingback which are DESIGNED for machine-to-machine communication, so anything like captcha would be useless for them.

    I really do have blind friends, and I get emails from blind users of WordPress whenever we break something.

    Also, anything like a math test or a "magic word" works fine as a one-off, but as soon as it becomes widespread enough to become a target it'll be defeated. The Akismet system scales (and improves) the more people use it.
  • As I said here, I think cancelling comments means you are tired of learning from your readers (shades of Dan Gillmor). While you are on my site you can check my comments with text entry verification (or mini Turing test) and threaded comments, which is the best things... well, since comments!
  • I too have blind readers so I stick to Akismet. It's caught everything so far.
  • Christopher Coulter
    Do you ever once goto something, ungeeky? Like maybe a Lecture Series/Academic Conference, Film Festival, Literary Event, Theater/Stage/Ballet/Dance? It's all bloggers conferences, blogger dinners, romps with Dave, techie conferences, Chris Prillio-styled parties, and junket book tours and start-up party spree's and inside-baseball techie power-chats. Frankly, it's just not all that interesting. Maybe if you'd partake of some culture sometime...or do something beyond showing your total ignorance for the world beyond the Seattle and Silicon Valley corridor.

    With all your power and access. You could be interesting. Could be.
  • Spam Karma 2 has saved my blog from trackback and comment spam. It works extremely well for me. 2587 blocked spam comment since I installed it in early December. http://unknowngenius.com/blog/wordpress/spam-ka...
  • I used Spam Karma on my 1.5 blog and it worked really well. Akismet hasn't let anything through (if I recall correctly) but it has stopped at least 2-3 legitimate comments from being posted.

    I guess you have the choice between two evils when designing spam-stoppers:

    a) make it tight enough that it won't let any spam go through, but risk it will stop a small number of real comments
    b) guarantee that it will not stop any real comments, but run the risk some spam might make it to the blog.

    I think the solution is diversity. We need more than one anti-spam system. If everybody uses the same one, that makes us vulnerable if the spammers find their way around it (think "biological diversity").

    I'd say the strength of Akismet is its centralisation: what one blog learns, another benefits from. (It can also be a weakness, of course.) Spam Karma is nice in the way that it is also a framework for which one can design custom-made anti-spam plugins (SK is a plugin made of plugins) and therefore leaves quite a bit of flexibility to the user about how it's going to stop spam. It makes the spam filter easy to be designed by the "collective intelligence" out there.
  • Jaseone:

    Regarding Spam Karma 2 (SK2): it's long been WP2 compatible now. As long as you use SK2.1 or up (SK2.2 is in final beta stage and should be out any day now), you'll be fine.
  • Here is an example of comment spam which just made it through Akismet, on this very blog:

    http://scobleizer.wordpress.com/2006/01/11/the-...

    Did anybody else spot it? It looks like a nice comment until you get the tv remote codes broken link at the end.
  • Steph: thanks, I removed that.

    Christopher: yes, I do (went to a play in Ireland, for instance) but I don't care to be read for my interest in movies or plays or sporting events or other non-geeky pursuits.

    Maybe you should start your own blog and write about those things?
  • Christopher Coulter
    Ahhh, but the solution to everything, doesn't end in "blog". Not for me.

    Sorry, I guess you are right, your blog, free to do what want. And I should just change the channel if I don't like. Just all like a one-hit-wonder played over and over, like weather in Southern Florida, need not even pay attention, know it already. I know what I will get going in, so nary a complaint needed. I guess for culture, will read the other Scoble. ;)
  • Innocent Bystander
    I think this is one of those arguments for more software diversity. Windows is popular and thus a frequent victim of bad acts. Easy to do because, as with all other things on computers, you only have to do it once to do it a million times.

    Consider that I write a different bit of blog software - something that looks like wordpress externally but is implemented with completely different technology.

    What are the odds I'll get auto-spammed? Pretty much zero (this is a lot of why I eschew Windows and other mainstream products - usually I just write my own - its about as much work as installing the popular stuff and much safer).

    Hey - different is good.
  • Captcha doesn't work, because anything GENERATED programmatically can be PARSED programmatically. There are entire PHP libraries just for figuring out what the text in a graphic is.

    Blacklists don't work. They have to be accurate, and they have to respond in minutes to new types of attacks.

    Wordpress's anti-spam rocks. Even just the default setup works wonders.

    And, yeah, the new "your feed doesn't work" comments are killing me. You just WANT to approve them. And, as far as "just turning off links" or "just turning off comments", that won't change anything for the positive. Turning off links doesn't stop people from trying to leave comments anyways. And turning off comments doesn't help conversations at all.

    I'm not 100% sold on Akismet yet (largely because we'd be using it commercially), but I love Wordpress's overall anti-spam stuff :)
  • It's taking me a while to dig through all of this, but you say "Matt designed a whole company to stop spam named Akismet", but it appears that the "company" is still Automattic, the same "company" that "produces" WordPress. The Akismet web site has a badge that identifies it as "An Automattic Production", whatever that means. I suspect it means something a little different than "This site powered by WordPress", but I don't know that for sure.

    Is it the intent of Automattic that Akismet will eventually be spun off to form an independent company, and *that's* what is being referred to in this post?

    The related issue is how tightly Akismet will be bound to WordPress. Is it WordPress-specific forever, or only in its initial incarnation? Is it intended that it will be rolled out to TypePad, MoveableType, (Blogger??), etc. users as well?

    Obviously much of that could/should be asked on the Akismet web site/blog, but I'd guess that they wouldn't mind using this comment list to promote themselves a little more to a wider audience.

    -- Jack Krupansky
  • No no no don't get rid of comments - they are the flavours that linger on the tongue after the wine has been swallowed (lordy but that sounds pretentious - sorry). Really, though, it is the posts with the most comments that draw one in, get one to read more, get one to go off exploring new parts of the web. And they force the blogger / reader to think, I believe, in a way that posts on the readers own blog never would. It is about conversation, not preaching, and a conversation has to be posted in conversational form.
  • Robert - I disagree with the Russell Beatie/no comments idea completely. Keep, 'em. Definitely. Here's why. And it was your post the other day that led me to that decision. Ironic, huh? ;o)
  • http://www.howseoworks.com/v7ndotcom-elursrebmem
    I am pleased to visit your site and find it,s contents and detail very much useful. [v7ndotcom elursrebmem] I have also tried from mine side for this contest. please visit mine link you will also find some very useful information.
    http://www.howseoworks.com/v7ndotcom-elursrebmem

    Thankx
    Kashif Mirza
  • Hi,
    we are testing a new free form-protection service (www.cerospam.com.ar), for blogs and for any kind of web site. It is easy to setup each form with this system, and it is very useful for protecting comment forms from spammers.
    It is based on captcha method. Until now it seems to work fine. No matter what kind of blog software you are using, this is not a plugin.
    Please, test it and do not hesitate to send us your comments!
    Thank you.
  • Like most bloggers, I've had waves of trackback, comment, and referrer spam hit my site from time to time. You think you're cruising free and easy until, wham!, you get blindsided by a thousand little spam entries that need to be deleted by hand, and then you have to go hunting for the perfect solution. Like many here, I've found Akismet to be a great solution for comment spam. Only maybe ten false positives and two or three false negatives out of 1,912 spams eaten.

    To combat referrer spam I use referrer karma, but you have to keep a closer eye on RK to prevent legit users from being blocked (RK tests the referrer for a real link to the blog--which can fail if the referring page is behind a login, such as with webmail services).

    For me, for now, turning off comments is simply not an option. Commenting has created relationships that simply wouldn't have existed without the sense of dialog that forms around posts. And not all commenters have blogs, and not all bloggers know to use trackback links (or care), and not all bloggers feel like turning an aside into a full-blown posts. Comments are a valuable part of the dialog.

    Requiring all "comments" to be blog posts with trackbacks is like saying, "I will address you from my soapbox, and you must be silent. If you wish to speak to me, you must ascend to your own box."

    When I blogged on a church shooting, the grandchild of one of the victims posted on my site. Then his mother contacted me via private post. When I blogged on Justin Berry, a personal friend of Justin's commented and contacted me privately. There are a few other examples of connections made and interesting relationships formed just from allowing comments and fomenting dialogue.

    Let the Akismets and Bad Behaviors and Spam Karmas thrive. They are protecting a valuable commodity: dialog.

    Regards,

    Rich
    BlogRodent
  • È luogo piacevole, devo dire! Buona fortuna a voi:)
  • Luogo abbastanza intersting ed utile. Buon
  • Jim
    Haven't you seen the same thing on Guestbooks and Forums these days, damn automatic spamming programs...
  • Nxvutopq
    Yes, you can see more about this here:,
blog comments powered by Disqus