I’ve lost control of my comments…

You don’t see it, but some of my comments are getting deleted by Akismet, the spam blocker that WordPress.com uses. Osman S Borutecene says he’s noticed comments being deleted here seemingly randomly. Truth is commenting in blogging is broken and is getting worse. In just the past 24 hours I’ve manually had to delete about 1,000 spams and in my already blocked page in WordPress.com there are about 10,000 spams. I have absolutely no way to go through and look for false positives, or real comments you’ve left that have been deleted automatically cause they triggered the spam system.

I do notice some trends, though. If you don’t use your full real name (two words like “Robert Scoble”) you’ll get deleted more often. Most spammers use one word names, or three word names. Also, if your post has URLs in it, you’ll be more likely to get deleted.

If you notice your comment gets deleted if you can give me the approximate time you posted it I can usually dig it out of the deleted bucket. Sorry for the troubles, but the comment spam problem is getting dramatically worse. You don’t see it because I also moderate comments from first-time commenters (they wait on a page for my approval, which can take up to a day to get approved, but usually are approved in an hour or two — that’s why spam isn’t getting onto my blog, but I sure see it).

If I ever end up closing my comments this will be why. It’s unbearable at 1,000 spams a day. I can’t imagine how life will be if the spammers start writing more human-like comments and/or if they start moving up to 10,000 or 100,000 per day (Akismet blocks millions of messages across all the blogs every day).

Akismet is both looking at patterns of messages, as well as using the community to block spam. So, if a blogger marks something as spam that really isn’t it’ll get deleted system wide until enough bloggers mark it as not spam. Funny enough my own comments were getting deleted last week too. So it happens to me too!

Comments

  1. I found a full proof way to get around the spam blocker by trial and error. Do not put any line breaks in the comment. Make it one very, very long line and Askimet will not flag it, and it will get wrapped.

  2. I found a full proof way to get around the spam blocker by trial and error. Do not put any line breaks in the comment. Make it one very, very long line and Askimet will not flag it, and it will get wrapped.

  3. No, I mean no line breaks or carriage returns. If you have a comment that doesn’t pass the spam filter, then go back and repost the comment removing all the line breaks. The same exact comment will now pass the filter with no problem.

    I did not mean to say that every comment with line breaks gets flagged. I was stating that comments that do get flagged can make it past the filter if you repost it sans line breaks, meaning the return key. URLs need not be on a new line, a space before and after a URL suffices to get it to autolink.

  4. No, I mean no line breaks or carriage returns. If you have a comment that doesn’t pass the spam filter, then go back and repost the comment removing all the line breaks. The same exact comment will now pass the filter with no problem.

    I did not mean to say that every comment with line breaks gets flagged. I was stating that comments that do get flagged can make it past the filter if you repost it sans line breaks, meaning the return key. URLs need not be on a new line, a space before and after a URL suffices to get it to autolink.

  5. Just being a Z-list blogger like me 5-10 comments get past the Junk filter and I have to manually delete them. I could not imagine having 1000 comments to manually clean.

    If I did, I’d close comments.

    Ahh… here is a great opportunity for some enterprising developer – creat a spam proof commenting system that can be plugged into WordPress, Typepad and the such.

    It is sad though that there are jerk-offs out there who have nothing better to do than create spam-bots.

  6. Just being a Z-list blogger like me 5-10 comments get past the Junk filter and I have to manually delete them. I could not imagine having 1000 comments to manually clean.

    If I did, I’d close comments.

    Ahh… here is a great opportunity for some enterprising developer – creat a spam proof commenting system that can be plugged into WordPress, Typepad and the such.

    It is sad though that there are jerk-offs out there who have nothing better to do than create spam-bots.

  7. Comment administration is totally broken. There are some clear instances in WP for example when I want to do a mass admin and am not allowed to. Like, I’ve got a bunch of old spam on a post I came across yesterday and can I mark them all spam in one click? Nope. It’s just silly.

  8. Comment administration is totally broken. There are some clear instances in WP for example when I want to do a mass admin and am not allowed to. Like, I’ve got a bunch of old spam on a post I came across yesterday and can I mark them all spam in one click? Nope. It’s just silly.

  9. No wonder that most of my friends that commented on some of my entries said that i didn’t approved their comments. Too bad. :(

  10. No wonder that most of my friends that commented on some of my entries said that i didn’t approved their comments. Too bad. :(

  11. You can annoy legit users a bit and add some decent captcha (by decent I mean readable). I know I’m not telling you anything you don’t know and yes captcha can be beaten, but I had a blog that I don’t even know how it became popular among spammers (because it had very few legit readers) and it was getting over 100 spams a day. I added a simple captcha and the spam went down to zero.

    Before closing comments just because of spam, that’s something I would certainly consider.

  12. You can annoy legit users a bit and add some decent captcha (by decent I mean readable). I know I’m not telling you anything you don’t know and yes captcha can be beaten, but I had a blog that I don’t even know how it became popular among spammers (because it had very few legit readers) and it was getting over 100 spams a day. I added a simple captcha and the spam went down to zero.

    Before closing comments just because of spam, that’s something I would certainly consider.

  13. Not just blog comment spam. I am getting much more spam in my inboxes. I guess spammers had some kind of technological breakthrough….

  14. Not just blog comment spam. I am getting much more spam in my inboxes. I guess spammers had some kind of technological breakthrough….

  15. I certainly wish WordPress or Akismet would introduce a CAPTCHA option for comments. That would make it a LOT harder to spammers to leave comments in the first place.

    Robert, perhaps if you suggest it to the folks at WordPress, they might listen?

  16. I certainly wish WordPress or Akismet would introduce a CAPTCHA option for comments. That would make it a LOT harder to spammers to leave comments in the first place.

    Robert, perhaps if you suggest it to the folks at WordPress, they might listen?

  17. I was trying yesterday to write a program to automatically turn off comments on all posts that are older than 60 days just to make managing spam less work.

    I’m at 1500 spam in akismet a day now.

  18. I was trying yesterday to write a program to automatically turn off comments on all posts that are older than 60 days just to make managing spam less work.

    I’m at 1500 spam in akismet a day now.

  19. Dang, that is pretty painful – I had no idea spam is so heavy on blogs like this. Mine own is a teeny little social livejoural, so I don’t really seem to get any.

  20. Dang, that is pretty painful – I had no idea spam is so heavy on blogs like this. Mine own is a teeny little social livejoural, so I don’t really seem to get any.

  21. Thanks for the tools Phil, Spam is getting worse all the time. I think i heard Matt (wordpress founder) in a CNet interview say something along the lines that he doesn’t use e-mail anymore. ??!! There isn’t that much spam is there? Say, engtech, tell me when you get some duct tape for all that spam you are getting.

  22. Thanks for the tools Phil, Spam is getting worse all the time. I think i heard Matt (wordpress founder) in a CNet interview say something along the lines that he doesn’t use e-mail anymore. ??!! There isn’t that much spam is there? Say, engtech, tell me when you get some duct tape for all that spam you are getting.

  23. It’s a shame too, because I think comments are what make blogging so great. You post something, then someone has the chance to add to what you’ve written. It’s a great way to exchange information, opinions, and the like.

    Bummer. :(

  24. It’s a shame too, because I think comments are what make blogging so great. You post something, then someone has the chance to add to what you’ve written. It’s a great way to exchange information, opinions, and the like.

    Bummer. :(

  25. This is why I run my own software. Yes, it’s a pain in the ass sometimes, but at least I don’t have to worry about someone else’s version of “spam” deleting stuff for me.

    You seriously should think about it Robert. Once you get a good WordPress or MovableType system set up, it’s pretty simple to deal with.

  26. This is why I run my own software. Yes, it’s a pain in the ass sometimes, but at least I don’t have to worry about someone else’s version of “spam” deleting stuff for me.

    You seriously should think about it Robert. Once you get a good WordPress or MovableType system set up, it’s pretty simple to deal with.

  27. Thought you might be interested on this take in the full feed rss debate:

    http://scienceblogs.com/cognitivedaily/2007/05/the_end_of_the_rss_experiment.php
    – The end of the RSS experiment

    Last week we reported on our site statistics after going to a full RSS feed. The results were disappointing; our numbers went down. We said we’d continue the experiment for another week to see if the trend was reversed once more people heard about the option of viewing all CogDaily content in RSS feeds.

  28. Thought you might be interested on this take in the full feed rss debate:

    http://scienceblogs.com/cognitivedaily/2007/05/the_end_of_the_rss_experiment.php
    – The end of the RSS experiment

    Last week we reported on our site statistics after going to a full RSS feed. The results were disappointing; our numbers went down. We said we’d continue the experiment for another week to see if the trend was reversed once more people heard about the option of viewing all CogDaily content in RSS feeds.

  29. I had every comment and every post nuked by Russian spammers last week who got in through some WP hole and went to town. Ticked me off.

  30. I had every comment and every post nuked by Russian spammers last week who got in through some WP hole and went to town. Ticked me off.

  31. Scoble – it sounds like most of your comment-spam is bot generated. Click on my name to learn how we effectively got rid of all bot-generated comment spam. Let me know if you want the WordPress step-by-step.

  32. Scoble – it sounds like most of your comment-spam is bot generated. Click on my name to learn how we effectively got rid of all bot-generated comment spam. Let me know if you want the WordPress step-by-step.

  33. I don’t know why there isn’t a CAPTCHA built into wordpress.

    It would eliminate spam and the need for akismet.

    I know that having a captcha would reduce the revenue from ‘corporate’ akismet subscriptions but it doesn’t look to me like they are that interested in making money of akismet – it’s only $50 even for a corporate licence.

  34. I don’t know why there isn’t a CAPTCHA built into wordpress.

    It would eliminate spam and the need for akismet.

    I know that having a captcha would reduce the revenue from ‘corporate’ akismet subscriptions but it doesn’t look to me like they are that interested in making money of akismet – it’s only $50 even for a corporate licence.

  35. @robert:

    So I just wrote a program for WordPress.com blogs that turns off comments for all posts older than 60 days (using XML-RPC). I’ll do a study over a few weeks to see if that drops my spam levels (1500/day right now) to something more manageable. If it is worth doing, then I’ll release the program.

    Cheers.

  36. @robert:

    So I just wrote a program for WordPress.com blogs that turns off comments for all posts older than 60 days (using XML-RPC). I’ll do a study over a few weeks to see if that drops my spam levels (1500/day right now) to something more manageable. If it is worth doing, then I’ll release the program.

    Cheers.

  37. Robert, I’m using WP, but doing the hosting myself. One way I solved for the tons of spam was to rename my comments.php to something else. The spammers look for comments.php and trackback.php on the million+ WP sites…let them go for the big kill…while I was getting nowhere near the kind of spam you get (nor the traffic), I was seeing days of 2,500 pieces of junk. So, I renamed by two files and implemented a capcha, and voila! No more spam. Immediately. I wonder if that might work on your WP-hosted site. Seems nuts that your are being bombarded and can’t keep up with it…WP should help.

  38. Robert, I’m using WP, but doing the hosting myself. One way I solved for the tons of spam was to rename my comments.php to something else. The spammers look for comments.php and trackback.php on the million+ WP sites…let them go for the big kill…while I was getting nowhere near the kind of spam you get (nor the traffic), I was seeing days of 2,500 pieces of junk. So, I renamed by two files and implemented a capcha, and voila! No more spam. Immediately. I wonder if that might work on your WP-hosted site. Seems nuts that your are being bombarded and can’t keep up with it…WP should help.

  39. I Feel your pain, man

    I once tried to put a follow on comment on your blog within a few minutes of the first and it deleted that…may be another pattern?

    On spammers. I have heard most of the spam in the US originates in a FL town and the local authorities know about it and will not do anything about it. Not sure how true but its amazing Interpol, FBI cannot do much about the crisis it has become…

  40. I Feel your pain, man

    I once tried to put a follow on comment on your blog within a few minutes of the first and it deleted that…may be another pattern?

    On spammers. I have heard most of the spam in the US originates in a FL town and the local authorities know about it and will not do anything about it. Not sure how true but its amazing Interpol, FBI cannot do much about the crisis it has become…

  41. On my personal blog, I use a plugin that closes comments after 3 weeks plus a captcha program that asks you to add 2 numbers together. I get very little spam – if it does get through it’s usually caught by Akismet. On the other blog, which has no captcha, I get lots of spam – almost all Akismet caught but I still have to go with it. Installing a captcha is definitely the next step

  42. On my personal blog, I use a plugin that closes comments after 3 weeks plus a captcha program that asks you to add 2 numbers together. I get very little spam – if it does get through it’s usually caught by Akismet. On the other blog, which has no captcha, I get lots of spam – almost all Akismet caught but I still have to go with it. Installing a captcha is definitely the next step

  43. One easy way to kill almost all automatic SPAM is to use Ajax with a custom protocol for posting the comment. Most SPAMmers take advantage of the standard HTML posting routines which are supported universally. Once one requires Ajax (JavaScript) and a custom protocol, SPAMmers would need to go out of their ways and use slower tools like real browsers or other JavaScript emulators to try to fake their postings, and not only that, as they would need to understand the protocol and adapt their tools to your unique protocol convention.

    Basically, one day of programming for someone experienced with WordPress to add such a thing. It’s really tiresome to allow for SPAMmers to fill the comments, even with anti-SPAM checks in place like bayesian filtering.

    As most Web programming out there requires JavaScript, it should be pretty standard anyway, despite going against some of the sacred Web standards out there, though. :-)

    Just a tip!

  44. One easy way to kill almost all automatic SPAM is to use Ajax with a custom protocol for posting the comment. Most SPAMmers take advantage of the standard HTML posting routines which are supported universally. Once one requires Ajax (JavaScript) and a custom protocol, SPAMmers would need to go out of their ways and use slower tools like real browsers or other JavaScript emulators to try to fake their postings, and not only that, as they would need to understand the protocol and adapt their tools to your unique protocol convention.

    Basically, one day of programming for someone experienced with WordPress to add such a thing. It’s really tiresome to allow for SPAMmers to fill the comments, even with anti-SPAM checks in place like bayesian filtering.

    As most Web programming out there requires JavaScript, it should be pretty standard anyway, despite going against some of the sacred Web standards out there, though. :-)

    Just a tip!

  45. I like the idea of CAPTCHAs, but all the implementations out there looked really easy to break. I saw this site while browsing around though:

    http://recaptcha.net/plugins/wordpress/

    It’s a really interesting type of CAPTCHA that uses words from books rather than generating the CAPTCHAs. In the end, the CAPTCHA can be used to read (do OCR) on the book!

  46. I like the idea of CAPTCHAs, but all the implementations out there looked really easy to break. I saw this site while browsing around though:

    http://recaptcha.net/plugins/wordpress/

    It’s a really interesting type of CAPTCHA that uses words from books rather than generating the CAPTCHAs. In the end, the CAPTCHA can be used to read (do OCR) on the book!

  47. Robert,

    Probably you should close the comments and use a forum software for any discussion. This way, you can avoid those spams and people can still post their opinions inside the forum. Just my 2 cents.

  48. Robert,

    Probably you should close the comments and use a forum software for any discussion. This way, you can avoid those spams and people can still post their opinions inside the forum. Just my 2 cents.

  49. Robert, dude..if you delete all those comments how are you ever going to find out where to get your viagra and male enhancement products?

  50. Robert, dude..if you delete all those comments how are you ever going to find out where to get your viagra and male enhancement products?

  51. Just in the past week I’ve noticed a huge jump in comment spam to my low-traffic blog. I used to get a few per day. Now it’s 60-70 spam comments per day. I can’t imagine how you deal with 1,000 per day.

  52. Just in the past week I’ve noticed a huge jump in comment spam to my low-traffic blog. I used to get a few per day. Now it’s 60-70 spam comments per day. I can’t imagine how you deal with 1,000 per day.

  53. I think the best solution is to make the web a lot less anonymous. Their needs to be a clear way to identify who is doing what. I know I know about privacy and all those concerns. If the internet was the real world, then spammers would be like people putting up thousands of signs in Wall mart every day telling people to buy things in a back ally somewhere. Wall mart simply would have the people thrown out or arrested, but becuase we can’t track these people down effectively we have NO recourse.

  54. I think the best solution is to make the web a lot less anonymous. Their needs to be a clear way to identify who is doing what. I know I know about privacy and all those concerns. If the internet was the real world, then spammers would be like people putting up thousands of signs in Wall mart every day telling people to buy things in a back ally somewhere. Wall mart simply would have the people thrown out or arrested, but becuase we can’t track these people down effectively we have NO recourse.

  55. The story I am getting here really made me crazy. I have just started to write a blog about lives of people from Bangladesh, and I m now really thinking what will I do if I fall in a problem like yours! Canvas of life – The lives of real people

  56. The story I am getting here really made me crazy. I have just started to write a blog about lives of people from Bangladesh, and I m now really thinking what will I do if I fall in a problem like yours! Canvas of life – The lives of real people

  57. Spam comment and spam trackbacks is the #1 reason I switched to Drupal.

    Neither MovableType nor WordPress can deal with the spamm attacks my blogs suffer everyday –at about 5K a day combined and in a slow day.

    Just culturekitchen alone has come under attack with 1K spam comments and trackbacks an hour. Back in 2005, the last attack cost me over 1K in overage bandwidth and I ended up with over 25K spamments to delete –and that’s on my high traffic blog, not on any slow posting, abandoned site.

    I have been battling this now since 2005 and only see it getting worse with now the emergence of spamming cartels in China and Brasil. And yes, they are like the frigging mafia. Russian spammers are the worse.

  58. Spam comment and spam trackbacks is the #1 reason I switched to Drupal.

    Neither MovableType nor WordPress can deal with the spamm attacks my blogs suffer everyday –at about 5K a day combined and in a slow day.

    Just culturekitchen alone has come under attack with 1K spam comments and trackbacks an hour. Back in 2005, the last attack cost me over 1K in overage bandwidth and I ended up with over 25K spamments to delete –and that’s on my high traffic blog, not on any slow posting, abandoned site.

    I have been battling this now since 2005 and only see it getting worse with now the emergence of spamming cartels in China and Brasil. And yes, they are like the frigging mafia. Russian spammers are the worse.

  59. I had one day of over 1000 spam comments (almost all pharmacy spam) and after I deleted all of those in less than an hour I had about 50 more. That was exceptional; it must have been some major spam attack. It does seem to come in big waves like that. Usually I get about 100 a week. Since I started, Akismet caught 12,336 spams.

  60. I had one day of over 1000 spam comments (almost all pharmacy spam) and after I deleted all of those in less than an hour I had about 50 more. That was exceptional; it must have been some major spam attack. It does seem to come in big waves like that. Usually I get about 100 a week. Since I started, Akismet caught 12,336 spams.

  61. Sorry if I sound jaded, but with these kind of issues like with the whole Kathy Sierra thing, I just find it extremely annoying leading bloggers seem to think they are the first that have to deal with these kind of issues.

    Spam and misbehaving anti-spam filters and bots are stuff we’ve had to deal with for over a decade now on various internet-platforms.

    No “commenting in blogging is broken”. More like, move along, nothing to see…

  62. Sorry if I sound jaded, but with these kind of issues like with the whole Kathy Sierra thing, I just find it extremely annoying leading bloggers seem to think they are the first that have to deal with these kind of issues.

    Spam and misbehaving anti-spam filters and bots are stuff we’ve had to deal with for over a decade now on various internet-platforms.

    No “commenting in blogging is broken”. More like, move along, nothing to see…

  63. On the WordPress server software I can install the Comment Policy plugin. After installing that the blog spam dropped way low. Really only manual spam got through.

  64. On the WordPress server software I can install the Comment Policy plugin. After installing that the blog spam dropped way low. Really only manual spam got through.

  65. I have had to stop using my primary online “handle” because it was getting my comments deleted at an increasing number of blogs. Changing it is hardly onerous, but it’s kind of annoying that I had to do so just because of spam.

    If anyone’s curious what it was, I’ll spell it out and hope that it doesn’t get this comment tossed:

    f-i-a-t l-u-x

    My assumption is that the first word is not just Latin, but also a brand name, and thus gets flagged.

    Annoying.

  66. I have had to stop using my primary online “handle” because it was getting my comments deleted at an increasing number of blogs. Changing it is hardly onerous, but it’s kind of annoying that I had to do so just because of spam.

    If anyone’s curious what it was, I’ll spell it out and hope that it doesn’t get this comment tossed:

    f-i-a-t l-u-x

    My assumption is that the first word is not just Latin, but also a brand name, and thus gets flagged.

    Annoying.

  67. Fixing this is on my short list of things to do for WordPress 2.3 And as things tend to flow from WordPress trunk to WordPress MU trunk to WordPress.com fairly quickly, hopefully WP.com users can benefit too.

    I haven’t worked out all the details, but the half-baked plan would already greatly speed up comment administration, which will make Akismet’s bad days (or theoretical demise) easier to handle.

  68. Fixing this is on my short list of things to do for WordPress 2.3 And as things tend to flow from WordPress trunk to WordPress MU trunk to WordPress.com fairly quickly, hopefully WP.com users can benefit too.

    I haven’t worked out all the details, but the half-baked plan would already greatly speed up comment administration, which will make Akismet’s bad days (or theoretical demise) easier to handle.

  69. [...] I’m joking about quantity of spam as a measure of blog worth. But what I’m not joking about is how much more spam I am getting now compared to a year ago. I’d like to think it’s because my blog is so much more popular now, but the sad truth is that spam is an epidemic that’s affecting bloggers from all walks of life. Even Robert Scoble. [...]

  70. I know at least one commenter has suggested Spam Karma. With a twist, I make the same suggestion:

    There is an Akismet plugin for Spam Karma. This is different than running the two WordPress plugins at the same time: this makes Akismet work _within_ Spam Karma as one of its many tests.

    I use it, along with Bad Behavior, and it’s surprising rare that I actually get spam comments. Briefly put, it works very, very well.

  71. I know at least one commenter has suggested Spam Karma. With a twist, I make the same suggestion:

    There is an Akismet plugin for Spam Karma. This is different than running the two WordPress plugins at the same time: this makes Akismet work _within_ Spam Karma as one of its many tests.

    I use it, along with Bad Behavior, and it’s surprising rare that I actually get spam comments. Briefly put, it works very, very well.

  72. I have found two interesting sources and would like to give the benefit of my experience to you.
    I am tuning my pc by the best software for free, with the file search engine http://fileshunt. com and http://filesfinds.com . May be you have your own experience and could give some useful sites too. Because this two social sites help me much.

  73. I have found two interesting sources and would like to give the benefit of my experience to you.
    I am tuning my pc by the best software for free, with the file search engine http://fileshunt. com and http://filesfinds.com . May be you have your own experience and could give some useful sites too. Because this two social sites help me much.