Mirror mirror on the wall, which blog search is best of them all?

Last Friday I visited the famous South Park area in San Francisco. It’s a small park south of Market street where a number of cool Web 2.0 startups are located (Twitter’s parent, Obvious Corp, is located in a building on one end of the park). Anyway, I was talking with a number of people and I heard that Technorati’s blog search quality had seen a fairly large improvement lately and was better than Google’s blog search again. It’s hard to test blog search engines unless you know when an event started and can count up the good links and count up the spam.

There’s no way I can — alone — test out the search engine’s quality.

So, thought I’d open it up to all of you. Which engine is now better? Why? Give us some things to test out and reproduce what you’re seeing.

Here, I’ll start. It’s Videoblogging Week this week. So let’s compare both for “videobloggingweek2007.” That’s a top search on Technorati, so it should bias pretty well for Technorati, right? Let’s see!

Technorati for videobloggingweek2007. 147 results.
Google Blog Search for videobloggingweek2007. 76 results.

I didn’t see any obvious spam, did you? I think there might have been one on Google’s results, but it’s hard to tell spam blog, or splog, from actual real blogs anymore.

Also, I didn’t notice any duplicates on either service, did you? How about on the searches you’ve tried? It does look like Technorati has pulled ahead again in this race. The UI on Technorati is certainly ahead of Google’s, especially with the little chart of how many mentions a term has gotten. Search Technorati for EMI, for instance, and you can see that it really spiked today because they announced non-DRM music which got a lot of bloggers all excited.

So, try some searches of your own. Which engine is working best for you?

Comments

  1. Very interesting. Thanks for the comparison. My experience is different, but that’s just based on my own eclectic searches and the user interface that I prefer. But, overall I agree with your conclusions. Thanks!

  2. Very interesting. Thanks for the comparison. My experience is different, but that’s just based on my own eclectic searches and the user interface that I prefer. But, overall I agree with your conclusions. Thanks!

  3. Robert,

    Here’s another, Kathy Sierra’s latest post with an update on her appearance on CNN with Chris Locke:

    http://technorati.com/search/http%3A//headrush.typepad.com/creating_passionate_users/2007/04/updatejoint_sta.html?authority=n&start=0
    Technorati currently shows 71 links.

    http://blogsearch.google.com/blogsearch?hl=en&ie=UTF-8&q=link:http://headrush.typepad.com/creating_passionate_users/2007/04/updatejoint_sta.html&scoring=d&sa=N&start=0
    Google Blog Search currently reports 53 links. However, when you click on page 5 (to see the 53rd link):
    http://blogsearch.google.com/blogsearch?hl=en&ie=UTF-8&q=link:http://headrush.typepad.com/creating_passionate_users/2007/04/updatejoint_sta.html&scoring=d&sa=N&start=40

    it tells you that Google really only knows about 36 links.

    Grist for your mill. Thanks for the post, we really have been working very hard at continuing to improve Technorati!

    Dave

  4. Robert,

    Here’s another, Kathy Sierra’s latest post with an update on her appearance on CNN with Chris Locke:

    http://technorati.com/search/http%3A//headrush.typepad.com/creating_passionate_users/2007/04/updatejoint_sta.html?authority=n&start=0
    Technorati currently shows 71 links.

    http://blogsearch.google.com/blogsearch?hl=en&ie=UTF-8&q=link:http://headrush.typepad.com/creating_passionate_users/2007/04/updatejoint_sta.html&scoring=d&sa=N&start=0
    Google Blog Search currently reports 53 links. However, when you click on page 5 (to see the 53rd link):
    http://blogsearch.google.com/blogsearch?hl=en&ie=UTF-8&q=link:http://headrush.typepad.com/creating_passionate_users/2007/04/updatejoint_sta.html&scoring=d&sa=N&start=40

    it tells you that Google really only knows about 36 links.

    Grist for your mill. Thanks for the post, we really have been working very hard at continuing to improve Technorati!

    Dave

  5. Robert,

    Here’s another for you. Go look at the top item on Techmeme currently, the EMI Apple DRM announcement:

    http://www.emigroup.com/Press/2007/press18.htm

    Here’s the blogs linking to it as shown by Technorati:
    http://technorati.com/search/www.emigroup.com%2FPress%2F2007%2Fpress18.htm
    Currently showing 764 links.

    Here it is on Google Blog Search:
    http://blogsearch.google.com/blogsearch?hl=en&q=link:http://www.emigroup.com/Press/2007/press18.htm&ie=UTF-8&scoring=d
    Currently showing 491 links.

    Dave

  6. Robert,

    Here’s another for you. Go look at the top item on Techmeme currently, the EMI Apple DRM announcement:

    http://www.emigroup.com/Press/2007/press18.htm

    Here’s the blogs linking to it as shown by Technorati:
    http://technorati.com/search/www.emigroup.com%2FPress%2F2007%2Fpress18.htm
    Currently showing 764 links.

    Here it is on Google Blog Search:
    http://blogsearch.google.com/blogsearch?hl=en&q=link:http://www.emigroup.com/Press/2007/press18.htm&ie=UTF-8&scoring=d
    Currently showing 491 links.

    Dave

  7. Just from my own experience, I’m not really a huge fan of either. Technorati’s index seems adequate most of the time, but for some reason it failed to index a new post for over a week, until I posted again. Now I have a bunch of duplicate posts shown (although this may be because I started running my RSS through FeedBurner.)

    As for Google… I have a personal blog hosted on Blogspot, and on no occasion has Google blog search ever turned up a single page for it, even when I *know* I’m using search terms that occured in the relevant posts. I have much better luck with a vanilla site search, which I’ve found to work great for searching the archives (whereas new posts have to wait for the googlebot to crawl the site again.)

    Just my personal experiences, perhaps others have had better luck.

  8. Just from my own experience, I’m not really a huge fan of either. Technorati’s index seems adequate most of the time, but for some reason it failed to index a new post for over a week, until I posted again. Now I have a bunch of duplicate posts shown (although this may be because I started running my RSS through FeedBurner.)

    As for Google… I have a personal blog hosted on Blogspot, and on no occasion has Google blog search ever turned up a single page for it, even when I *know* I’m using search terms that occured in the relevant posts. I have much better luck with a vanilla site search, which I’ve found to work great for searching the archives (whereas new posts have to wait for the googlebot to crawl the site again.)

    Just my personal experiences, perhaps others have had better luck.

  9. Technorati is always down or broken when I want to use it. The unreliability led me over to Google.

  10. Technorati is always down or broken when I want to use it. The unreliability led me over to Google.

  11. Dave

    It’s hard to take a look when your algorithm completely ignores certian blogs.

    Our wordpress.com blog (the same one linked to in this comment) only gets updated on Technorati about every four months or so (its sitting at 89 days since the last update as of this comment). Our spanish language blog (http://casacomprar.wordpress.com) hasnt bee updated in over 200 days.

    - we post almost every day on our blogs
    - They are on a wordpress.com blog that pings technorati automatically
    - we also use pingoat about once every two weeks to ping the other pinging services who seem to find us without issue

    When we started a year ago, we’d be updated on Technorati in about 5 minutes…then one day it all just stopped.

    We also know that we aren’t the ones to experience this issue. You’ll gain a new fan once this issue is fixed across the board.

    Tony

  12. Dave

    It’s hard to take a look when your algorithm completely ignores certian blogs.

    Our wordpress.com blog (the same one linked to in this comment) only gets updated on Technorati about every four months or so (its sitting at 89 days since the last update as of this comment). Our spanish language blog (http://casacomprar.wordpress.com) hasnt bee updated in over 200 days.

    - we post almost every day on our blogs
    - They are on a wordpress.com blog that pings technorati automatically
    - we also use pingoat about once every two weeks to ping the other pinging services who seem to find us without issue

    When we started a year ago, we’d be updated on Technorati in about 5 minutes…then one day it all just stopped.

    We also know that we aren’t the ones to experience this issue. You’ll gain a new fan once this issue is fixed across the board.

    Tony

  13. Technorati vs. Google Blog Search and the Conversation

    Mirror mirror on the wall, which blog search is best of them all? - asks Robert Scoble.  My answer is very simple: Technorati has by far more features, they continue to be the innovation leader. It is because of these features that&nbsp…

  14. Has anyone noticed that Google Blog Search now finds comments, which Technorati still ignores? After all, comments are part of the conversation:-)

    Here’s a simple example, take David’s #12 above: Google finds itTechnorati doesn’t (but it took 50 seconds to search).

    (Sorry for dupe, trouble with URL at first attempt)

  15. Has anyone noticed that Google Blog Search now finds comments, which Technorati still ignores? After all, comments are part of the conversation:-)

    Here’s a simple example, take David’s #12 above: Google finds itTechnorati doesn’t (but it took 50 seconds to search).

    (Sorry for dupe, trouble with URL at first attempt)

  16. Since when is quantity of links a measure of quality success?

    How many of those are splogs or documented miscountings? http://www.seobythesea.com/?p=538
    Is the Technorati 100 accurate or is it manually manipulated?

    Where is Icerocket, Sphere and others in the conversation? Did all of the other blog search egnines close down and I missed the news?

  17. Since when is quantity of links a measure of quality success?

    How many of those are splogs or documented miscountings? http://www.seobythesea.com/?p=538
    Is the Technorati 100 accurate or is it manually manipulated?

    Where is Icerocket, Sphere and others in the conversation? Did all of the other blog search egnines close down and I missed the news?

  18. Tony,

    You’re right, casacomprar.wordpress.com wasn’t being indexed. We made a tweak to our spiders this morning, and now it is being indexed. Please do let me know if you experience any other indexing issues.

    Dave

  19. Tony,

    You’re right, casacomprar.wordpress.com wasn’t being indexed. We made a tweak to our spiders this morning, and now it is being indexed. Please do let me know if you experience any other indexing issues.

    Dave

  20. In my experience, Google returns far more results, but they also get periodically overrun with spam.

    I monitor the Google blog search feed for a technology I’m interested in, “OSGi”, and currently the feed is clean. Every now and again though, it gets flooded with spam for a couple of weeks, and then it all goes away. Of course it’s an arms race… the sploggers find a way to get their content into the feed; Google find a way to block it off; and then the cycle repeats.

    Technorati doesn’t have any of these problems, but it also doesn’t return any where near as many results.

  21. In my experience, Google returns far more results, but they also get periodically overrun with spam.

    I monitor the Google blog search feed for a technology I’m interested in, “OSGi”, and currently the feed is clean. Every now and again though, it gets flooded with spam for a couple of weeks, and then it all goes away. Of course it’s an arms race… the sploggers find a way to get their content into the feed; Google find a way to block it off; and then the cycle repeats.

    Technorati doesn’t have any of these problems, but it also doesn’t return any where near as many results.

  22. When looking at who is best I think one must look at two things.

    * Quality of Results (i.e. SPAM is huge problem)
    * Quantity of Results (i.e. Up to date, coverage)

    To me the quantity of results don’t mean a whole lot especially if I have thousands of results returned and a good majority of it are SPAM. So quality is more important, than quantity in my book.

    For example, I frequently (once per week) like to do blog searches to find out what is happening in real estate for a particular area (search: “real estate austin”).

    http://technorati.com/search/real+estate+austin

    For this search, on the first page the 3rd, 5th – 10th are all SPAM. There are 10 search results on the first page, so 70% of the results are SPAM. I the 1st, 2nd, and 4th result contain the words “real” , “estate”, and “austin” but have nothing to do with what I am looking for.

    The same search on Google Blogs don’t contain any Spam on the first page.
    http://blogsearch.google.com/blogsearch?hl=en&ie=UTF-8&oe=UTF-8&tab=wb&q=real+estate+austin&btnG=Search+Blogs

    The first result is a blog post about Hyde Park real estate in Austin. Highly relevant to what I am looking for.

    Some more competitive areas are ripe with SPAM. Check out the following:
    http://technorati.com/search/phentermine
    http://technorati.com/search/ringtones

    Given that a recent Microsoft research study mentioned that Blogspot contained more than 70% Spam Blogs I am not surprised by the results that I am seeing. Google seems to be doing a much better job at filtering SPAM than Technorati.

  23. When looking at who is best I think one must look at two things.

    * Quality of Results (i.e. SPAM is huge problem)
    * Quantity of Results (i.e. Up to date, coverage)

    To me the quantity of results don’t mean a whole lot especially if I have thousands of results returned and a good majority of it are SPAM. So quality is more important, than quantity in my book.

    For example, I frequently (once per week) like to do blog searches to find out what is happening in real estate for a particular area (search: “real estate austin”).

    http://technorati.com/search/real+estate+austin

    For this search, on the first page the 3rd, 5th – 10th are all SPAM. There are 10 search results on the first page, so 70% of the results are SPAM. I the 1st, 2nd, and 4th result contain the words “real” , “estate”, and “austin” but have nothing to do with what I am looking for.

    The same search on Google Blogs don’t contain any Spam on the first page.
    http://blogsearch.google.com/blogsearch?hl=en&ie=UTF-8&oe=UTF-8&tab=wb&q=real+estate+austin&btnG=Search+Blogs

    The first result is a blog post about Hyde Park real estate in Austin. Highly relevant to what I am looking for.

    Some more competitive areas are ripe with SPAM. Check out the following:
    http://technorati.com/search/phentermine
    http://technorati.com/search/ringtones

    Given that a recent Microsoft research study mentioned that Blogspot contained more than 70% Spam Blogs I am not surprised by the results that I am seeing. Google seems to be doing a much better job at filtering SPAM than Technorati.

  24. Technorati has gotten much better. even before the recent updates I got a lot less spam from Technorati when compared to Google.

    Monitoring a persistent search string or keyword on the Netvibes blog search engine is a very effective way to let the blog search engines go tete-a-tete. Since our users use a host of blog search engines to monitor RSS feeds via IM/SMS/Email, this allows us to constantly monitor how well each engine is doing. IceRocket often surprises me with finds that the others haven’t caught onto yet but they don’t always find all the results.

    On to your test: We had 3 mentions in the blogosphere in the last 12 hours. If you do a sweep across all blog search engines using Netvibes, you’ll see that:
    - Technorati caught 2 of 3
    - Google also caught 2 of 3 (not the same 2 as Technorati)
    - Sphere caught 2 of 3
    - IceRocket caught 1
    This could change as the indexing catches up in the next few hours but thats what it looks like right now.

  25. Technorati has gotten much better. even before the recent updates I got a lot less spam from Technorati when compared to Google.

    Monitoring a persistent search string or keyword on the Netvibes blog search engine is a very effective way to let the blog search engines go tete-a-tete. Since our users use a host of blog search engines to monitor RSS feeds via IM/SMS/Email, this allows us to constantly monitor how well each engine is doing. IceRocket often surprises me with finds that the others haven’t caught onto yet but they don’t always find all the results.

    On to your test: We had 3 mentions in the blogosphere in the last 12 hours. If you do a sweep across all blog search engines using Netvibes, you’ll see that:
    - Technorati caught 2 of 3
    - Google also caught 2 of 3 (not the same 2 as Technorati)
    - Sphere caught 2 of 3
    - IceRocket caught 1
    This could change as the indexing catches up in the next few hours but thats what it looks like right now.

  26. Dave
    Its my Netvibes page that I used. If you type in ZapTXT in their blog search module, you’ll see the results stack up for all default blog search engines that come with the blog search module they offer.

  27. Dave
    Its my Netvibes page that I used. If you type in ZapTXT in their blog search module, you’ll see the results stack up for all default blog search engines that come with the blog search module they offer.

  28. hmm, i think there’s still lots of room for improvement for both.

    IMHO, Technorati is still slow and it often takes a couple of days until the blog posts are up-to-date.

  29. hmm, i think there’s still lots of room for improvement for both.

    IMHO, Technorati is still slow and it often takes a couple of days until the blog posts are up-to-date.

  30. Robert – I posted an article (click my name) about why a comparison btw Technorati and Google Blog search is NOT valid. I also include another question you should be asking your visitors in your initial post here.

  31. Robert – I posted an article (click my name) about why a comparison btw Technorati and Google Blog search is NOT valid. I also include another question you should be asking your visitors in your initial post here.

  32. I have found that ContextWeb’s blog posts often do not get indexed by Technorati for a day or two unless I go into my profile on Technorati and ping. The WordPress blog is all set up to autoping. Not sure what the issue is. – It’s always possible it’s me, though.

    Google seems to take care of itself but I don’t track the pings as closely as Technorati. I still prefer Mr. Sifry’s site. I agree with you Robert about the UI.. also, just feels like more of a community there. That’s what I’m looking for.

    My two cents.

  33. I have found that ContextWeb’s blog posts often do not get indexed by Technorati for a day or two unless I go into my profile on Technorati and ping. The WordPress blog is all set up to autoping. Not sure what the issue is. – It’s always possible it’s me, though.

    Google seems to take care of itself but I don’t track the pings as closely as Technorati. I still prefer Mr. Sifry’s site. I agree with you Robert about the UI.. also, just feels like more of a community there. That’s what I’m looking for.

    My two cents.

  34. @Allen Stern -

    I read your article and while I enjoyed it, I politely and respectful disagree entirely.

    Whether or not blogs are being incorporated into the main index or excatly what users are actually using to perform blog searches is irrelevant to this discussion. Google has a tool readily available called “Blog Search” and until Google decommissions the app and just integrates and ranks blogs appropriately in their main index and only their main index, then the only fair comparison doesn’t include the main index.

    Of course, you’re 100% right in what you say – what is a blog, no one actually uses the Google Blog Search, the main index suffices for people’s needs, etc., but the main index really has no place in the comparison.

  35. @Allen Stern -

    I read your article and while I enjoyed it, I politely and respectful disagree entirely.

    Whether or not blogs are being incorporated into the main index or excatly what users are actually using to perform blog searches is irrelevant to this discussion. Google has a tool readily available called “Blog Search” and until Google decommissions the app and just integrates and ranks blogs appropriately in their main index and only their main index, then the only fair comparison doesn’t include the main index.

    Of course, you’re 100% right in what you say – what is a blog, no one actually uses the Google Blog Search, the main index suffices for people’s needs, etc., but the main index really has no place in the comparison.

  36. Excellent to see Dave Sifry so active on this thread!

    I have not used the search function in Technorati for a while just because it didn’t really do what I wanted (er, it gave results that I had to wade through). But it sounds like things have improved quite a bit – I’ll go give it another shot.

    Jason Alba
    CEO – JibberJobber.com

  37. Excellent to see Dave Sifry so active on this thread!

    I have not used the search function in Technorati for a while just because it didn’t really do what I wanted (er, it gave results that I had to wade through). But it sounds like things have improved quite a bit – I’ll go give it another shot.

    Jason Alba
    CEO – JibberJobber.com

  38. Hagrin – it’s ok to disagree – it’s the only way we can move forward together.

    Yes, you are right. they have a product called blog search. yes, technorati can compare themselves to it.

    Let me illustrate my point.

    I will put up a blog search tonight – we will call it “Moo Blog Search” – it would be easy for Dave to say “Technorati is bigger than Moo Blog Search” — of course no one uses Moo and so how valid is that?

    Clearly Google main search is being used over the blog search. That’s why it’s not a fair comparison.

  39. Hagrin – it’s ok to disagree – it’s the only way we can move forward together.

    Yes, you are right. they have a product called blog search. yes, technorati can compare themselves to it.

    Let me illustrate my point.

    I will put up a blog search tonight – we will call it “Moo Blog Search” – it would be easy for Dave to say “Technorati is bigger than Moo Blog Search” — of course no one uses Moo and so how valid is that?

    Clearly Google main search is being used over the blog search. That’s why it’s not a fair comparison.

  40. There are a lot of ways to compare blog search engines. You could look at which engine returns the most links to a post. You could compare queries to see which engine has the most spam. You could look at which engine returns the most results for some relatively rare phrase like [matt cutts embiggen]. You could look at the speed of indexing individual posts. You could ask which engine had neater features, UI, or flair. You could look at how long each engine takes to return results for a query.

    Every person is going to care about slightly different things. Me, I care less about backlinks and more about recall and speed of getting results back, for example. The great thing is that with a lot of engines competing, every engine is working hard and continues to get better. That’s better for every user.

    My personal advice would be to try several engines and see what you like. Every 2-3 months, try icerocket.com or sphere or t’rati or Google again, just to see which one works best for your search needs.

  41. There are a lot of ways to compare blog search engines. You could look at which engine returns the most links to a post. You could compare queries to see which engine has the most spam. You could look at which engine returns the most results for some relatively rare phrase like [matt cutts embiggen]. You could look at the speed of indexing individual posts. You could ask which engine had neater features, UI, or flair. You could look at how long each engine takes to return results for a query.

    Every person is going to care about slightly different things. Me, I care less about backlinks and more about recall and speed of getting results back, for example. The great thing is that with a lot of engines competing, every engine is working hard and continues to get better. That’s better for every user.

    My personal advice would be to try several engines and see what you like. Every 2-3 months, try icerocket.com or sphere or t’rati or Google again, just to see which one works best for your search needs.

  42. Just from my own experience, I’m not really a huge fan of either. Technorati’s index seems adequate most of the time, but for some reason it failed to index a new post for over a week, until I posted again. I think the post might have been duplicated as well.

    As for Google… I have a personal blog hosted on blogspot, and on no occasion has Google blog search ever turned up a single page for it, even when I *know* I’m using search terms that occured in the relevant posts. I have much better luck with a vanilla site search, which I’ve found to work great for searching the archives (whereas new posts have to wait for the googlebot to crawl the site again.)

    Just my personal experiences, perhaps others have had better luck.

  43. Just from my own experience, I’m not really a huge fan of either. Technorati’s index seems adequate most of the time, but for some reason it failed to index a new post for over a week, until I posted again. I think the post might have been duplicated as well.

    As for Google… I have a personal blog hosted on blogspot, and on no occasion has Google blog search ever turned up a single page for it, even when I *know* I’m using search terms that occured in the relevant posts. I have much better luck with a vanilla site search, which I’ve found to work great for searching the archives (whereas new posts have to wait for the googlebot to crawl the site again.)

    Just my personal experiences, perhaps others have had better luck.

  44. Looking at the logs for my blog I’m seeing very few people arrive from blog search engines – almost everyone comes from normal Google.

    Standard Goolge seems to do a very good job on quickly indexing and intergrating blog posts into their search index. I saw my blog posts about Sierra et al. appear very quickly (and high up) on Google, and that brought me a lot of traffic.

  45. Looking at the logs for my blog I’m seeing very few people arrive from blog search engines – almost everyone comes from normal Google.

    Standard Goolge seems to do a very good job on quickly indexing and intergrating blog posts into their search index. I saw my blog posts about Sierra et al. appear very quickly (and high up) on Google, and that brought me a lot of traffic.

  46. Here’s the one thing I LOVE about google blog search that Technorati doesn’t offer:

    The advanced search features offer me the opportunity to search specific time periods.

    So, for example, if I want to read blog posts from a specific day in history (9/11 or Aug. 29, 2005) I can do that easily.

    More to point: when I search blogs, I don’t always only want to know “what people are saying right now” I want to know whatever it is I’m looking for, usually the truth, not the current spin.

  47. Here’s the one thing I LOVE about google blog search that Technorati doesn’t offer:

    The advanced search features offer me the opportunity to search specific time periods.

    So, for example, if I want to read blog posts from a specific day in history (9/11 or Aug. 29, 2005) I can do that easily.

    More to point: when I search blogs, I don’t always only want to know “what people are saying right now” I want to know whatever it is I’m looking for, usually the truth, not the current spin.

  48. I have a question for anyone who knows the answer…

    On my blog, I have google ads…and was wondering if google have some favor towards blogs with ads because they would make money.

    I think you call it an algorithm or something like.

    So, if i’m getting say 10 clicks per day on my google ads (which I am), but it’s a relatively new website, will that help my quest to climb the search engine faster?

    I look forward to your thoughts on this. Thanks.

    Dean

  49. I have a question for anyone who knows the answer…

    On my blog, I have google ads…and was wondering if google have some favor towards blogs with ads because they would make money.

    I think you call it an algorithm or something like.

    So, if i’m getting say 10 clicks per day on my google ads (which I am), but it’s a relatively new website, will that help my quest to climb the search engine faster?

    I look forward to your thoughts on this. Thanks.

    Dean

  50. Check the BlogBlogs.Com.Br, its the best blog searching for Brazilian and Portuguese Speaking blogs. Its also a social network of bloggers and readers with many cool stuff like the Readers Widget.

  51. Check the BlogBlogs.Com.Br, its the best blog searching for Brazilian and Portuguese Speaking blogs. Its also a social network of bloggers and readers with many cool stuff like the Readers Widget.

  52. Meet the Bloggerati at Technorati…

    Welcome to the 2nd installment of “Heroes,” a series of stories explaining how and why I use social media tools to enhance productivity and better my life. My thoughts are interspersed with the words ……