We need better statistics…

Almost every entrepreneur I talk to lately whines privately about the stats they see on places like Compete.com, Comscore, and Alexa. Today Tom Conrad of Pandora told me that they are extremely low. He says his service requires registration, so he has very accurate stats of who’s signed into Pandora and he can’t figure out why the stats services are so far off of the real stats.

Marshall Sponder, over on Web Metrics Guru, looks into Comscore’s stats of Second Life’s users and finds the same problem.

The thing is these services rely on toolbars (I can’t even use any of the toolbars on the Macintosh for some reason, and how many of you even have one of these folks’ toolbars loaded? None of my friends do and I’ve been checking). Or they rely on “panels” of Web users that they survey regularly. Do you know the selection mechanisms? How do they know they are getting a representative sample? Clearly very few people who run Web companies find their stats accurate. Yet we’re supposed to believe in them?

Also, there are lots of sites who seem to have more traffic than, say, my blog, but they get less comments on every post and if we both link to someone new, the new site gets a lot more traffic from me. I have such a site in mind, but I don’t want to get into an argument with that site. Translation: the engagement levels on some blogs are quite different, but advertisers are being sold on these stats companies and on pure “uniques.”

I don’t know what the solution is, though. What stats do you think are the most important? What’s the most accurate way to measure your sites’ visitors? What will advertisers insist on seeing in the future?

Oh, and in the future people aren’t going to visit your page at all. Most of PodTech’s traffic comes from its embeddable gadget. So, are you visiting a blog that has our gadget embedded when you watch one of my videos or are you visiting PodTech? I bet most normal people will answer “a blog.” That’d mean that PodTech’s traffic will get way underrepresented in these services (which matches what we’re seeing in our server logs when we compare our real traffic with what Alexa/Compete/Comscore are telling us).

81 thoughts on “We need better statistics…

  1. I’ve always used Hitwise and it has worked out well, still the absolute best is
    Google Analytics. It takes a bit time to tweak it, but it is all worth it in the end.

  2. I’ve always used Hitwise and it has worked out well, still the absolute best is
    Google Analytics. It takes a bit time to tweak it, but it is all worth it in the end.

  3. Speaking seo traffic, do not consider redzee as a source of viable click through. We did a campaign with them and all the traffic never went past the first page. I think they are doing some shady stuff. We own a printing company, AREA Printing & Design http://www.areaprinting.com and we have instant online chat, none of the clicks ever requested a chat session. It was a waste of money.
    printing

  4. Speaking seo traffic, do not consider redzee as a source of viable click through. We did a campaign with them and all the traffic never went past the first page. I think they are doing some shady stuff. We own a printing company, AREA Printing & Design http://www.areaprinting.com and we have instant online chat, none of the clicks ever requested a chat session. It was a waste of money.
    printing

  5. Hi Robert – Take a look at Quantcast. Bunch of really smart engineers using statistics and computer science to produce really accurate, interesting results. They also have sites (including Wordress and Facebook) using their tracking Javascript code. Here’s an example:

    http://www.quantcast.com/facebook.com

  6. Hi Robert – Take a look at Quantcast. Bunch of really smart engineers using statistics and computer science to produce really accurate, interesting results. They also have sites (including Wordress and Facebook) using their tracking Javascript code. Here’s an example:

    http://www.quantcast.com/facebook.com

  7. I think there are three issues here that are being lumped together, and in so doing it’s creating more confusion.

    1. General Stats – The Alexas of the web will always be off based on their methodologies using panel or toolbar data. the only way to shore it up is to create a better system that can prevent cheating…but this isn’t going to happen soon.

    2. Web Analytics – Obviously in this case there are a host of competitors ranging from free to really expensive and also varying wildly in their offering. This category is the definitive source to track everything going on at your local server…but it’s not going to solve the bloggers dilemma. And it’s hard to get two systems to say the same number…but at the end of the day it doesn’t matter since you’re looking for trending and directional information for the most part.

    3. Blog Analytics – I recently did a post on my blog about the complete void within the blog analytics space…since that post I’ve heard that Google will be releasing Measure Map, a blog specific tool that seems quite powerful and takes a step beyond the current web analytics tools out there. The measure map UI was lifted for the new version of Google Analytics and is quite slick.

  8. I think there are three issues here that are being lumped together, and in so doing it’s creating more confusion.

    1. General Stats – The Alexas of the web will always be off based on their methodologies using panel or toolbar data. the only way to shore it up is to create a better system that can prevent cheating…but this isn’t going to happen soon.

    2. Web Analytics – Obviously in this case there are a host of competitors ranging from free to really expensive and also varying wildly in their offering. This category is the definitive source to track everything going on at your local server…but it’s not going to solve the bloggers dilemma. And it’s hard to get two systems to say the same number…but at the end of the day it doesn’t matter since you’re looking for trending and directional information for the most part.

    3. Blog Analytics – I recently did a post on my blog about the complete void within the blog analytics space…since that post I’ve heard that Google will be releasing Measure Map, a blog specific tool that seems quite powerful and takes a step beyond the current web analytics tools out there. The measure map UI was lifted for the new version of Google Analytics and is quite slick.

  9. Great Question.

    I’m actually researching this now to justify a corporate initiative. I’m arguing to use a combination of the following:

    Authority – based on links

    Conversation Index – See Stowe Boyd’s blog

    Feed subscriptions – much like a conversion rate in sales

    Am I nuts?

    Regards,

    Mark Krupinski

  10. Great Question.

    I’m actually researching this now to justify a corporate initiative. I’m arguing to use a combination of the following:

    Authority – based on links

    Conversation Index – See Stowe Boyd’s blog

    Feed subscriptions – much like a conversion rate in sales

    Am I nuts?

    Regards,

    Mark Krupinski

  11. @33 Rod: the same thought ocurred to me while doing some competitive research, but guess what, you can easily game GA results too! If you can mess with Alexa (see comment 4, Chris) what prevents you from doing stuff on your own site to make the urchin script go nuts? as chris says in comment 10, you really cant trust the site’s own metrics. So that option is out too…

    ISPs hold probably the only data that could count. But what is their incentive?

  12. @33 Rod: the same thought ocurred to me while doing some competitive research, but guess what, you can easily game GA results too! If you can mess with Alexa (see comment 4, Chris) what prevents you from doing stuff on your own site to make the urchin script go nuts? as chris says in comment 10, you really cant trust the site’s own metrics. So that option is out too…

    ISPs hold probably the only data that could count. But what is their incentive?

  13. I just wanted to point readers to a suggestion I made about a month ago that pertains to Robert’s timely post. In a nutshell, the suggestion is that Google allow webmasters to make public a subset of their Analytics information.

    http://techfold.com/2007/04/03/how-google-can-kill-alexa-in-one-simple-step/

    “Adding a “Sharing” option to Google Analytics and surfacing stats in “site:” searches (for those site owners who have elected the sharing option in their Analytics account) would do the job nicely. Let site owners control the degree of information shared, keep everything opt-in, and rock and roll. I know I’d share my high-level views & visits stats in a second. In addition to providing all of the value Alexa does, it would also add a layer of transparency to making ad-buys – something else I would appreciate.”

  14. I just wanted to point readers to a suggestion I made about a month ago that pertains to Robert’s timely post. In a nutshell, the suggestion is that Google allow webmasters to make public a subset of their Analytics information.

    http://techfold.com/2007/04/03/how-google-can-kill-alexa-in-one-simple-step/

    “Adding a “Sharing” option to Google Analytics and surfacing stats in “site:” searches (for those site owners who have elected the sharing option in their Analytics account) would do the job nicely. Let site owners control the degree of information shared, keep everything opt-in, and rock and roll. I know I’d share my high-level views & visits stats in a second. In addition to providing all of the value Alexa does, it would also add a layer of transparency to making ad-buys – something else I would appreciate.”

  15. Companies complaining that widget views don’t count as pageviews is rediculus. When is the last time syndicated content as traffic for Associated Press?

  16. Companies complaining that widget views don’t count as pageviews is rediculus. When is the last time syndicated content as traffic for Associated Press?

  17. Ummm, Nielsen Television Index “won” this debate back in 1950. We’ve been there, done that.

    The only ‘sure way’ is to pass a governmental law requiring all ISPs/Networks to give up data, and then some special commission can be appointed to filter it out. Police state spyware, but hey, accurate ratings.

  18. Ummm, Nielsen Television Index “won” this debate back in 1950. We’ve been there, done that.

    The only ‘sure way’ is to pass a governmental law requiring all ISPs/Networks to give up data, and then some special commission can be appointed to filter it out. Police state spyware, but hey, accurate ratings.

  19. For those that are saying “stats are overrated” and so-forth, the sad reality for an advertising-supported site that sells its own ads is that your ranking on Comscore in particular has a HUGE impact on the likelyhood that you’ll be able to close top-tier accounts. If Comscore says you don’t have the traffic to handle a campaign that wants to see a million uniques then you’re not going to win that business. At least for me that’s why this issue is so sensitive.

  20. For those that are saying “stats are overrated” and so-forth, the sad reality for an advertising-supported site that sells its own ads is that your ranking on Comscore in particular has a HUGE impact on the likelyhood that you’ll be able to close top-tier accounts. If Comscore says you don’t have the traffic to handle a campaign that wants to see a million uniques then you’re not going to win that business. At least for me that’s why this issue is so sensitive.

  21. Don’t lump in Hitwise with the others. Hitwise’s methodology is not based on toolbars or surveys or log books, it’s raw numbers they get from deep inside ISP networks. I’d trust Hitwise far more.

  22. Don’t lump in Hitwise with the others. Hitwise’s methodology is not based on toolbars or surveys or log books, it’s raw numbers they get from deep inside ISP networks. I’d trust Hitwise far more.

Comments are closed.