New audience metric needed: engagement

I was just reading Jeneane Sessum’s post about the latest Ze Frank/Rocketboom dustup and she’s right, we need to measure stuff other than just whether a download got completed or not. She says we need a “likeability” stat. I think it goes further than that.

There’s another stat out there called “engagement.” No one is measuring it that I know of.

What do I mean?

Well, I’ve compared notes with several bloggers and journalists and when the Register links to us we get almost no traffic. But they claim to have millions of readers. So, if millions of people are hanging out there but no one is willing to click a link, that means their audience has low engagement. The Register is among the lowest that I can see.

Compare that to Digg. How many people hang out there every day? Maybe a million, but probably less. Yet if you get linked to from Digg you’ll see 30,000 to 60,000 people show up. And these people don’t just read. They get involved. I can tell when Digg links to me cause the comments for that post go up too.

So, why should engagement matter to an advertiser?

Well, as an advertiser I want to talk to an audience who’ll actually DO something. Yeah, I’m hoping to get a sale.

Yesterday Buzz Bruggeman CEO of Active Words, was driving me around and told the story of when he was in USA Today. He got 32 downloads. When he got linked to by my blog? Got about 400.

My audience was (and is) a lot smaller than USA Today, but the engagement of the blog audience got his attention.

How could we measure audience engagement?

Is this something that Steve Gillmor’s GestureLab could do? If he could, that’d be a valuable company that advertisers would die to buy stuff from.

Comments

  1. Audience metrics are desperately needed; I don’t know how many people completely watch my videos when you only have to start one to be counted.

    Link rank can and is gamed, clickbots are out there, but how does Nelson rate TV shows now that we all have remotes in our hands surfing the dial?

  2. Audience metrics are desperately needed; I don’t know how many people completely watch my videos when you only have to start one to be counted.

    Link rank can and is gamed, clickbots are out there, but how does Nelson rate TV shows now that we all have remotes in our hands surfing the dial?

  3. Robert — there’s a metric we use in the learning/training world called “Kirkpatrick’s Levels” that could be remodeled for a framework.

    http://coe.sdsu.edu/eet/articles/k4levels/index.htm

    Basic premise is that when you teach someone a new idea, there are four levels of learning.

    1. Reactions (they heard you)
    2. Learning (they understood and retained)
    3. Transfer (they took in the information and can apply)
    4. Results (they use the learning to achieve a goal)

    An example would be telling your child about tieing his shoes, him understanding the principle of making the bunny ears, doing the task with your coaching, and finally tieing his shoes alone.

    I propose a four-level model for measuring engagement:

    1. Click – A reader arrived (current metric)
    2. Consume – A reader read the content
    3. Understoon – A reader understood the content and remembers
    4. Applied – A reader applies the content in another venue

    Each of these could be measured (with accuracy pretty closely connected to your budget) and compared with other ways of getting the information out.

    What do you think?

  4. Robert — there’s a metric we use in the learning/training world called “Kirkpatrick’s Levels” that could be remodeled for a framework.

    http://coe.sdsu.edu/eet/articles/k4levels/index.htm

    Basic premise is that when you teach someone a new idea, there are four levels of learning.

    1. Reactions (they heard you)
    2. Learning (they understood and retained)
    3. Transfer (they took in the information and can apply)
    4. Results (they use the learning to achieve a goal)

    An example would be telling your child about tieing his shoes, him understanding the principle of making the bunny ears, doing the task with your coaching, and finally tieing his shoes alone.

    I propose a four-level model for measuring engagement:

    1. Click – A reader arrived (current metric)
    2. Consume – A reader read the content
    3. Understoon – A reader understood the content and remembers
    4. Applied – A reader applies the content in another venue

    Each of these could be measured (with accuracy pretty closely connected to your budget) and compared with other ways of getting the information out.

    What do you think?

  5. This is an OLD topic of conversation. I can remember diving into it in 2001 pretty seriously, the writing was on the wall back then.

    It would be great to dig up some of that old material and find out what others have been doing with this as well.

  6. This is an OLD topic of conversation. I can remember diving into it in 2001 pretty seriously, the writing was on the wall back then.

    It would be great to dig up some of that old material and find out what others have been doing with this as well.

  7. Good point on engagement. This is the part of the conversation that breaks out of the traditional CPM model (or ups the cpm cost depending on how you break it down); each eyeball has half a mouth. :)

  8. Good point on engagement. This is the part of the conversation that breaks out of the traditional CPM model (or ups the cpm cost depending on how you break it down); each eyeball has half a mouth. :)

  9. Direct marketers have been using engagement measures on the web for a while, it’s easy with the logging available on a web site.

    “The more people interact with you the more they will buy from you.”

  10. Direct marketers have been using engagement measures on the web for a while, it’s easy with the logging available on a web site.

    “The more people interact with you the more they will buy from you.”

  11. whoa… one of your best posts in a long time.
    indeed, it’s hard to parametrize an audience, but I think engagement is definitly something to be measure more accurately for marketers.

  12. whoa… one of your best posts in a long time.
    indeed, it’s hard to parametrize an audience, but I think engagement is definitly something to be measure more accurately for marketers.

  13. I was thinking about this the other day from the point of view of blog promotion.

    Social networks are fine and dandy, but the ideal audience for a blogger is other bloggers because they can link, they are likely to get involved, and they have their own audience.

  14. I was thinking about this the other day from the point of view of blog promotion.

    Social networks are fine and dandy, but the ideal audience for a blogger is other bloggers because they can link, they are likely to get involved, and they have their own audience.

  15. [...] Robert Scoble – who still matters, even though he’s left Microsoft – makes an interesting point about ‘engagement’. All the talk these days is about it. Yet there’s no easy way to measure it. He offers a few interesting numbers as a starting point, though: When the Register links to us we get almost no traffic. But they claim to have millions of readers. Compare that to Digg. How many people hang out there every day? Maybe a million, but probably less. Yet if you get linked to from Digg you’ll see 30,000 to 60,000 people show up. And these people don’t just read. They get involved. Yesterday Buzz Bruggeman CEO of Active Words, was driving me around and told the story of when he was in USA Today. He got 32 downloads. When he got linked to by my blog? Got about 400. [...]

  16. One of the registers failings is that it very seldom links to anything (this is a generic problem with “news” type sites – article about something or someone and no link to the product page or their bio, it can be done – the BBC does it very well).

  17. One of the registers failings is that it very seldom links to anything (this is a generic problem with “news” type sites – article about something or someone and no link to the product page or their bio, it can be done – the BBC does it very well).

  18. “…no one is willing to click a link, that means their audience has low engagement…”

    Or they are claiming that their audience has high engagement with THEIR site and its articles and so doesn’t want to leave to go to another site. If content is too compelling (esp. video) surely there’s a tendancy for one to not click on a link, which is why video content needs video advertising and we’re back to CPM deals again instead of CPC.

  19. “…no one is willing to click a link, that means their audience has low engagement…”

    Or they are claiming that their audience has high engagement with THEIR site and its articles and so doesn’t want to leave to go to another site. If content is too compelling (esp. video) surely there’s a tendancy for one to not click on a link, which is why video content needs video advertising and we’re back to CPM deals again instead of CPC.

  20. I couldn’t agree more with the idea of measuring engagement which I also refer to as the energy an individual or group will commit to a product/idea. So, to tackle the idea of measuring it.

    1. While I recognize it’s obvious, track which sites are driving the most viewers to the product/idea specific page.
    2. Measure how much discussion is taking place about the product/idea by creating a metrics tool that uses Technorati, Google Groups, and so forth. The software will then pull a random sample of comments so that somebody can put a qualitative score on whether the discussion is favorable or not to and how relevant is the discussion.

    For videos, a question I have regarding Quicktime development is whether it is possible to (1) layer graphics over the quicktime movie which pull from the internet and (2)can some rudimentary logic (through Qscript) be built into to Quicktime to test for an internet connection and if so, display the track that uses graphics from a web site otherwise use the track that has graphics imbedded into the movie? If that is possible, it seems this is the answer to how much of a video someone has watched and then the ability to track whether they took action on a link in the video.

    My final idea related to this is a quicktime type server that wraps the video track with a track containing links back to the server so that each video downloaded has a unique identifier built into the link. It’s then possible to get an idea of how often people are sharing the video by checking which IP addresses between initial downloads of the video and links via the video.

  21. I couldn’t agree more with the idea of measuring engagement which I also refer to as the energy an individual or group will commit to a product/idea. So, to tackle the idea of measuring it.

    1. While I recognize it’s obvious, track which sites are driving the most viewers to the product/idea specific page.
    2. Measure how much discussion is taking place about the product/idea by creating a metrics tool that uses Technorati, Google Groups, and so forth. The software will then pull a random sample of comments so that somebody can put a qualitative score on whether the discussion is favorable or not to and how relevant is the discussion.

    For videos, a question I have regarding Quicktime development is whether it is possible to (1) layer graphics over the quicktime movie which pull from the internet and (2)can some rudimentary logic (through Qscript) be built into to Quicktime to test for an internet connection and if so, display the track that uses graphics from a web site otherwise use the track that has graphics imbedded into the movie? If that is possible, it seems this is the answer to how much of a video someone has watched and then the ability to track whether they took action on a link in the video.

    My final idea related to this is a quicktime type server that wraps the video track with a track containing links back to the server so that each video downloaded has a unique identifier built into the link. It’s then possible to get an idea of how often people are sharing the video by checking which IP addresses between initial downloads of the video and links via the video.

  22. While I see the value for better overall definition of an audience, “engagement” falls into that vague “eyeballish” zone. It doesn’t really mean anything. And “engaged” exactly how, as ‘engaged’ audiences might be twice the effort and cost, for the same result. Maybe for some advertisers better to narrow on the extreme targeted or busy types who want raw facts fast. You can’t expect the CFO’s to play engagement games.

    Some people are busy running companies and don’t want to cuddle up with vendors all the live day long. If said product doesn’t work or doesn’t meet needs or isn’t at a good pricepoint, they will just get a new vendor. And sometimes (in fact, imho, most often) the most engaged audiences are those with the least amount of power (as they have the raw time to engage). Don’t discount the busy and aloof, that’s sometimes the real gold.

    And you can’t really compare USA Today audiences with bloggers, maybe only a few clicked thru, but maybe one was a busy CIO (who has no time for blog chatter) but saw it in USA Today when they shoved it under his hotel door, a CIO who decides to order 100 copies. Clicks aren’t the end-all, that’s too blogger nose-leveled view.

    The only metric (to the advertiser that counts) is increased marketshare, engaged or not. But don’t get me wrong, “engagement” or happy customers is vital too, just not the end all. Engaged customers do spread that ‘word’ around. And CIO types ask other CIOs and pick up the general vibe. And “engagement” gives a product a sense of being alive and provides a support framework.

  23. While I see the value for better overall definition of an audience, “engagement” falls into that vague “eyeballish” zone. It doesn’t really mean anything. And “engaged” exactly how, as ‘engaged’ audiences might be twice the effort and cost, for the same result. Maybe for some advertisers better to narrow on the extreme targeted or busy types who want raw facts fast. You can’t expect the CFO’s to play engagement games.

    Some people are busy running companies and don’t want to cuddle up with vendors all the live day long. If said product doesn’t work or doesn’t meet needs or isn’t at a good pricepoint, they will just get a new vendor. And sometimes (in fact, imho, most often) the most engaged audiences are those with the least amount of power (as they have the raw time to engage). Don’t discount the busy and aloof, that’s sometimes the real gold.

    And you can’t really compare USA Today audiences with bloggers, maybe only a few clicked thru, but maybe one was a busy CIO (who has no time for blog chatter) but saw it in USA Today when they shoved it under his hotel door, a CIO who decides to order 100 copies. Clicks aren’t the end-all, that’s too blogger nose-leveled view.

    The only metric (to the advertiser that counts) is increased marketshare, engaged or not. But don’t get me wrong, “engagement” or happy customers is vital too, just not the end all. Engaged customers do spread that ‘word’ around. And CIO types ask other CIOs and pick up the general vibe. And “engagement” gives a product a sense of being alive and provides a support framework.

  24. PS – Getting Steve Gillmor on anything would just buzzword it all up, angel-dust freebasing pure unintelligible nonsense, and scare off 90% of advertisers. You really think anyone from Peoria would understand a word Steve says?

    And Steve is well just loony and agenda-driven, witness…

    TV is dead because of the Internet. TV is dead because we don’t have time for it.

    Geee really? With Full Season DVDs, with Tivo/Replay, Media Center’s and PVR’s de jour, with Portable Video Players and iTunes and ilk, and even gray-area MPEG4s, (XVID, DIVX) formats on MASSIVE Torrent swarms, we have MUCH MUCH more time than ever before. The Internet ENHANCES TV, makes it MORE valuable, it provides that “engagement” level. The long-story arc becomes more viable, giving us better TV; death to the sitcomy episodic. Heck, has he never met the Battlestar Galactica, Veronica Mars, Lost or 4400 fan? People spend hours on forums talking, debating about TV. And what does he think makes up 50% of YouTube….TV clips. Oh we are watching, much much more than before, the big question is how to fund it all, those models are changing.

  25. PS – Getting Steve Gillmor on anything would just buzzword it all up, angel-dust freebasing pure unintelligible nonsense, and scare off 90% of advertisers. You really think anyone from Peoria would understand a word Steve says?

    And Steve is well just loony and agenda-driven, witness…

    TV is dead because of the Internet. TV is dead because we don’t have time for it.

    Geee really? With Full Season DVDs, with Tivo/Replay, Media Center’s and PVR’s de jour, with Portable Video Players and iTunes and ilk, and even gray-area MPEG4s, (XVID, DIVX) formats on MASSIVE Torrent swarms, we have MUCH MUCH more time than ever before. The Internet ENHANCES TV, makes it MORE valuable, it provides that “engagement” level. The long-story arc becomes more viable, giving us better TV; death to the sitcomy episodic. Heck, has he never met the Battlestar Galactica, Veronica Mars, Lost or 4400 fan? People spend hours on forums talking, debating about TV. And what does he think makes up 50% of YouTube….TV clips. Oh we are watching, much much more than before, the big question is how to fund it all, those models are changing.

  26. Very good points. Comscore vs Danah on Myspace stats recently got into the challenges facing the difference between a user who simply loads a page and leaves vs a user who engages with the site.

    Experiments are needed, since it may be as simple as taking a ratio of total unique visitors to total time online to get a sense of how engaged the visitors are.

    Of course that does NOT necessarily translate into a buyer. I’m engaged here at Scobleizer, but not looking at your ads.

  27. Very good points. Comscore vs Danah on Myspace stats recently got into the challenges facing the difference between a user who simply loads a page and leaves vs a user who engages with the site.

    Experiments are needed, since it may be as simple as taking a ratio of total unique visitors to total time online to get a sense of how engaged the visitors are.

    Of course that does NOT necessarily translate into a buyer. I’m engaged here at Scobleizer, but not looking at your ads.

  28. Hey, I was wrong – I did know you advertised Naked Conversations (which I have and enjoyed reading). You have no other ads up so I can’t say I didn’t see them!

  29. Hey, I was wrong – I did know you advertised Naked Conversations (which I have and enjoyed reading). You have no other ads up so I can’t say I didn’t see them!

  30. I support the local climbing shop for a reason. Even though the local price is higher than online. Sorry it has taken me so long to purchase your book Robert.

    It is my humble way of saying I support your efforts to enlighten the messes ; ).

  31. I support the local climbing shop for a reason. Even though the local price is higher than online. Sorry it has taken me so long to purchase your book Robert.

    It is my humble way of saying I support your efforts to enlighten the messes ; ).

  32. [...] Far be it from me to take issue with Jeneane Sessum, who doesn’t suffer fools gladly, or with Robert Scoble for that matter, but until we can measure likeability or engagement then downloads will have to do. And it seems obvious that unless Andrew Baron of Rocketboom is a complete and utter liar when he talks about his statistics, Rocketboom is leagues ahead of Ze Frank. This all stems from the recent throwdown between Ze and the Rocket. [...]

  33. [...] Robert Scoble has an interesting post on the need of measuring the engagement level of the users of a web site. He talks about the difference in user engagement between the Register and Digg: Well, I’ve compared notes with several bloggers and journalists and when the Register links to us we get almost no traffic. But they claim to have millions of readers. So, if millions of people are hanging out there but no one is willing to click a link, that means their audience has low engagement. The Register is among the lowest that I can see. [...]

  34. Seriously? Last time I got linked from the Register, there was an avalanche which made Slashdot look like some kid’s Livespace journal.

    Perhaps you just got a link from one of Orlowski’s pages. Nobody reads those. :)

  35. Seriously? Last time I got linked from the Register, there was an avalanche which made Slashdot look like some kid’s Livespace journal.

    Perhaps you just got a link from one of Orlowski’s pages. Nobody reads those. :)

  36. Scoble Diggs Up Cost Per Influence

    Robert Scoble asks:How could we measure audience engagement?What he is after isn’t engagement, but influence. Here’s the answer I’ve been trying to encourage others to implement: Metric: Cost Per Influence Format: Sell Side Advertising

  37. Three thoughts:

    LIKEABILITY see http://en.wikipedia.org/wiki/Q_Score, although I can’t say much for Nielsen Ratings these days, maybe next year

    ACTIONABILITY go beyond GestureLab, comSCORE seems overly qualified

    INFLUENCE POWER wait for the algo look toward political marketing agencies bet on real social research that seeks to solve real problems beyond meme optimization

  38. Three thoughts:

    LIKEABILITY see http://en.wikipedia.org/wiki/Q_Score, although I can’t say much for Nielsen Ratings these days, maybe next year

    ACTIONABILITY go beyond GestureLab, comSCORE seems overly qualified

    INFLUENCE POWER wait for the algo look toward political marketing agencies bet on real social research that seeks to solve real problems beyond meme optimization

  39. [...] What’s more amusing? Scoble and New Media folks discover “engagement,” a term that the old advertising establishment has been “engaged” with for quite some time. Or, that hot and utterly hip video blogging has been caught up in a he said, he said spat over audience measurement. Welcome to media! These guys sound like a bunch of stuffy old TV networks. [...]

  40. hot potato! it is an area oft tread but not thoroughly reviewed nor defined. investigations have been cursory for predefined needs. will look forward to seeing all that this elicits.

  41. hot potato! it is an area oft tread but not thoroughly reviewed nor defined. investigations have been cursory for predefined needs. will look forward to seeing all that this elicits.

  42. Robert,

    We could not agree with you more.

    Compete (www.compete.com) just released a product that features 13 month traffic trends on any and every site our member community has ever visited… Including two very important components of engagement – Average Stay in minutes and Pages viewed per Visit.

    *Think Alexa on steroids*.

    Disclosure: Compete is my Employer

  43. Robert,

    We could not agree with you more.

    Compete (www.compete.com) just released a product that features 13 month traffic trends on any and every site our member community has ever visited… Including two very important components of engagement – Average Stay in minutes and Pages viewed per Visit.

    *Think Alexa on steroids*.

    Disclosure: Compete is my Employer

  44. Again, I am reminded why I like Chris Coulter. It is because he is smart.

    ‘Engagement’ is difficult to define, not to mention measure. And, truth be told, just going where the herd is directed is very common in the blogoshpere. I remember being pleased to be a popular blogger back in the day, only to be surprised by how little time most readers spent at my blog when they visited. Often, it was not long enough for them to have actually read a single entry in full. On the other hand, merely leaving a page open for a longer time can be confused with being engaged. Often, the person just forgot to close that window in his browser.

    As for likeability, that is an invitation for superficiality. Often people who are intelligent and know their craft well are not particularly likeable. I would rather read or watch that kind of blogger than someone considered ‘nice.’

  45. Again, I am reminded why I like Chris Coulter. It is because he is smart.

    ‘Engagement’ is difficult to define, not to mention measure. And, truth be told, just going where the herd is directed is very common in the blogoshpere. I remember being pleased to be a popular blogger back in the day, only to be surprised by how little time most readers spent at my blog when they visited. Often, it was not long enough for them to have actually read a single entry in full. On the other hand, merely leaving a page open for a longer time can be confused with being engaged. Often, the person just forgot to close that window in his browser.

    As for likeability, that is an invitation for superficiality. Often people who are intelligent and know their craft well are not particularly likeable. I would rather read or watch that kind of blogger than someone considered ‘nice.’

  46. One of the important measures of engagement is the ratio between posts and comments as well as the links to post ratio. This can be further determined by some of the attention issues around number of posts bookmarked and number of bookmarkers.

    But I dont honestly know if engagement is the thing – it is a better thing to strive for then reach and popularity, but I tihnk we can do better even.

    Supposedly the big advertising associations are thinking about this same metric (engagement via social media) but I don’t trust them with it. It is kind of like asking the car salesman which bicycle to buy…

  47. One of the important measures of engagement is the ratio between posts and comments as well as the links to post ratio. This can be further determined by some of the attention issues around number of posts bookmarked and number of bookmarkers.

    But I dont honestly know if engagement is the thing – it is a better thing to strive for then reach and popularity, but I tihnk we can do better even.

    Supposedly the big advertising associations are thinking about this same metric (engagement via social media) but I don’t trust them with it. It is kind of like asking the car salesman which bicycle to buy…

  48. [...] Robert Scoble looks at the need for better web metrics, focusing on a need to measure “engagement” on a website. He illustrates his point comparing two sites, and what would happen if they added a link to your website on their page. He posits that a site such as The Register would result in few clicks over to you, which a mention on Digg would send a ton of traffic your way. [...]

  49. Engagement is a difficult thing to measure as it is so dependent on time. For me the best way to tell if I have been ‘engaged’ by a site (or anything else for that matter) is to see if I am still using it after 2 or 3 months. Humans love following fads (hence web 2.0!) and I find myself using new sites/products intensively for a matter of days or weeks before moving on to something else. Only a very small minority of these will see me returning consistently over a long period of time. I’ve no idea how you are meant to measure that!

  50. Engagement is a difficult thing to measure as it is so dependent on time. For me the best way to tell if I have been ‘engaged’ by a site (or anything else for that matter) is to see if I am still using it after 2 or 3 months. Humans love following fads (hence web 2.0!) and I find myself using new sites/products intensively for a matter of days or weeks before moving on to something else. Only a very small minority of these will see me returning consistently over a long period of time. I’ve no idea how you are meant to measure that!

  51. Engagement

    What Scobe says is right. Its not simply the size of the audience, but the level of engagement that matters.
    Scoble explains:
    Well, I’ve compared notes with several bloggers and journalists and when the Register links to us we get almost no traffic. …

  52. I’m amazed at all of the web navel gazing here and that no one has stepped back and asked the obvious question. What is the difference between USA Today readers and Robert’s readers? That is the simple Marketing 101 question. Robert’s readers are by definition more technically minded and much more likely to be attracted to ActiveWords as a product. It’s not a question of engagement, it’s a question of segmentation and matching the product’s attraction to the most profitable market subsegments. What Buzz is actually saying is that USA Today readership is a broad market with a low Heavy Usage Index (a measure of the relative intensity of consumption) for the software product category under which Active Words falls. Clearly Scoble’s readers who are largely techs and business people who work for tech companies are focused on the cutting edge and are much more likely to purchase Active Words because they are willing to implement a productivity solution on their computer and futz around with Macros. Engagement? That is a contrived metric because it pretends to say that it measures how willing people are to “engage” with a website, when the reality changes depending on the consumer segment who is attracted to your particular market. The true measurements of interest should be metrics like purchase habits, loyalty, heavy usage index, and segment utilities. When you are able to properly target your products, then engagement comes naturally. So the real question is, how tightly does a website align with the key interests that define your highest potential consumers and how frequently do they come back to it.

  53. I’m amazed at all of the web navel gazing here and that no one has stepped back and asked the obvious question. What is the difference between USA Today readers and Robert’s readers? That is the simple Marketing 101 question. Robert’s readers are by definition more technically minded and much more likely to be attracted to ActiveWords as a product. It’s not a question of engagement, it’s a question of segmentation and matching the product’s attraction to the most profitable market subsegments. What Buzz is actually saying is that USA Today readership is a broad market with a low Heavy Usage Index (a measure of the relative intensity of consumption) for the software product category under which Active Words falls. Clearly Scoble’s readers who are largely techs and business people who work for tech companies are focused on the cutting edge and are much more likely to purchase Active Words because they are willing to implement a productivity solution on their computer and futz around with Macros. Engagement? That is a contrived metric because it pretends to say that it measures how willing people are to “engage” with a website, when the reality changes depending on the consumer segment who is attracted to your particular market. The true measurements of interest should be metrics like purchase habits, loyalty, heavy usage index, and segment utilities. When you are able to properly target your products, then engagement comes naturally. So the real question is, how tightly does a website align with the key interests that define your highest potential consumers and how frequently do they come back to it.

  54. Most metrics from enterprise system administrators gage logon’s and time in the system as key indicators of favorable use of the system. Which is bunk, mainly due to it not indicating problems with crashes, analysis stalling, etc etc.

    I agree with the “likeability metric” per user. The system “sucks or not” and reasons why would give better feedback to key developers and CIO’s without the need for costly and ineffective surveys which can be fudged or skewed in favor of the old “what do you want it to be?” guys which may be linked to profit driven support.

    RSS KISS SOS. INNOVATE don’t stagnate. This new conversation media is a step in the right direction to both consumer satisfaction and bottom-line profit if used correctly. IMHO and probably wrong O. No flaming required. ; )

  55. Most metrics from enterprise system administrators gage logon’s and time in the system as key indicators of favorable use of the system. Which is bunk, mainly due to it not indicating problems with crashes, analysis stalling, etc etc.

    I agree with the “likeability metric” per user. The system “sucks or not” and reasons why would give better feedback to key developers and CIO’s without the need for costly and ineffective surveys which can be fudged or skewed in favor of the old “what do you want it to be?” guys which may be linked to profit driven support.

    RSS KISS SOS. INNOVATE don’t stagnate. This new conversation media is a step in the right direction to both consumer satisfaction and bottom-line profit if used correctly. IMHO and probably wrong O. No flaming required. ; )

  56. [...] Update: More conversation by Jeneane Sessum on likeability. I thought she was going another way until the very end because Ze Frank has a lot of positives but likeability isn’t one I’d ascribe – more like “manic creepy charisma.”  Perhaps she means the show as a whole is likeable. I think it is offputting by design rather than likeable but however you want to score it. Scoble suggests engagement as a metric which is not unlike what I’ve been saying for the last two years. [...]

  57. [...] Over the past few days, Ze Frank and Andrew Baron have had a bit of a catfight over the actual viewers of Rocketboom. It began in Ze’s “the show” two days ago, and was quickly countered by Andrew Baron. The story was picked up in the blogosphere, and was covered by video blogging heavyweights like Robert Scoble. [...]

  58. [...] This phenomenon is mirrored online. Blogebrity Robert Scoble expressed a desire on his blog that some algorithmic genius develop a measure of engagement, or likeability, to better represent the impact of bloggers. He speaks from personal experience to illustrate the power of the online relationship: [...]

  59. [...] This phenomenon is mirrored online. Blogebrity Robert Scoble expressed a desire on his blog that some algorithmic genius develop a measure of engagement, or likeability, to better represent the impact of bloggers. He speaks from personal experience to illustrate the power of the online relationship: [...]

  60. As someone who has worked in web metrics for the past four years, I can tell you all web metrics are flawed. I can go one further and say all off-line media metrics are flawed as well.

    Metrics are a currency for valuing advertising and media properties. The only thing a metric needs to be is consistent and accepted as a unit of measure between the those selling and buying media properties and advertising.

    That’s way easier said than done. Setting a metrics standard can put the thumb on the scale in favor of one player or another – Rocketboom or Zefrank.

    A metric’s only good if it can be accepted as a medium of exchange.

    This issue is something the web analytics association has been grappling with for a while.

    http://www.webanalyticsassociation.org/

    Enrique

  61. As someone who has worked in web metrics for the past four years, I can tell you all web metrics are flawed. I can go one further and say all off-line media metrics are flawed as well.

    Metrics are a currency for valuing advertising and media properties. The only thing a metric needs to be is consistent and accepted as a unit of measure between the those selling and buying media properties and advertising.

    That’s way easier said than done. Setting a metrics standard can put the thumb on the scale in favor of one player or another – Rocketboom or Zefrank.

    A metric’s only good if it can be accepted as a medium of exchange.

    This issue is something the web analytics association has been grappling with for a while.

    http://www.webanalyticsassociation.org/

    Enrique

  62. Robert – add to your equation “relevancy” which people seem to be forgetting. Even if your readers click on my link to Diva Marketing (engagement) they will not stay long, go deep or come back unless the content is relevant to them.

    So while engagement is the first step relevancy is critical and should be integrated into any measurement model. Without it we are counting old fashioned ‘hits’ which may be great for the ego but from a business stance is a big So What.

  63. Robert – add to your equation “relevancy” which people seem to be forgetting. Even if your readers click on my link to Diva Marketing (engagement) they will not stay long, go deep or come back unless the content is relevant to them.

    So while engagement is the first step relevancy is critical and should be integrated into any measurement model. Without it we are counting old fashioned ‘hits’ which may be great for the ego but from a business stance is a big So What.

  64. Why has nobody mentioned YouTube in this context yet? I know when I look at a video (before I hit PLAY) I see how many video responses it’s gotten, how many times favorited, how many comments, the star rating, etc, rather than just number of viewings.

    They do an excellent job of articulating the degree of audience engagement for each and every video, I just wish there was some way to search for “most video responses” videos, etc.

  65. Why has nobody mentioned YouTube in this context yet? I know when I look at a video (before I hit PLAY) I see how many video responses it’s gotten, how many times favorited, how many comments, the star rating, etc, rather than just number of viewings.

    They do an excellent job of articulating the degree of audience engagement for each and every video, I just wish there was some way to search for “most video responses” videos, etc.

  66. Hi Robert (and all other readers).
    I’m the CEO of a NuConomy. You probably didn’t hear about us until now as we were in stealth mode until a few days ago.
    Our offering is a platform for Internet sites that does just that – Gives you a sophisticated generic way to develop your own engagement formulas (we call it ranking for now.
    By using our platform you can rank your users and learn more about your business, and then also run financial schemes upon the results (rewards programs, revenue sharing, etc…).
    I would love to hear what you think about our offering (the good and the bad). Feel free to check out our web site or contact me directly in shahar at nuconomy.com

  67. Hi Robert (and all other readers).
    I’m the CEO of a NuConomy. You probably didn’t hear about us until now as we were in stealth mode until a few days ago.
    Our offering is a platform for Internet sites that does just that – Gives you a sophisticated generic way to develop your own engagement formulas (we call it ranking for now.
    By using our platform you can rank your users and learn more about your business, and then also run financial schemes upon the results (rewards programs, revenue sharing, etc…).
    I would love to hear what you think about our offering (the good and the bad). Feel free to check out our web site or contact me directly in shahar at nuconomy.com

  68. Security, Control, Trackability in Online Media Distribution

    The Ze Frank vs. Rocketboom video blog popularity contest sparked a lot of chatter this past week on how to best measure the value of web-based media shows (podcasts / vlogs / vodcasts / whatever).  Is it possible that Ze Frank’s shows are wo…

  69. It goes back to the whole idea of mind share that was so hot back in the dot-com era. A site can get a lot of hits, and you can interpret it as a measure of popularity and awareness of the brand. However, does it actually mean anything? Are users actually going to come back or even remember your site? The current system of measuring audience engagement is a throwback to older methods of measuring ratings, but better tools can now be deployed to more fully gauge what something means to an audience.

  70. It goes back to the whole idea of mind share that was so hot back in the dot-com era. A site can get a lot of hits, and you can interpret it as a measure of popularity and awareness of the brand. However, does it actually mean anything? Are users actually going to come back or even remember your site? The current system of measuring audience engagement is a throwback to older methods of measuring ratings, but better tools can now be deployed to more fully gauge what something means to an audience.

  71. The Danger of Communities

    I was reading Robert Scoble’s post where he talked about the need to measure ‘engagement’ more than just hits or downloads and it made me think about Chill. You might be aware that i’ve helped out with Chill, it’s one of our new digital stations. …

  72. [...] Es ist kein Geheimnis, dass die Werbetreibenden immer mehr in Richtung Cost-per-Click oder Cost-per-Action Modelle tendieren. Mit ein Grund ist natürlich, dass die anderen Modelle sehr leicht manipuliert werden können bzw. sehr schwer zu messen sind. Während der Diskussion über die Reichweite von Rockeboom erlaubte Andrew Baron, der Produzenten von Rocketboom, verschiedenen Leuten die Severlogs von Rocketboom einzusehen (die Excel-Datei gibt es hier). Aus diesen Logfiles geht hervor, dass Rocketboom im Oktober durchschnittlich 211 000 komplette Downloads hatte. Nimmt man diese Zahlen hat Rocketboom einen CPM von $75 (die Werbung für eine Woche kostet $80 000). Soweit so gut, nur geht Baron leider davon aus, dass diese Downloads auch von 211 000 Zuschauern gesehen wurden und hier liegt der Fehler: Rocketboom hat einen Autoplay, ist in vielen Vodcasts Clients als Standard hinterlegt und wird von TiVos usw automatisch heruntergeladen. Zudem hat die BBC bei einem Experiment festgestellt, dass gerade einmal jeder zweite Download auch gesehen wird und nehmen wir das an halbiert sich das Publikum von Rocketboom und der CPM schießt auf $150. Die Werbetreibenden zahlen also für eine Reichweite, die sie in der Form gar nicht erhalten. Das heißt jedoch nicht, dass die Werbung bei Rocketboom die $80 000 nicht wert ist, sonder nur dass die Messung schlecht und ungeeignet ist. Nicht umsonst fordert Robert Scoble im Hinblick auf Videoblogs: New audience metric needed: engagement.Noch einfacher kann man die Problematik an Text Link Ads verdeutlichen, deren Link Worth Calculator nimmt als Basis der Berechnung das Alexa-Ranking. Hat man vor der Anmeldung ein wenig mit der Alexa-Toolbar gespielt sind die zu erwartenden Einnahmen um ein vielfaches höher als der eigentliche Wert der Links. [...]

  73. [...] “How do you measure engagement” on Robert Scoble’s blog: “I’ve compared notes with several bloggers and journalists and when the Register links to us we get almost no traffic. But they claim to have millions of readers. So, if millions of people are hanging out there but no one is willing to click a link, that means their audience has low engagement. (…) Compare that to Digg. How many people hang out there every day? Maybe a million, but probably less. Yet if you get linked to from Digg you’ll see 30,000 to 60,000 people show up. And these people don’t just read. They get involved. I can tell when Digg links to me cause the comments for that post go up too.” [...]

  74. [...] Scobleizer – New audience metric needed: engagement: Well, I’ve compared notes with several bloggers and journalists and when the Register links to us we get almost no traffic. But they claim to have millions of readers. So, if millions of people are hanging out there but no one is willing to click a link, that means their audience has low engagement. The Register is among the lowest that I can see. [...]

  75. [...] Some experts even think we should find ways to measure engagement. Tech Evangelist Robert Scoble explains why: As an advertiser I want to talk to an audience who’ll actually DO something. Yeah, I’m hoping to get a sale. [...]

  76. Is the difficulty of measuring engagement inherent to the browser or the browser-based way we engage on the internet? It seems users need to be empowered in some way to measure their own level of engagement – but it needs to happen on the client side because so much of it is opaque to the server.

  77. Is the difficulty of measuring engagement inherent to the browser or the browser-based way we engage on the internet? It seems users need to be empowered in some way to measure their own level of engagement – but it needs to happen on the client side because so much of it is opaque to the server.