Should services charge “super users”?

Om Malik says that Twitter should charge super users like me and come up with a business model.

Dare Obasanjo, in a separate, but similar post comes to the conclusion that Twitter’s problems are due to super users like me.

Interesting that both of these guys are wrong.

First of all, Twitter doesn’t store my Tweets 25,000 times. It stores them once and then it remixes them. This is like saying that Exchange stores each email once for each user. That’s totally not true and shows a lack of understanding how these things work internally.

Second of all, why can FriendFeed keep up with the ever increasing load? I have 10,945 friends on FriendFeed (all added in the past three months, which is MUCH faster growth than Twitter had) and it’s staying up just fine.

But to the point, why not charge super users? I’d pay. But, if Dare and Om are right, there’s no way that I’d support the service enough to pay for my real cost on the service.

Either way, Twitter’s woes were happening long before my account got super huge. Remember SXSW last year? I only had 500 followers and Leo Laporte had something like 800. The service still went down. If this were a straight “n-scale” problem the crashing problems wouldn’t have shown up so early.

Why not just limit account size, like Facebook did? Well, that’s one way to deal with the problem, but if you look at my usage of Facebook it’s gone down to only a few minutes every month. I don’t even answer messages there anymore. Why? Cause I get frustrated at getting messages from people who wonder why I won’t accept them as a friend. It’s no business “utility” if I can’t make infinitely large friend lists and use those lists in the same way I use email (which Facebook also bans).

So, what do I do? I get excited by FriendFeed which lets 11,000 people interact with me in a public way. I have a feeling that that rapid growth will continue unabated and so far Friendfeed has stayed “Google fast.”

Nice try, though.

139 thoughts on “Should services charge “super users”?

  1. Mr. Scoble,

    I must say that I have not really cared enough to read your column in the past. After reading this post, however, I will make sure never to read anything else you choose to write. What I see is a person that is clearly ignorant about a complex set of topics related to application design and scalability, speaking sophomorically about them. Perhaps you should take some time, in your case, a great deal of it, and educate yourself about these matters before speaking great volumes of nonsense relating to the technical implementation of this or any other application.

    Start by reading the comments here and questioning some of the very smart people, who have graciously taken the time to try and educate you. Please, for the sake of the thousands of people that clearly believe you to be an authority on matters of technology, stop this idiocy.

  2. Mr. Scoble,

    I must say that I have not really cared enough to read your column in the past. After reading this post, however, I will make sure never to read anything else you choose to write. What I see is a person that is clearly ignorant about a complex set of topics related to application design and scalability, speaking sophomorically about them. Perhaps you should take some time, in your case, a great deal of it, and educate yourself about these matters before speaking great volumes of nonsense relating to the technical implementation of this or any other application.

    Start by reading the comments here and questioning some of the very smart people, who have graciously taken the time to try and educate you. Please, for the sake of the thousands of people that clearly believe you to be an authority on matters of technology, stop this idiocy.

  3. Life is a very short journey and we are not here to waste it in using Twitter, Facebook, iPhone, Flickr,ETC.. then simple die.I just don’t get it how do we benefits the community and the people by using these things what is the meaning of these things in life. people are dying all over the world and we are using Twitter to stay in touch with friends.

  4. Life is a very short journey and we are not here to waste it in using Twitter, Facebook, iPhone, Flickr,ETC.. then simple die.I just don’t get it how do we benefits the community and the people by using these things what is the meaning of these things in life. people are dying all over the world and we are using Twitter to stay in touch with friends.

  5. “First of all, Twitter doesn’t store my Tweets 25,000 times. It stores them once and then it remixes them.”

    I do know how Twitter is built. However, “back in the day,” I was the development manager for a real-time, stock quote delivery system, so I do have some experience with architectural issues Twitter may be facing.

    Let’s look at the procedure Robert refers to as “remixes them.” In the simplest architecture, there would be a single list (database, flat file, etc) of all the twitters created by everyone stored in chronological order. You may, as a storage optimization, just store a user id with the twitter string, and tweet time stamp (aka a tweet).

    In this single architecture, a “remix” would require a query across all the tweets for a period of time for all people that a user follows. This query would be fairly fast when the number of tweets in the specified period of time is fairly small, and the number of users a person follows is fairly small. You can see that this type of query becomes more expensive when the number of users you follow increases and the overall number of tweets per period increases.

    So to speed up this query, you could build some kind of index based on users. But maintaining this index would become expensive, especially during high incoming tweet periods.

    So one might try to optimize this architecture by breaking up the universal store into list of tweets per person. Now each incoming tweet can be easily added to the user’s tweet list.

    Then the “remix” of tweets of the people you follow would require a join across each list and then sorted by chronological order. This would become increasingly more expensive when a user starts to increase the number of people they follow. It would be particularly expensive for super users who follow lots of users.

    A reasonable compromise might be to keep a single universal stream of tweets in chronological order and two lists for each user: a list of pointers of all their tweets, and a list of pointers to all the tweets from the people they follow.

    Maintaining these three lists would look something like: sender publishes a tweet, it is added to the universal store, a pointer is then added to the sender’s tweet list, and then “push the tweet to followers” by walking sender’s list of followers and add a pointer to the tweet to each “follow” list.

    This approach scales fairly well. It allows the act of updating the follow lists to be partitioned across multiple servers. Each server can just take (using shared queues) a tweet from the universal store and “fan it out” to the appropriate followers. It also can be separates the operation from the inbound tweet processing.

    To optimize the “fan it out” process, messaging publish and subscribe product like JMS or TIBCO Rendezvous and broadcast the tweets to the servers that manage follow lists. This would require a universal store process to publish all tweets and a cloud of follow list managers listening (aka subscribing) to tweet broadcasts updates for each followed person.

    This approach also nicely addresses Twitter’s need to separate outbound follower queues for users that have requested point to point delivery of messages via Instant Messaging and SMS.

    For further scaling optimzation, you can have several tweet stores instead of one single universal store. You just need to ensure that all incoming tweets from a particular user are added to the same store to maintain ordered delivery to followers.

    So it is quite reasonable to copy (at least references) each of Robert’s tweets 25,000 times, just do so in a scalable manner.

  6. “First of all, Twitter doesn’t store my Tweets 25,000 times. It stores them once and then it remixes them.”

    I do know how Twitter is built. However, “back in the day,” I was the development manager for a real-time, stock quote delivery system, so I do have some experience with architectural issues Twitter may be facing.

    Let’s look at the procedure Robert refers to as “remixes them.” In the simplest architecture, there would be a single list (database, flat file, etc) of all the twitters created by everyone stored in chronological order. You may, as a storage optimization, just store a user id with the twitter string, and tweet time stamp (aka a tweet).

    In this single architecture, a “remix” would require a query across all the tweets for a period of time for all people that a user follows. This query would be fairly fast when the number of tweets in the specified period of time is fairly small, and the number of users a person follows is fairly small. You can see that this type of query becomes more expensive when the number of users you follow increases and the overall number of tweets per period increases.

    So to speed up this query, you could build some kind of index based on users. But maintaining this index would become expensive, especially during high incoming tweet periods.

    So one might try to optimize this architecture by breaking up the universal store into list of tweets per person. Now each incoming tweet can be easily added to the user’s tweet list.

    Then the “remix” of tweets of the people you follow would require a join across each list and then sorted by chronological order. This would become increasingly more expensive when a user starts to increase the number of people they follow. It would be particularly expensive for super users who follow lots of users.

    A reasonable compromise might be to keep a single universal stream of tweets in chronological order and two lists for each user: a list of pointers of all their tweets, and a list of pointers to all the tweets from the people they follow.

    Maintaining these three lists would look something like: sender publishes a tweet, it is added to the universal store, a pointer is then added to the sender’s tweet list, and then “push the tweet to followers” by walking sender’s list of followers and add a pointer to the tweet to each “follow” list.

    This approach scales fairly well. It allows the act of updating the follow lists to be partitioned across multiple servers. Each server can just take (using shared queues) a tweet from the universal store and “fan it out” to the appropriate followers. It also can be separates the operation from the inbound tweet processing.

    To optimize the “fan it out” process, messaging publish and subscribe product like JMS or TIBCO Rendezvous and broadcast the tweets to the servers that manage follow lists. This would require a universal store process to publish all tweets and a cloud of follow list managers listening (aka subscribing) to tweet broadcasts updates for each followed person.

    This approach also nicely addresses Twitter’s need to separate outbound follower queues for users that have requested point to point delivery of messages via Instant Messaging and SMS.

    For further scaling optimzation, you can have several tweet stores instead of one single universal store. You just need to ensure that all incoming tweets from a particular user are added to the same store to maintain ordered delivery to followers.

    So it is quite reasonable to copy (at least references) each of Robert’s tweets 25,000 times, just do so in a scalable manner.

  7. Does it really cost that much to host someone’s Tweets, even if they are quite frequent? If so, that seems like an inefficient model. I feel like most sites practice some form of economies of scale to where frequent users actually become more affordable (and therefore receive price breaks)… so, for example, Twitter might charge 2 cents for every 10 posts to relatively infrequent users like myself, and 2 cents for every 100 posts to frequent updaters. Keeping costs low for busy Twitterers would also increase business for them, since people are more likely to check your site frequently when it is frequently updated. If Twitter has difficulties understanding this, I can recommend a Basic Economics textbook (ebooks version).

  8. Does it really cost that much to host someone’s Tweets, even if they are quite frequent? If so, that seems like an inefficient model. I feel like most sites practice some form of economies of scale to where frequent users actually become more affordable (and therefore receive price breaks)… so, for example, Twitter might charge 2 cents for every 10 posts to relatively infrequent users like myself, and 2 cents for every 100 posts to frequent updaters. Keeping costs low for busy Twitterers would also increase business for them, since people are more likely to check your site frequently when it is frequently updated. If Twitter has difficulties understanding this, I can recommend a Basic Economics textbook (ebooks version).

  9. I don’t see why not to charge the supertweeters, they pay whoever hosts their blogs, don’t they? They’re using Twitter as their personal blogging platform, no difference.

  10. I don’t see why not to charge the supertweeters, they pay whoever hosts their blogs, don’t they? They’re using Twitter as their personal blogging platform, no difference.

  11. Wow. Lots of interesting comments. Then again, lots of BLAH, BLAH, BLAH.

    Here’s a fact that people are overlooking. Traffic brings revenue. Let’s say that the 25,000 posts get counted. What does that really amount to?

    That means that 25,000 people are looking at what Robert Scoble is saying. If I was a person that wants to get out my product name, then I think I would pay Twitter to keep the service going. Better yet I might ask and pay Robert to push my wares.

    About a month back I @(replied) Robert on something. I believe it was during one of many Twitters’ “Problems”. The exchange was short and sweet. However, I looked at my Followers an hour later, it jumped up 15 (which it doesn’t normally do).

    I tested the water by @ another person. The same thing happened. I gain more followers by replying to high profile twitters.

    Now, apply that all to a marketing model. Communication can mean $$. I guess that’s why Twitter was able to raise 20 million on it’s own.

    The problem isn’t the Twitterflood. If that was the case then sites like MySpace and Facebook would be going down on a daily basis. If it DOES do what OM Malik suggests, then Twitter needs to look at their internal structure. Not at Robert Scoble, or Leo Laporte.

    Limits and subscription fees are a great way to kill the idea. Some will pay for it, while others will say “See ya”. Twitter will fall like a ball of flame into the Pacific Ocean.

    They keep the idea fresh. To most, Twitter is an “Oh, I heard of that”. People might know about it, but never signed up. Oncemore, Twitter can easily become a cash cow. The data that comes into Twitter is like when Daffy Duck found the Sultan’s cave.

    I’M RICH! I’M WEALTY BEYOND MY WILDEST DREAMS!!!…

    Keep going Scoble. I’m listening…

  12. Wow. Lots of interesting comments. Then again, lots of BLAH, BLAH, BLAH.

    Here’s a fact that people are overlooking. Traffic brings revenue. Let’s say that the 25,000 posts get counted. What does that really amount to?

    That means that 25,000 people are looking at what Robert Scoble is saying. If I was a person that wants to get out my product name, then I think I would pay Twitter to keep the service going. Better yet I might ask and pay Robert to push my wares.

    About a month back I @(replied) Robert on something. I believe it was during one of many Twitters’ “Problems”. The exchange was short and sweet. However, I looked at my Followers an hour later, it jumped up 15 (which it doesn’t normally do).

    I tested the water by @ another person. The same thing happened. I gain more followers by replying to high profile twitters.

    Now, apply that all to a marketing model. Communication can mean $$. I guess that’s why Twitter was able to raise 20 million on it’s own.

    The problem isn’t the Twitterflood. If that was the case then sites like MySpace and Facebook would be going down on a daily basis. If it DOES do what OM Malik suggests, then Twitter needs to look at their internal structure. Not at Robert Scoble, or Leo Laporte.

    Limits and subscription fees are a great way to kill the idea. Some will pay for it, while others will say “See ya”. Twitter will fall like a ball of flame into the Pacific Ocean.

    They keep the idea fresh. To most, Twitter is an “Oh, I heard of that”. People might know about it, but never signed up. Oncemore, Twitter can easily become a cash cow. The data that comes into Twitter is like when Daffy Duck found the Sultan’s cave.

    I’M RICH! I’M WEALTY BEYOND MY WILDEST DREAMS!!!…

    Keep going Scoble. I’m listening…

  13. Does anyone *actually* know how Twitter does what is does? Given the 140 byte limit, Twitter seems very “doable” with some basic design choices. This is why I think Ruby/Rails may really be a culprit here…it’s too high level to support some of the things Twitter needs to do.

  14. Does anyone *actually* know how Twitter does what is does? Given the 140 byte limit, Twitter seems very “doable” with some basic design choices. This is why I think Ruby/Rails may really be a culprit here…it’s too high level to support some of the things Twitter needs to do.

  15. Like they say, you get what you pay for. You want better SLA’s and uptime? Crack open your wallet.

  16. Like they say, you get what you pay for. You want better SLA’s and uptime? Crack open your wallet.

  17. With Nick’s explanation on why de-normalization is needed to scale, it is clear one of the complex issues Twitter has to solve. The other one mentioned in this post is the business model, or when to charge.

    Angus has the right idea – charging the followers – although I don’t agree on the analysis. Still using Robert as the super-user, he should not be charged because of his tweets, but for the number of people he is following. Each tweet sent by the friends Robert is following will be copied on his queue (well the tweet ID) and the size and freshness of this queue (visually the ‘With Others’) can be used as the factor to charge.

  18. With Nick’s explanation on why de-normalization is needed to scale, it is clear one of the complex issues Twitter has to solve. The other one mentioned in this post is the business model, or when to charge.

    Angus has the right idea – charging the followers – although I don’t agree on the analysis. Still using Robert as the super-user, he should not be charged because of his tweets, but for the number of people he is following. Each tweet sent by the friends Robert is following will be copied on his queue (well the tweet ID) and the size and freshness of this queue (visually the ‘With Others’) can be used as the factor to charge.

  19. Another aspect of the problem is that so many people use twitter in near-constant streams throughout multi-hour events so as to give “live” coverage of an event or just their stream of thought (case in point: the owner of this blog). I’m certain that taking all those SMS’s in, displaying them, and broadcasting them out to webpages, RSS feeds, and then more phones is taxing to say the least. Look at the times it goes down and I’m sure you’ll see a major correlation with tech events. I understand that some have found this useful, but I really don’t and it’s a sure fire way to get me to unfollow you, and I can guarantee why Twitter’s user base hasn’t diversified beyond “social techies.”

    Solution? Create a new service on top of twitter for twitter-streams, because obviously people don’t get the idea behind 120-character limits (by the way, SMS has a 160-character limit) and hold their tweetstorms in a buffer to digest and spew out to followers when the server load can handle it.

  20. Another aspect of the problem is that so many people use twitter in near-constant streams throughout multi-hour events so as to give “live” coverage of an event or just their stream of thought (case in point: the owner of this blog). I’m certain that taking all those SMS’s in, displaying them, and broadcasting them out to webpages, RSS feeds, and then more phones is taxing to say the least. Look at the times it goes down and I’m sure you’ll see a major correlation with tech events. I understand that some have found this useful, but I really don’t and it’s a sure fire way to get me to unfollow you, and I can guarantee why Twitter’s user base hasn’t diversified beyond “social techies.”

    Solution? Create a new service on top of twitter for twitter-streams, because obviously people don’t get the idea behind 120-character limits (by the way, SMS has a 160-character limit) and hold their tweetstorms in a buffer to digest and spew out to followers when the server load can handle it.

  21. “Sweet how you never had to work with an Exchange server which did exactly that, and then added ‘All’ as a recipient to the address book of every user.

    Exchange never ever stored a message per user. If all users are on the same Exchange server and sent a message from someone on the same Exchange server, it it only stored ONCE. That’s been the case since Exchange 4.0. Bedlam had more to do with people hitting Reply All to an alias that had users on different servers. It was the message queue that caused the primary problems during Bedlam.

    In Exchange 2007 there is a deemphasis on SIS–it only applies to attachments. Not sure what the scaling problems are with Twitter as I have no idea how the system is designed. But, it would be safe to figure that whether or not they use SIS is not the source of their instability.

    Now, back to your regularly scheduled debate about an non-scalable, useless communication tool.

  22. “Sweet how you never had to work with an Exchange server which did exactly that, and then added ‘All’ as a recipient to the address book of every user.

    Exchange never ever stored a message per user. If all users are on the same Exchange server and sent a message from someone on the same Exchange server, it it only stored ONCE. That’s been the case since Exchange 4.0. Bedlam had more to do with people hitting Reply All to an alias that had users on different servers. It was the message queue that caused the primary problems during Bedlam.

    In Exchange 2007 there is a deemphasis on SIS–it only applies to attachments. Not sure what the scaling problems are with Twitter as I have no idea how the system is designed. But, it would be safe to figure that whether or not they use SIS is not the source of their instability.

    Now, back to your regularly scheduled debate about an non-scalable, useless communication tool.

  23. Om’s gotten it totally wrong. Twitter should charge your followers (past perhaps the first 100) for the privilege of getting access to your opinions faster. The website could remain free, but most twitterers I know use desktop clients to keep abreast, and so following someone is the only way to do that.

    Charging 10 cents per month to each follower after the first would reap a far greater income for them, and annoy interesting twitterers less!

    See my blog post for more:
    http://falkayn.blogspot.com/2008/05/oms-got-wrong-business-model-for.html

  24. Om’s gotten it totally wrong. Twitter should charge your followers (past perhaps the first 100) for the privilege of getting access to your opinions faster. The website could remain free, but most twitterers I know use desktop clients to keep abreast, and so following someone is the only way to do that.

    Charging 10 cents per month to each follower after the first would reap a far greater income for them, and annoy interesting twitterers less!

    See my blog post for more:
    http://falkayn.blogspot.com/2008/05/oms-got-wrong-business-model-for.html

  25. We used to have this thing in the ’90s for people who wanted to broadcast to a large audience — it was called “television”. And we had this other thing you could use to interact with the broadcaster, called “the U.S. Postal Service”. The fun part was when a guy like Dave Letterman would read your letter right there on the air, when he was broadcasting.

  26. We used to have this thing in the ’90s for people who wanted to broadcast to a large audience — it was called “television”. And we had this other thing you could use to interact with the broadcaster, called “the U.S. Postal Service”. The fun part was when a guy like Dave Letterman would read your letter right there on the air, when he was broadcasting.

  27. The whole debate is stupid! Why does no one seem to get it?

    Could a team of competent software engineers build a system which could handle this many users? Yes!
    Should twitter have a system which can handle this many users? Yes!

    I don’t understand why people are so keen to defend poor service. If it’s broke, then the twitter guys should fix it. That means better code, more servers, what ever it takes.

    If the problem is that they can’t find a way to monetize it, then that’s a different problem, but one where having lots of users should help, not hinder.

  28. The whole debate is stupid! Why does no one seem to get it?

    Could a team of competent software engineers build a system which could handle this many users? Yes!
    Should twitter have a system which can handle this many users? Yes!

    I don’t understand why people are so keen to defend poor service. If it’s broke, then the twitter guys should fix it. That means better code, more servers, what ever it takes.

    If the problem is that they can’t find a way to monetize it, then that’s a different problem, but one where having lots of users should help, not hinder.

Comments are closed.