The noise reduction system

David Risley this morning wrote about all the noise in all these systems like Twitter and FriendFeed. Of course that kicked off a whole discussion over on FriendFeed.

Oh, the glorious noise! Everyone loves beating me up for causing the noise. No, I am not the cause. I pass it along. You should see my inbound streams. Every second or two a new Twitter is aimed at me. Every few seconds, a new blog post comes into Google Reader. Every few seconds, a new thing on FriendFeed.

24 hours a day of noise. And we’re not even counting the professional noise over on TechMeme and Google News.

Buried by noise.

So, how do we get out?

Well, we have a couple of choices.

1. We can choose to remain ignorant. Billions of people choose this route every day. Pop open a beer and pretend nothing interesting is happening in the world. That explains why American Media would rather talk about Britney Spears than about anything really important (like what Barack Obama’s new policies are).

2. We can try to swim in all the noise and soak it in. That’s what I do, but only a small number of people are going to have time or willingness to do that.

3. We can build noise reduction systems. Techmeme is one such system. It shows you only what the bloggers think is important. Google News is another. That shows you only what professional journalists think is important (or, at least their algorithms are designed to show you that and, while the algorithms don’t always match real-world behavior, they do get close enough to have high value).

4. We can use search to only present high value items. For instance, let’s say you work for my sponsor, Seagate, wouldn’t you be very interested in only items that mention Seagate? Like this search on TweetScan? Yes, you would. There’s still SOME noise there, but a lot less for someone interested in stuff about Seagate than there is coming through, say, TwitterVision, which shows a random selection of all Tweets being posted in the last few minutes.

The problem? Twitter and FriendFeed have brought new noise into our lives (at least for the early adopter types) and there aren’t good ways to reduce the noise.

But FriendFeed shows us a way out. How about seeing only posts that have at least two “likes?” Isn’t that a way to reduce the noise? Yes! In fact, my eyes are already doing that. I scan the page of FriendFeed looking for things that stick out of the noise and I’ve noticed that items with lots of votes and lots of comments stand out.

Tonight I’ll be attending a FriendFeed party and I’ll ask them just what their plans are in terms of giving us new views into their streams of info: one with noise, one with noise removed. Yes, of course I’ll post videos to my Qik feed and they get forwarded to my FriendFeed account too (which shows up on my blog’s sidebar too). More noise ahead! :-)

What kinds of noise reduction systems are you seeing? What kinds do we need?

Oh, and here’s a FriendFeed search for all items that include the word “noise” in them. That’s one reason I wrote this post. The noise has our attention and we need to damp it back down.

UPDATE: In just half an hour we’ve gotten tons of more comments on this blog post over on FriendFeed.

UPDATE 2: another way to remove noise is to just watch the things I’m commenting on or liking. That ensures that my noise isn’t there, and that I’ve hand filtered the noise for you. Another way? Don’t subscribe to many people, just to people you know will provide you interesting stuff and little noise.

Comments

  1. Robert, have you tried out using Summize for Twitter search? I’m incredibly impressed with the way that it structures “conversations” on Twitter, and the way that it allows you to do some pretty advanced searches.

    I’m finding that Summize, plus turning off all Twitters in FriendFeed (except ones with comments/likes attached) and Google Reader is letting me manage info-inputs much more efficiently.

  2. Robert, have you tried out using Summize for Twitter search? I’m incredibly impressed with the way that it structures “conversations” on Twitter, and the way that it allows you to do some pretty advanced searches.

    I’m finding that Summize, plus turning off all Twitters in FriendFeed (except ones with comments/likes attached) and Google Reader is letting me manage info-inputs much more efficiently.

  3. It would be nice if I had some kind of engine on my machine or attached to my FriendFeed account that tracked what post I actually clicked through too read. Then it could index the words (and any other tags as well as the author) on that post. As I went along this engine would soon be able to predict which posts or people I show the most interest in and score them. It would also be able to show me similar posts and even recommend posts I might want to see. Google kind of does the recommendation part.

  4. It would be nice if I had some kind of engine on my machine or attached to my FriendFeed account that tracked what post I actually clicked through too read. Then it could index the words (and any other tags as well as the author) on that post. As I went along this engine would soon be able to predict which posts or people I show the most interest in and score them. It would also be able to show me similar posts and even recommend posts I might want to see. Google kind of does the recommendation part.

  5. so.. One remains ignorant if they don’t participate in Twitter or friendfeed for their information? Frankly I’ve not found any discussion on either that rises above a britney spears story. The level of discourse is the same, only the topic is different

  6. so.. One remains ignorant if they don’t participate in Twitter or friendfeed for their information? Frankly I’ve not found any discussion on either that rises above a britney spears story. The level of discourse is the same, only the topic is different

  7. Sorry for not posting this response in FriendFeed :)

    I quit doing item 2 when I started following over 200 people in Twitter. Now my approach is “dip my toe in the water every once in the while, and perform vanity searches and interesting item searches every once in a while.”

    One choice that you didn’t mention is being selective in whom you follow. Presumaly if you’re following less than 100 people, the noise remains at a manageable level. I wouldn’t characterize this approach as intentional ignorance, but more as intentional prioritization. After all, there are hundreds of thousands of people in Twitter that you DON’T follow, so even you are being selective.

  8. Sorry for not posting this response in FriendFeed :)

    I quit doing item 2 when I started following over 200 people in Twitter. Now my approach is “dip my toe in the water every once in the while, and perform vanity searches and interesting item searches every once in a while.”

    One choice that you didn’t mention is being selective in whom you follow. Presumaly if you’re following less than 100 people, the noise remains at a manageable level. I wouldn’t characterize this approach as intentional ignorance, but more as intentional prioritization. After all, there are hundreds of thousands of people in Twitter that you DON’T follow, so even you are being selective.

  9. Forgot to say – filtering by minimum number of likes or comments does sound like an interesting idea, if they can figure out an intuitive way to implement it in the UI, and a way to turn it on and off on the fly. (My point of comparison is finance.yahoo.com message boards, which allows you to filter based upon star ratings – a valuable tool, based upon the number of questionable posts in Yahoo! Finance message boards.)

  10. Forgot to say – filtering by minimum number of likes or comments does sound like an interesting idea, if they can figure out an intuitive way to implement it in the UI, and a way to turn it on and off on the fly. (My point of comparison is finance.yahoo.com message boards, which allows you to filter based upon star ratings – a valuable tool, based upon the number of questionable posts in Yahoo! Finance message boards.)

  11. Ontario: you are defined by who you follow. I think that’s a very logical approach! Actually you only need to follow a small percentage of people to learn pretty much everything important going through one of these systems.

  12. Ontario: you are defined by who you follow. I think that’s a very logical approach! Actually you only need to follow a small percentage of people to learn pretty much everything important going through one of these systems.

  13. I think ‘information overload’ is a huge emerging problem (especially for those of us in IT), and I personally experience the problem more from blogs/feeds than I do with twitter (probably because I subscribe to about 100 feeds and I follow about 15 people on Twitter). The problem is that I frequently come across feeds I’d like to subscribe to, but I already get more posts coming in than I have time to read, and I don’t want to unsubscribe from anything. I guess you could say I want to have my cake and eat it too, but I think technology can largely solve this problem. All I would really need is a way to rank the feeds I subscribe to in terms of their importance to me, in general, and then have something like Google reader show me unread posts in order of feed importance and then date, instead of just by date. This would mean I could just always read whatever is at the top of my list at Google Reader and not worry about how many unread posts are left when I run out of time – # of unread posts would become irrelevant because the technology is helping me balance my priorities. This type of seemingly small change could be revolutionary in the way we take in information, IMO.

    As more and more people start blogging, and more and more people get on Twitter (meanwhile the pace of change in the software world continues to hasten), this is going to become more and more of a problem. In the world of software developers, I fear that we will see an increasing dichotomy between the “alpha geeks” who spend just about every waking moment reading feeds/twitter and the “5:01 developers” who just don’t bother to read anything. We need a tool for those of who would like to stay in touch with what’s going on without dedicating their *ENTIRE* life to it, and I tend to think more and more people will be in this camp as the cost of going to either extreme grows. I think whatever site can help us intelligently filter information in real time will be the next Web 2.0 mega success story.

  14. I think ‘information overload’ is a huge emerging problem (especially for those of us in IT), and I personally experience the problem more from blogs/feeds than I do with twitter (probably because I subscribe to about 100 feeds and I follow about 15 people on Twitter). The problem is that I frequently come across feeds I’d like to subscribe to, but I already get more posts coming in than I have time to read, and I don’t want to unsubscribe from anything. I guess you could say I want to have my cake and eat it too, but I think technology can largely solve this problem. All I would really need is a way to rank the feeds I subscribe to in terms of their importance to me, in general, and then have something like Google reader show me unread posts in order of feed importance and then date, instead of just by date. This would mean I could just always read whatever is at the top of my list at Google Reader and not worry about how many unread posts are left when I run out of time – # of unread posts would become irrelevant because the technology is helping me balance my priorities. This type of seemingly small change could be revolutionary in the way we take in information, IMO.

    As more and more people start blogging, and more and more people get on Twitter (meanwhile the pace of change in the software world continues to hasten), this is going to become more and more of a problem. In the world of software developers, I fear that we will see an increasing dichotomy between the “alpha geeks” who spend just about every waking moment reading feeds/twitter and the “5:01 developers” who just don’t bother to read anything. We need a tool for those of who would like to stay in touch with what’s going on without dedicating their *ENTIRE* life to it, and I tend to think more and more people will be in this camp as the cost of going to either extreme grows. I think whatever site can help us intelligently filter information in real time will be the next Web 2.0 mega success story.

  15. More intelligent (think visual) ways of viewing and sorting through information and eliminating duplicates is the direction, imo.

    Duplicates alone are 20% of my noise in RSS, and it’s a lot higher than that on FriendFeed.

    It’s an important problem for sure.

  16. More intelligent (think visual) ways of viewing and sorting through information and eliminating duplicates is the direction, imo.

    Duplicates alone are 20% of my noise in RSS, and it’s a lot higher than that on FriendFeed.

    It’s an important problem for sure.

  17. What I love about Twitter is the 140 character limit. It forces the poster to be concise about what they need to say. Of course, the poster can 1) go nuts and spread a single thought across many tweets or 2) have nothing interesting to say in those 140 characters. But the solution to that noise problem is one “remove” click away.

    I am interested in how microfeeds on Twitter might help the RSS crush that we all get, as a lighter weight approach to syndication. I just blogged about my SaaS microfeed last night, if you are interested to understand this approach:
    http://dev2dev.bea.com/blog/plaird/archive/2008/05/twitter_microfe.html

    Jury is still out on whether this is an effective channel, but giving it a shot.

  18. What I love about Twitter is the 140 character limit. It forces the poster to be concise about what they need to say. Of course, the poster can 1) go nuts and spread a single thought across many tweets or 2) have nothing interesting to say in those 140 characters. But the solution to that noise problem is one “remove” click away.

    I am interested in how microfeeds on Twitter might help the RSS crush that we all get, as a lighter weight approach to syndication. I just blogged about my SaaS microfeed last night, if you are interested to understand this approach:
    http://dev2dev.bea.com/blog/plaird/archive/2008/05/twitter_microfe.html

    Jury is still out on whether this is an effective channel, but giving it a shot.

  19. I guess it does say a lot about me. It says I prefer more depth and thought in a conversation than be elicited with 140 characters. Moreever, looking at your link I see and overabundance of babble about the pros/cons of Twitter, FriendFeed. High level election coverage that is comparable to what one gets from local TV news. Random bits if mildly interesting stories that one would see on a slow news day on CNN, MSNBC,FOX. I see nothing there they makes one more enlightened, or makes me say: “Hmmm..never would have thought of that”. So, if avoiding Twitter and FriendFeed leads to one being ignorant of what is going on in the world, I’ll happily go all in on that bet and call your bluff.

  20. I guess it does say a lot about me. It says I prefer more depth and thought in a conversation than be elicited with 140 characters. Moreever, looking at your link I see and overabundance of babble about the pros/cons of Twitter, FriendFeed. High level election coverage that is comparable to what one gets from local TV news. Random bits if mildly interesting stories that one would see on a slow news day on CNN, MSNBC,FOX. I see nothing there they makes one more enlightened, or makes me say: “Hmmm..never would have thought of that”. So, if avoiding Twitter and FriendFeed leads to one being ignorant of what is going on in the world, I’ll happily go all in on that bet and call your bluff.

  21. always has been sensory overload… we dim it down, a yogi does not…. because the yogi knows how to admit everything and alos have the faith and trust that what he needs to know will show up in the attention stream, and the rest is just the beautiful river

    meditate more, and all this flow is no problem at all

    enjoy, gregory

  22. always has been sensory overload… we dim it down, a yogi does not…. because the yogi knows how to admit everything and alos have the faith and trust that what he needs to know will show up in the attention stream, and the rest is just the beautiful river

    meditate more, and all this flow is no problem at all

    enjoy, gregory

  23. 1. Missing a day’s fill of blogger social-networkingese gossip, hardly makes you ignorant, some people actually have things to do, videos to edit, and stuff to accomplish.

    2. Swim in the noise? Perfect for people with 30 second attention spans, Beth Goza mode on. You know what I LOVE about Arts and Letters Daily? You have to reeeeead, as this RSSing, headline-skimming and 140 characters, is the very definition of ignorant, at best you miss the context wholesale, at worst you appear incompetent. Listening to everything at once, crowds out the real music.

    3. Hyperfocused narrowcasting is not a “noise reduction system”, it’s just and extreme excess, clamoring endlessly on one topic, drum-beating to one’s own song. Techmeme itself is noise, just highly-linked noise. Gawd, another three-letter buzzword, NRS.

    4. Geee whiz, you mean in 2008, we can categorize and search for things? Wow, high concept idea, there. Poor Melvil Dewey, in 1876, he didn’t even have a computer.

    PS – The PROBLEM with ‘searching’ for ‘items’ in stuff like Friendfeed, is that everything is one big long (endless and pointless) conversation, so to get the context, you really have to read most of it (not to mention the back and forth on several dimensional levels). But of course, you can embrace the noise, picking out bits and pieces, missing the context, landing you right back where you started.

  24. 1. Missing a day’s fill of blogger social-networkingese gossip, hardly makes you ignorant, some people actually have things to do, videos to edit, and stuff to accomplish.

    2. Swim in the noise? Perfect for people with 30 second attention spans, Beth Goza mode on. You know what I LOVE about Arts and Letters Daily? You have to reeeeead, as this RSSing, headline-skimming and 140 characters, is the very definition of ignorant, at best you miss the context wholesale, at worst you appear incompetent. Listening to everything at once, crowds out the real music.

    3. Hyperfocused narrowcasting is not a “noise reduction system”, it’s just and extreme excess, clamoring endlessly on one topic, drum-beating to one’s own song. Techmeme itself is noise, just highly-linked noise. Gawd, another three-letter buzzword, NRS.

    4. Geee whiz, you mean in 2008, we can categorize and search for things? Wow, high concept idea, there. Poor Melvil Dewey, in 1876, he didn’t even have a computer.

    PS – The PROBLEM with ‘searching’ for ‘items’ in stuff like Friendfeed, is that everything is one big long (endless and pointless) conversation, so to get the context, you really have to read most of it (not to mention the back and forth on several dimensional levels). But of course, you can embrace the noise, picking out bits and pieces, missing the context, landing you right back where you started.

  25. I pick my friends very selectively to limit noise. I really think Twitter has a lot of room to grow in terms of making some great noise-reducing tools.

    (let you up or downmod friends comments and use that data to make more ‘favored’ Twitterers’ posts show up more visible for instance.)

    Not attempting any self-promotion here, but since you asked for suggestions, I did blog about it more in-depth the other day actually.

    http://borderlinetheory.com/?p=126

  26. I pick my friends very selectively to limit noise. I really think Twitter has a lot of room to grow in terms of making some great noise-reducing tools.

    (let you up or downmod friends comments and use that data to make more ‘favored’ Twitterers’ posts show up more visible for instance.)

    Not attempting any self-promotion here, but since you asked for suggestions, I did blog about it more in-depth the other day actually.

    http://borderlinetheory.com/?p=126

  27. Robert, I never said, nor implied that I wanted sensationalism. How you inferred that is baffling. You are putting words in my mouth. Rather than rely on “the wisdom of crowds” (which most often appeals to the lowest common denominator), I get my information from a myriad of sources. None of which is gossip or aggregated. Nor is it relying on someone to filter blogs to “give me the best stuff so I don’t have to” I simply said the level of discourse I have seen on FriendFeed or Twitter is hardly enlightening and certainly does not reduce one’s level of ignorance. Rarely does a conversation deviate too far from the value of FriendFeed. If it’s not that, it’s amateur opinions on what is wrong with Microsoft, banal discussions about the election. If I start to see deeply engaged discussions about the situation in Darfur, solutions for peace in the Middle East, the worsening situation in Burma, and other more serious topics, I’ll take my chances at remaining ignorant.

  28. Robert, I never said, nor implied that I wanted sensationalism. How you inferred that is baffling. You are putting words in my mouth. Rather than rely on “the wisdom of crowds” (which most often appeals to the lowest common denominator), I get my information from a myriad of sources. None of which is gossip or aggregated. Nor is it relying on someone to filter blogs to “give me the best stuff so I don’t have to” I simply said the level of discourse I have seen on FriendFeed or Twitter is hardly enlightening and certainly does not reduce one’s level of ignorance. Rarely does a conversation deviate too far from the value of FriendFeed. If it’s not that, it’s amateur opinions on what is wrong with Microsoft, banal discussions about the election. If I start to see deeply engaged discussions about the situation in Darfur, solutions for peace in the Middle East, the worsening situation in Burma, and other more serious topics, I’ll take my chances at remaining ignorant.

  29. This is a great set of comments and I agree – there is just TOO MUCH information or ‘noise’ out there.

    That’s why I elect to be selective; and use just a few 2.0 noise-makers!

    Twitter is something I use as a recource, and it’s a great benefit to me as the guy behind; “The Ideas Blog.” I am always looking for new deas and Twitter’s a pretty good way to find them.

    BTW: For all the Scoble haters out there – I regularly chat with Robert on Twitter and (unlike some) the guy always replies with something useful, funny or slightly insane!

    In my honest opinion, the ‘trick’ is to be selective and not just join everything.

    Jim Connolly
    http://theideasblog.com

  30. This is a great set of comments and I agree – there is just TOO MUCH information or ‘noise’ out there.

    That’s why I elect to be selective; and use just a few 2.0 noise-makers!

    Twitter is something I use as a recource, and it’s a great benefit to me as the guy behind; “The Ideas Blog.” I am always looking for new deas and Twitter’s a pretty good way to find them.

    BTW: For all the Scoble haters out there – I regularly chat with Robert on Twitter and (unlike some) the guy always replies with something useful, funny or slightly insane!

    In my honest opinion, the ‘trick’ is to be selective and not just join everything.

    Jim Connolly
    http://theideasblog.com

  31. Maybe you should try our private beta system as a “noise reduction systems”.

    It’s a personal content portal where you pick the topics you’re interested in and the system works for you and brings you only the recent popular content on the web in any topics you’re interested in.

  32. Maybe you should try our private beta system as a “noise reduction systems”.

    It’s a personal content portal where you pick the topics you’re interested in and the system works for you and brings you only the recent popular content on the web in any topics you’re interested in.

  33. Twenty five years ago we didn’t have a lot of noise. For news we had ABC, CBS and NBC and papers for the most part.

    With Information Technology we had PC Week and a few other trade rags.

    Now we have so much noise in every aspect of life that we have to adopt media trustees.

    Right now for general news Brian Williams, NBC, is my general news media trustee, Entertainment Tonight for my “star” media trustee, Crigley, Scoble, Techmeme, Slashdot to name a few for my IT media trustees.

    Robert, unlike you I don’t have time to “read” 8000 items on Google reader. Therefore, I look for these trustees to filter the noise. Just because you are a trustee doesn’t mean I don’t think for myself. A trustee is only the gate keeper. It’s up to me to determine the quality and accuracy of the information.

  34. Twenty five years ago we didn’t have a lot of noise. For news we had ABC, CBS and NBC and papers for the most part.

    With Information Technology we had PC Week and a few other trade rags.

    Now we have so much noise in every aspect of life that we have to adopt media trustees.

    Right now for general news Brian Williams, NBC, is my general news media trustee, Entertainment Tonight for my “star” media trustee, Crigley, Scoble, Techmeme, Slashdot to name a few for my IT media trustees.

    Robert, unlike you I don’t have time to “read” 8000 items on Google reader. Therefore, I look for these trustees to filter the noise. Just because you are a trustee doesn’t mean I don’t think for myself. A trustee is only the gate keeper. It’s up to me to determine the quality and accuracy of the information.

  35. As I get older and evolve through this zany web wasteland, I’m finding that I’m turning back to the short bite-sized clips of data, not 40 minute ones. Regardless of what side of the Big or small media landscape I fall into, I’m still the original MTV generation.

    This is why I’m watching more CNET and less Fast Company TV. This is also why fanboys are appropriate… if they make enough noise, I can skim the buzz around stuff I don’t even follow.

    CNET can summarize, fanboys can cheerlead and the social scene can provide gossip (if I’m in the mood for fast-food grade intelligence).

    Easiest way to consume for me is by skimming behaviors, not content. If we learn what to expect from various sources, skimming is super-fast.

  36. As I get older and evolve through this zany web wasteland, I’m finding that I’m turning back to the short bite-sized clips of data, not 40 minute ones. Regardless of what side of the Big or small media landscape I fall into, I’m still the original MTV generation.

    This is why I’m watching more CNET and less Fast Company TV. This is also why fanboys are appropriate… if they make enough noise, I can skim the buzz around stuff I don’t even follow.

    CNET can summarize, fanboys can cheerlead and the social scene can provide gossip (if I’m in the mood for fast-food grade intelligence).

    Easiest way to consume for me is by skimming behaviors, not content. If we learn what to expect from various sources, skimming is super-fast.

  37. Robert – I live somewhere in between two and three. Like you, I love the noise (I personally find it very useful). But I do wish there was a better filter. I am starting to use FriendFeed and Social Thing more and more but haven’t figured out the best way to optimize my usage. Maybe we can do a follow up podcast to discuss these tools! ;)

    I look forward to reading more and giving FF/ST add’l attention over the coming weeks.

    Best,
    Aaron | @astrout

  38. Robert – I live somewhere in between two and three. Like you, I love the noise (I personally find it very useful). But I do wish there was a better filter. I am starting to use FriendFeed and Social Thing more and more but haven’t figured out the best way to optimize my usage. Maybe we can do a follow up podcast to discuss these tools! ;)

    I look forward to reading more and giving FF/ST add’l attention over the coming weeks.

    Best,
    Aaron | @astrout

  39. Until tools like Twitter and FriendFeed install features that let you prioritize followers, you can use feeds and Google Reader to help filter the glut of info.

  40. Until tools like Twitter and FriendFeed install features that let you prioritize followers, you can use feeds and Google Reader to help filter the glut of info.

  41. Two of your approaches which filters based on what either the most prominant bloggers say or by votes means that my prioritised reading gets scewed by other people rather than what I consider of most interest. This could be relatively easily addressed through key word analysis of the twitters or blog entries that I read as opposed to just see in my RSS feed. So if read blog entries which mentioned ‘scoble’ then other blog entries about you or referencing your articles would get bumped up my reading list.

    This is something that the likes of google could probably provide today given that key word analysis is all part of ad placement in search engines.

  42. Two of your approaches which filters based on what either the most prominant bloggers say or by votes means that my prioritised reading gets scewed by other people rather than what I consider of most interest. This could be relatively easily addressed through key word analysis of the twitters or blog entries that I read as opposed to just see in my RSS feed. So if read blog entries which mentioned ‘scoble’ then other blog entries about you or referencing your articles would get bumped up my reading list.

    This is something that the likes of google could probably provide today given that key word analysis is all part of ad placement in search engines.