In TechMeme for only 50 seconds (Update: or more)

December 21, 2006 By

Hey, I’m noticing that some things get into TechMeme for only a minute or so and then disappear. Hmmm.

I bet Gabe is watching for evidence of gaming and then removing anything that doesn’t belong.

The item in question, Photowalking, appeared for a few minutes (you can see it on the TechMeme River, but not on the main site) but then quickly disappeared.

Personally I would have removed it too. Thomas Hawk and I are friends. It got onto TechMeme cause both Thomas and me linked to it. We weren’t trying to get it on TechMeme, just both organically linked to it (we’re both involved).

Anyway, I’ve seen other evidence that a human is changing the results behind the scenes too. We’ve started talking about this at parties and, invariably, we think it’s good that Gabe’s doing that. Why? He’s good at picking what’s really interesting and what’s not.

Gabe, for his part, says that it’s his algorithms that decide whether something is worth staying on the page.

I wonder what the truth is: I’d love to sit with my camera and watch Gabe all day long and see what the system looks like to him. I bet he’s learning a lot about the linking patterns, and other patterns that are going on in the blogosphere.

By the way, it’s fun to compare the TechMeme river to my link blog. I notice that Gabe usually beats me with his algorithms, but that I can still beat him one out of every few stories. Why? Cause I am just looking at whether the post is interesting to me, not whether or not it has some inbound links. Funny enough usually within an hour the best of my “finds” are also on TechMeme cause they earn enough links from other people to get onto that page.

UPDATE: Now it’s back, that’s weird.

UPDATE 2: In related actions Digg is removing entire domains from its service.

UPDATE 3: It’s gone again.

UPDATE 4: Gabe says he doesn’t mess with the site in my comments and is on vacation right now with his family (but, says, that we should expect more of this kind of behavior).

Filed Under: 1
  • http://dotnetjunkies.com/WebLog/paul/default.aspxhttp://dotnetjunkies.com/WebLog/paul/default.aspx paul

    Yesterday it was about Blockbuster, today it’s Saturday Night Live – Entertainmeme

  • http://martinbreton.com/ brem

    When a single individual starts to decide for the masses, isn’t that what we call an autocrat? :D

  • http://dotnetjunkies.com/WebLog/paul/default.aspxhttp://dotnetjunkies.com/WebLog/paul/default.aspx paul

    Yesterday it was about Blockbuster, today it’s Saturday Night Live – Entertainmeme

  • http://martinbreton.com brem

    When a single individual starts to decide for the masses, isn’t that what we call an autocrat? :D

  • http://scobleizer.wordpress.com/ Robert Scoble

    Weird, the item just showed up again. Hi Gabe! :-)

  • http://scobleizer.wordpress.com/ Robert Scoble

    Weird, the item just showed up again. Hi Gabe! :-)

  • http://www.techmeme.com/ Gabe

    Hi Robert, just got back from dinner (on pseudo-vacation with friends and family on the East coast).

    Nope didn’t touch this one. Funny too, because I don’t see that post up presently, meaning it arrived, disappeared, arrived, and disappeared again.

    Expect to see more and more of this, especially for posts kind of at the margin of postability. Almost every week I introduce some new kind of rule or heuristic that adds humanlike elements of editorial judgement to headline selection, and the result is increasingly complicated behavior.

    BTW, several times a week political bloggers send me emails accusing me of intervening with their placement on memeorandum (my political site). What’s funny about this is I don’t even read what’s on that site. Probably give it a 10 second scan every day just to check for major problems.

    [I have more to say...just want to save this comment for now.]

  • http://www.techmeme.com/ Gabe

    Hi Robert, just got back from dinner (on pseudo-vacation with friends and family on the East coast).

    Nope didn’t touch this one. Funny too, because I don’t see that post up presently, meaning it arrived, disappeared, arrived, and disappeared again.

    Expect to see more and more of this, especially for posts kind of at the margin of postability. Almost every week I introduce some new kind of rule or heuristic that adds humanlike elements of editorial judgement to headline selection, and the result is increasingly complicated behavior.

    BTW, several times a week political bloggers send me emails accusing me of intervening with their placement on memeorandum (my political site). What’s funny about this is I don’t even read what’s on that site. Probably give it a 10 second scan every day just to check for major problems.

    [I have more to say...just want to save this comment for now.]

  • http://scobleizer.wordpress.com/ Robert Scoble

    Thanks Gabe! You just gave me a good reason to watch the TechMeme River to catch those “smaller stories” that probably got on the page for a few minutes and then rolled off. http://www.techmeme.com/river

  • http://scobleizer.wordpress.com/ Robert Scoble

    Thanks Gabe! You just gave me a good reason to watch the TechMeme River to catch those “smaller stories” that probably got on the page for a few minutes and then rolled off. http://www.techmeme.com/river

  • http://www.techmeme.com/ Gabe

    [Ok, promised more above. I have lots to say in fact. Will maybe leave a third comment.]

    You said something very interesting: “I’ve seen other evidence that a human is changing the results behind the scenes too”. Is an individual human changing the results? Absolutely. Individual humans such as yourself, and various other humans on the tech scene.

    Lemme explain: I am not sworn to showing popular items. Techmeme is not designed to “track memes”. It’s designed to use various gestures, big and small, to pick interesting news items. Sometimes the action of a single blogger such as yourself will cause something to appear, depending on other circumstances. Other times many citations are required for something to appear.

    Something even more interesting: sometimes new links can contribute to a post NOT appearing. In fact Robert, I recall an short private email thread where we discussed this. Perhaps that effect was a factor affecting your photowalk post. Perhaps not (haven’t checked).

  • http://www.techmeme.com/ Gabe

    [Ok, promised more above. I have lots to say in fact. Will maybe leave a third comment.]

    You said something very interesting: “I’ve seen other evidence that a human is changing the results behind the scenes too”. Is an individual human changing the results? Absolutely. Individual humans such as yourself, and various other humans on the tech scene.

    Lemme explain: I am not sworn to showing popular items. Techmeme is not designed to “track memes”. It’s designed to use various gestures, big and small, to pick interesting news items. Sometimes the action of a single blogger such as yourself will cause something to appear, depending on other circumstances. Other times many citations are required for something to appear.

    Something even more interesting: sometimes new links can contribute to a post NOT appearing. In fact Robert, I recall an short private email thread where we discussed this. Perhaps that effect was a factor affecting your photowalk post. Perhaps not (haven’t checked).

  • http://scobleizer.wordpress.com/ Robert Scoble

    Gabe: now I understand why Steve Gillmor doesn’t want me to link to him. :-)

  • http://scobleizer.wordpress.com/ Robert Scoble

    Gabe: now I understand why Steve Gillmor doesn’t want me to link to him. :-)

  • http://www.techmeme.com/ Gabe

    [Third installment. Maybe I'm done here. Rather happy to have written this up somewhere.]

    You said I claimed not to “mess” with the site in my first comment. Nope, I mess with it.

    Here’s an example: http://www.techmeme.com/061215/p17#a061215p17

    See that “Tribute to Lonelygirl15″ post? I removed it almost immediately after spotting it.

    Why? Its appearance was a fluke. It was basically a flaw in my system combined with an unusual type of redirection used by the longelgirl15 site that fooled my system into thinking people were discussing that post. They weren’t, so I felt best to zap it.

    Such examples are uncommon but they arise. Another: a couple times a month my scraper selects the wrong title for a given article url. Whether or not I fix the scraper, I may manually change the title if the erroneous title is awful enough.

    Another example, where I need to be deliberately vague (sorry): suppose I see evidence of excessive logrolling that degrades results, and therefore change a particular global system parameter that has an immediate impact on a headline currently live. Is that “messing”? I dunno. But stuff like that has happened once or twice in the past month.

    I guess to summarize everything: 1. Direct changes by me personally are rare, and normally to work around bugs. 2. Many, many individuals “mess” with Techmeme. 3. Please recognize my system is always getting smarter and will seem increasingly humanlike over time.

    Epilogue: interesting peripheral issue: Would Techmeme be better if editors under my direct control actively messed with the live site? Maybe…I’m undecided myself.

  • http://www.techmeme.com/ Gabe

    [Third installment. Maybe I'm done here. Rather happy to have written this up somewhere.]

    You said I claimed not to “mess” with the site in my first comment. Nope, I mess with it.

    Here’s an example: http://www.techmeme.com/061215/p17#a061215p17

    See that “Tribute to Lonelygirl15″ post? I removed it almost immediately after spotting it.

    Why? Its appearance was a fluke. It was basically a flaw in my system combined with an unusual type of redirection used by the longelgirl15 site that fooled my system into thinking people were discussing that post. They weren’t, so I felt best to zap it.

    Such examples are uncommon but they arise. Another: a couple times a month my scraper selects the wrong title for a given article url. Whether or not I fix the scraper, I may manually change the title if the erroneous title is awful enough.

    Another example, where I need to be deliberately vague (sorry): suppose I see evidence of excessive logrolling that degrades results, and therefore change a particular global system parameter that has an immediate impact on a headline currently live. Is that “messing”? I dunno. But stuff like that has happened once or twice in the past month.

    I guess to summarize everything: 1. Direct changes by me personally are rare, and normally to work around bugs. 2. Many, many individuals “mess” with Techmeme. 3. Please recognize my system is always getting smarter and will seem increasingly humanlike over time.

    Epilogue: interesting peripheral issue: Would Techmeme be better if editors under my direct control actively messed with the live site? Maybe…I’m undecided myself.

  • Pingback: One By One Media » Techmeme: A Technology River Guide

  • http://thomashawk.com/ Thomas Hawk

    Gabe, whatever you’re doing, keep doing it. TechMeme consistently serves up the best daily digest of tech news out there. It’s better than digg, it’s better than any of the tech news sites, it’s better than my RSS reader or any other aggregator. It’s super fast and consistently reliable.

    I like your addition of more photos (as you and I’ve discussed in the past) and certainly think your sponsored posts represent one of the smartest ways to approach advertising and monetization for your site.

    I don’t know if I could completely trust an algorithm. I’d probably tend to overtinker myself resulting in a worse product. Kristopher Tate and I have been talking a lot about this lately with Zooomr. We’ve been working with some search/rank stuff with regards to image search for a while now with him favoring an approach that is more algorithmic based and me favoring more of a blend where you actually bring in more than a few bodies to try to perfect at least the short head of image search.

    Anyways, keep up the great work.

  • http://thomashawk.com Thomas Hawk

    Gabe, whatever you’re doing, keep doing it. TechMeme consistently serves up the best daily digest of tech news out there. It’s better than digg, it’s better than any of the tech news sites, it’s better than my RSS reader or any other aggregator. It’s super fast and consistently reliable.

    I like your addition of more photos (as you and I’ve discussed in the past) and certainly think your sponsored posts represent one of the smartest ways to approach advertising and monetization for your site.

    I don’t know if I could completely trust an algorithm. I’d probably tend to overtinker myself resulting in a worse product. Kristopher Tate and I have been talking a lot about this lately with Zooomr. We’ve been working with some search/rank stuff with regards to image search for a while now with him favoring an approach that is more algorithmic based and me favoring more of a blend where you actually bring in more than a few bodies to try to perfect at least the short head of image search.

    Anyways, keep up the great work.

  • nivedita

    Thomas and I*

  • nivedita

    Thomas and I*

  • http://weblogs.macromedia.com/jd John Dowdell

    Robert wrote this paragraph, which gave me chills:

    “Anyway, I’ve seen other evidence that a human is changing the results behind the scenes too. We’ve started talking about this at parties and, invariably, we think it’s good that Gabe’s doing that. Why? He’s good at picking what’s really interesting and what’s not. Gabe, for his part, says that it’s his algorithms that decide whether something is worth staying on the page. I wonder what the truth is.”

    Think about this… someone with source evidence and a reputation for truthfulness talks about private matters, and the response is about “people I chat with at parties who have no direct knowledge of the situation”. You can get fooled plenty of ways if you go with the herd and don’t look at source evidence.

    Then look at one of the followup comments here: “Robert Scoble recently revealed to me that the popular technology news site is controlled on the back end by a human river guide.” Someone’s guesses are taken as fact by others!

    This is pretty seriously frightening stuff, these items of faith, this belief-based approach to the world…. :(

    (Gabe, are you keeping private chronological descriptions of the algorithms as they change? This may be useful in case some of the true-believers ever sue.)

  • http://weblogs.macromedia.com/jd John Dowdell

    Robert wrote this paragraph, which gave me chills:

    “Anyway, I’ve seen other evidence that a human is changing the results behind the scenes too. We’ve started talking about this at parties and, invariably, we think it’s good that Gabe’s doing that. Why? He’s good at picking what’s really interesting and what’s not. Gabe, for his part, says that it’s his algorithms that decide whether something is worth staying on the page. I wonder what the truth is.”

    Think about this… someone with source evidence and a reputation for truthfulness talks about private matters, and the response is about “people I chat with at parties who have no direct knowledge of the situation”. You can get fooled plenty of ways if you go with the herd and don’t look at source evidence.

    Then look at one of the followup comments here: “Robert Scoble recently revealed to me that the popular technology news site is controlled on the back end by a human river guide.” Someone’s guesses are taken as fact by others!

    This is pretty seriously frightening stuff, these items of faith, this belief-based approach to the world…. :(

    (Gabe, are you keeping private chronological descriptions of the algorithms as they change? This may be useful in case some of the true-believers ever sue.)

  • Pingback: Techmeme all algorithm or is there human involvement « Technically Speaking

  • http://rexdixon.wordpress.com/ rexdixon

    Yeah, I want to apologize to Gabe about my questioning also. Have a great vacation, and don’t worry – your algorithms have come to life of it’s own! Soon it will consume you too. hahah.. j/k!

    Rex

  • http://rexdixon.wordpress.com/ rexdixon

    Yeah, I want to apologize to Gabe about my questioning also. Have a great vacation, and don’t worry – your algorithms have come to life of it’s own! Soon it will consume you too. hahah.. j/k!

    Rex

  • http://www.techmeme.com/ Gabe

    Wow, thanks Thomas.

    John, not expecting it to come to that! Anyway, I’ve always cited gray areas as I have above. And I don’t advertise my sites as “fully automated” or anything like that because there’s no workable notion of “fully automated”. You can’t get a tech news vertical without manual configurations to ensure that.

    Rex, thanks. Actually, I described it as a “pseudo-vacation” above. I only work half of the time on pseudo-vacation.

  • http://www.techmeme.com/ Gabe

    Wow, thanks Thomas.

    John, not expecting it to come to that! Anyway, I’ve always cited gray areas as I have above. And I don’t advertise my sites as “fully automated” or anything like that because there’s no workable notion of “fully automated”. You can’t get a tech news vertical without manual configurations to ensure that.

    Rex, thanks. Actually, I described it as a “pseudo-vacation” above. I only work half of the time on pseudo-vacation.

  • http://www.onebyonemedia.com Jim “Genuine” Turner

    John I think you missed the point of my metaphor. I think the future of news sites will be similar to what we see with Gabe’s site. To his credit Gabe does a great job. He decides who the influential people are, and provides information based on that and I suppose a lot of other things I don’t pretend to understand.

    I think we should have more choices of these human types of news aggregators (river guides), and be able to decide for ourselves what is interesting news. Take me on the best river ride.

  • http://www.onebyonemedia.com Jim Turner

    John I think you missed the point of my metaphor. I think the future of news sites will be similar to what we see with Gabe’s site. To his credit Gabe does a great job. He decides who the influential people are, and provides information based on that and I suppose a lot of other things I don’t pretend to understand.

    I think we should have more choices of these human types of news aggregators (river guides), and be able to decide for ourselves what is interesting news. Take me on the best river ride.