Can Google please fix my search?

What google needs is some type of page ownership claim. Similar to claimid or technorati, but with google, you can control the pages that show in searches. Google gives you a code, you place it in your site, the claim is positive, and then you can say, "This is my site, but I don't use it anymore, instead, this is my main site now."

I know I am going to get slapped around by some google fanboys for saying that, or SEO freaks, but damn, I just want to be able to control what shows up and what doesn't, I don't care how high it shows.

Robert,

You are ignoring the real issue, Google is not the primary problem in this case, it's your cyberjunk!

Why isn't your old blog content integrated into this blog and the old blog shut down? It's because blogs are generally web 1.0 beta software and the creators are not focusing on what customers need and want to be good net citizens.

If we allowed littering everywhere in nature the way we currently do on the web, we'd all be living under giant piles of garbage! It's time for people to start picking up after themselves on the Internet where possible and for others to provide the tools to make that easy to accomplish.

I'll write more on this later today if I have time.

However, if anyone would seriously like to fund a more relevant search engine than Google, contact me and I'll put you in touch with the person with the business plan if the person is a qualified investor.

one easy answer ... no :)

Why not change the old page to do a 302 redirect? Google will follow these and update the link. My website is hardly close to the popularity yours is, but I am able to get google to update my links within a few days by doing this.

Then again it is possible you have no control over the old blog, and in that case you are going to have a tough time getting the links to update - until you can get more links on external pages pointing to the new site with similar keywords.

Steve,

Robert probably does not remember password to his old blog and now is blaming Google for this.

Hi Fanboys - the questions is *NOT* 'why google is not listing correctly' but 'why google is not listing correctly is .FR when it can do so in .com'.

It's not like Robert's old blog has a 'redirector' visible to google.fr

Hey blogger@wordpress - why the Fanboys comment? (I'm getting sick of that word.)

And actually the question in this post was “why google is not listing correctly”.

Google has always displayed slightly different results when you go to the different google country sites - I believe it places more emphasis on links/results from that country. Perhaps there are more people in the French "google world" that link to scobleizer.com/wordpress.com than in the global "google world".

@8 You are guessing and justifying google's actions here.

The question in the post linked here is about the difference between the fr and com results.

I am not a google hater. I feel this could simply be a mistake in google's algorithm or in result pruning

But you would rather justify things on behalf of google than to even think that this could have been a mistake. Why?

Whatever be the problem and your explanation on how Google technology works, the bottom line fact is that Google does find a 12 month old and stale link. This must be embarrasing for Google. And should be a proof that Google is falling behind in technology.

Search on a more advanced search engine, such as, live.com for the same query. You will get a relevant link on the top.

http://search.live.com/results.aspx?mkt=en-us&a...

Disclaimer: The commentator works for Microsoft and may have biases in the subjective part of the opinion. The objective part of the opinion can be verified publically.

Kamal,

It's interesting that http://www.msdewey.com and link you provided aren't in sync on these results.

Did you change the ranking this morning on live.com or does microsoft/msn/live.com/msdewey.com have the same syncronization problem that Google has?

Live.com still brings up the old blog in 3rd place, that's still a failure IMHO

Just doing a bit of testing and did a search for pizza and live.com comes up with http://www.kyzo.com/ as the number one entry? - kyzo sells computers

One wrong set of results will not prove that 'google is falling behind in technology'. May be its true. But cant conclude based on 'robert scoble' query alone.

Google search is ultimately 'software' and it is bound to make some mistakes. But whats strange is this leads to one of the 'extreme' stand
(a) Google can never make a mistake. So scoblizer hasnt done what he needs to
(b) THis marks the end of google.

Obvioulsy neither is true.

(BTW, it would be great to have a discussion on google/apple/play station without having to bring in Microsoft)

Phil: no, that's not a failure. Someone might be trying to find my old blog when they type "Scoble" into a search engine. It's just that my old one probably isn't the most relevant result.

I think it's a case of does newness always determine relevancy..

In this particular case, you're searching for "robert scoble" thus, BOTH links are highly relevant.

As to whether or not newness or age rank higher, this is still an ongoing question in search and it seems to vary by query.

Google has long held that an older site is more trusted than a newer site, while MSN has recently started giving a ranking boost to newer sites (as somebody who launches new sites often, I see them do really well in MSN, then poorly, then somewhere in the middle over time)

For a news query, new is better. But for a query like "buy cds" new isn't the best. You'd probably want an established merchant here.

This is a problem that can be fixed by classifying queries as informational, transactional or navigational.

For informational, new is usually better. For news new is always better. (searching for proposal 2 michigan still shows me last election's proposal, not this one's) For transactional, new isn't always better. For navigational, it can depend. I'd have to go with newer her too.

I just wanted to show how problems can occur when you assume anything based on age.

But if it is your old site that has all the linklove, what do you expect? Maybe you should have move your content and just provide a redirect?

Well - a 301 redirect (NOT a 302) would help - though I remeber Robert said that he for some reason can't do this.

> It’s just that my old one probably isn’t the most relevant result.

Robert, didn't you have your old site (http://scoble.weblogs.com/) for many more years than your current site? such that, there is much more material on yr old site and many more links pointing into yr former site from other sites when compared to your current site? so perhaps google is weighting the sheer volume of those links and the volume of yr content more heavily since they represent so much more of "scoble" on the web than does http://scobleizer.wordpress.com/? Btw the oldsite having much more content and having more links into it, isn't google making a reasonable decision in terms of saying where a majority of scoble's thoughts are represented on the web today?

Regarding the original complaint of Stéphane Rodriguez's at http://www.arstdesign.com/BBS/BulletinBoard.php....

Stéphane cites a google study that says returning 10 results per page is more desirable than returning 30. He then takes that and turns that into what i think is a completely different point than google as making, and says "I can only wonder whether anybody would trade speed for quality results. If Google spends twice more time but comes up with much better results, I am absolutely ready to take that because it means the stuff I am looking for are more likely to be not just in the first results page, but also among the first top 5 (excluding ads on the top of course)."

Perhaps i don't understand how google works, but i assume at least this:

- were google to return 30 results instead of 10 on one page, it would be the same results on one page as they now return on 3. So it's not a matter of any of the later results suddenly jumping in the ranking and becoming #1.

- returning 30 results instead of 10 on one page has nothing to do with "spending time" to come up with "better results". Google's point was that people didn't like the page loading time on 30. google wasn't saying they weren't spending the time they want to to perform a search.

It seems to me that Stéphane took a google comment about time to return html results on a single page and twisted it to be a more general statement about time to perform a search.

He also says:

"Let's debunk another myth about Google : quality and consistency of results."

a) i don't expect different search engines for different countries to return the same search results. I would certainly hope they do not!

b) I don't even expect that search engines in the same country will always return the exact same results. google's localization engine falls into this.

Once i drop these two expectations, i'm not sure what arguments are left to say that google does not present "quality and consistent results." i'm not saying they are "perfect" but what the hell does "perfect" mean anyway? it is the best search engine software i know of; that's my only criteria.

Yes, one search result does not say much. It is true that on some searches Google performs better on some Windows Live. The point to note is that the number of queries on the latter category is increasing.
Each new query in the latter category is a proof that Google is falling behind in technology.

One can explain why Google performed worse than Live on Robert Scoble, but the explanation does not change the bottom line fact. Relevancy of a search algorithm is mathematically described by the set of assumptions the algorithm makes.

Assumption which were true five years ago may not remain true anymore. For an example when Google was created, a link represented an authentic vote of relevancy from one website to another. Now a link may be a paid link, such as links in payperpost or even adsense/adwords links. Nothing is wrong with this, except it violates Google's assumption. In certain cases in the past when Google's assumption is violated, Google tried to punish the violators instead of changing its own set of assumptions. Google forgets that its job is to evaluate the web as is, and not dictate the web and its business models. That's not openness. In fact may hurt the open web and prevent the evolution of competing business models to Google.

In the case of query Robert Scoble, may be one of their assumption is violated. For an example the old webpage might have been evaluated to be more relevant because it may have more links coming in. Google has to figure out how to evolve their assumptions. A query like Robert Scoble is a proof that as the web evolves, the underline assumptions of search engines must evolve to keep the pace with it.

How about Search within Search?

Search for any popular keyword in Google. You will get...more results than you can handle.

Imagine, if you can lock this search result and search again, INSIDE IT. And do it all over again, and again, with multiple filters...

The results are likely to be fantastic.
http://ideaburger.blogspot.com/2006/11/making-g...

Sergey and Larry, are you listening?

Jay, from Bangalore

It's because you left your old website dangling with more links than you'd believe pointing to the old site. Plus Google hasn't done any manual hand-coding for this search.

Typically when you move sites, you should do a permanent (301) redirect to the new location. Just ask Matt Mullenweg to fix you up with a permanent redirect.

I read Stéphane Rodriguez's post, but that post has nothing to do with speed. It's true that Google shows different results for different languages and countries. That's intended, and it's a good idea. The query [football] in google.com is not the same as in google.co.uk, and that's as it should be.

"Typically when you move sites, you should do a permanent (301) redirect to the new location. Just ask Matt Mullenweg to fix you up with a permanent redirect."

You mean Google can't do the heavy lifting? It can't learn by itself? Wow, that's not what I expect from the best search engine.

"I read Stéphane Rodriguez’s post, but that post has nothing to do with speed."

The post Scoble linked to was a follow-up of another post about Gmail speed. Also the statement from Ms Mayer on speed (twice the time for 30 results) led me to inspect quality and consistency.

I am afraid I will have to debunk two more myths about Google : neutrality, and organizing the world's net information.

Matt: unfortunately my old blog is on a UserLand site and I no longer have control of it.

That was a really clueless post by Stéphane Rodriguez. Robert, are you simply blogging everyone that mentions your name?

Paul, you are a coward. Put your last name when you are name calling people.

If you are happy with Google state of the art, fine.

BTW, Matt did respond and is he correct; you, Robert, should put up a 301 moved permanently on your old site. T'is the way of the internet.

As I mentioned on another post, it is not just because of the old site having been left dangling.

Links like this one in your sidebar seriously don't help matters.

http://scoble.weblogs.com/stories/2004/12/15/sc...

Your internal linking structure leaks PR like a sieve.

Quite often the majority of links on any page of this site are external links, even though all your comments use nofollow.

It is possible to setup a wordpress blog that allows 50 external links per page, and still retains most of its pagerank.

It isn't "Gaming Google" increasing and optimizing your internal linking structure, as doing so can also add a huge amount of additonal usability for your readers.

It certainly isn't Google's fault that your current blog is ranking worse than your older one.

Scobie-Doh! > "Matt: unfortunately my old blog is on a UserLand site and I no longer have control of it."

Well, hopefully you've learned something about the importance of having control over the software you use to generate and manage your data.

@24:

Stephane says:

"You mean Google can’t do the heavy lifting? It can’t learn by itself? Wow, that’s not what I expect from the best search engine."

Stephane, what would you have google learn exactly? that the scoble from scoble.weblogs.com is the same scoble as scobleizer.com? yes i suppose it would be neat if it did that, but you seem to suggest that because it doesn't it's somehow some awful flaw (implied by your tone), which i think is hardly the case.

Also, once it does that, what then? it should automatically redirect the user to the link of the newer blog? why is that desirable behavior? i wouldn't call that heavy lifting, i'd call that heavy handed lifting, and of the totally "do evil" sort. google shouldn't be making an assumption about what to redirect and not. There are simple things already in place that handle such tasks.

Perhaps you're trying to balance what you see as google getting too much of a free ride in press/popular opinion and want to point out google's legitimate weaknesses? if so, fair enough, and more power to you, but the particular examples you've adduced on yr blog and above are, to many of its users, not flaws but features that work as designed, and/or are not the most important feature they are looking for google to "improve" on or even implement (e.g., determining that scoble.weblogs.com is the old site of scobleizer.com so that they can issue a redirect).

If i've misunderstood yr intent or yr examples, apologies. i'm definitely happy with someone stirring the pot to get the market leader to continue to excel, but nothing that you've said so far has made me think anything less of google or caused me to want to try a different search engine.

the only company that this thread has raised a question about, in my mind, is userland. robert has stated in the past and again here "my old blog is on a UserLand site and I no longer have control of it." robert is not a go quiet into the night kind of guy so i can't imagine he hasn't asked his old colleagues at userland to issue a permanent redirect for him. (?) Regardless of that though, i guess the only question i would have for robert is, even he did have control over his old site, does he really want a permanent redirect? wouldn't that pretty much make his old content cease to be reachable? or does the redirect only apply to a call to http://scoble.weblogs.com/ but, not, say, to a specific page like http://scoble.weblogs.com/2005/09/30.html.

what do you mean by fixing? if old site has more incoming links accumulated over time then it should be highr?

SEO? oh, I see. your way of thinking is so microsofties.

... microsoftish ...

So let me get this, Scoble wants good to tune up their search engine because his new blog doesn't come up first. Seriously that's nonsense. Google has way more important things to do then fix a problem which was really created by Mr. Scoble himself.

Maybe Google just figures it's OK if they send everyone to your old blog on the first result since your cell number is there for the taking. I can just call you and get your new URL. :-)

Lesson learned: Don't host your own blog. If you own the URL, you're in control.

1) consistency : Google.fr is different than Google.com. Why? This is a bug.

2) quality : there is no bias if Google figures out that the top result scoble.weblogs.com is very old. Any human being that would go there would consequently click the link to the newest site, the most relevant site. Since Google touts the top result as most relevant, the fact that it does not do that goes against it. As simple as that.

3) from Matt Cutts answer using [football] to justify differences in what Google returns based on your IP address, something relevant in principle, Google fails to provide visual cues that it is doing so. The radio buttons below the search box don't tell me Google is doing a "filtering" for me based on my IP address.

@36. Logically I agree with 2. What google algorithm is, whether it is gamed or not does not define what is the relevant link. It is the other way round. So all the explanations, including that of Matt Cutt does not hold water. Whether Robert makes the permanent (301) redirect or not, or whether the old blog has more incoming link does not make his old blog more relevant over the newer one. So when I search Robert Scoble on Google, I as a user do not care about these background issues.

Ethically I agree with 3 too. But may be it is a default understanding between users and a web-service provider, so may not require an explicit disclosure.

Google already allow webmasters to get PRIMARY domain name to be used in links for search results if content is same.

Bad this also need Robert to put some html file on host or meta tags at home page and will result that instead of two current results with Robert blogs - users will see only one.

IMHO, it's better to have two top-10 links instead of one ;-)

Lots of other commenters have pointed this out, but a lot of the rest of us who believe Robert's words may have value, have had to read the whole thing to learn this.

scoble.weblogs.com has tons more inbound links than scobleizer.com, and odds are strong that it is much higher with anchor text "robert scoble".

If you're asking that Google's algorithm weight newer links more highly, then say so. But inflammatory linkbait like "fix google" just draws attention, to useless effect, and cuts the value readers may hope to find in your words in the future.

Stephane, I understand your points from @36. I do believe that many people expect google.fr to show different results from google.com. And it's true that we don't provide visual cues that we're doing different things based on someone's IP address, or the language/country that they selected to search. On the other hand, if we provided visual cues for everything that Google does, our search results page would be very cluttered and busy. :) Instead, we usually choose to try to return good results given what we know from the query, and to keep the output simple. I don't think many (any?) other search engines tell all the factors that change how they return their results, nor do they include messages such as "You are visiting from an IP address in France; you will get different search results." Or at least, I'm not aware of any other engines doing that. :)

As far as the argument that we should recognize and combine Scoble's old site and his new site, I understand the thought that we should recognize that one is a newer site and return that. However, bear in mind that there are several other people in the world with the name Matt Cutts, for example. If we were to combine two sites that were really owned by different people, that mistake would be much worse. That's why we tend to leave it in the hands of site owners to use a permanent (301) redirect.

I realize that isn't the optimal solution for e.g. people who moved from a freehost to their own domain, because most freehosts make it hard to do a permanent (301) redirect. That's partly why we've started giving site owners the ability to indicate their preferred domain in our webmaster console. It's not perfect (right now a site owner can indicate whether they like their site with or without the "www"), but it's a step in the right direction.

Of course, even allowing site owners to combine two sites won't work if the site owner has no control over one of their sites any more--which it sounds like is the situation with Robert. At that point, you've reached the situation where someone says "I used to be at this site and I've moved," and Google says "How can you prove that you used to control this site? Can you modify the pages or put a redirect in place?" And if the answer to that is no, that's a dangerous situation, because person A could claim that person B's site is really theirs. That's fundamentally he-said/she-said situation, and most of the time it's not scalable for Google to investigate those situations; the correct answers would be to use a permanent redirect or to be able to prove ownership of the site.

Anyway, now I'm curious. Robert, in this case it's clear that it really was your old site. If you had a choice, would you like search engines to manually combine your old site and your new site? Of course, even that is problematic, since your old posts don't exist on your new site. If someone did a search for text that only existed in an old post (not for the words [Robert Scoble]), would you rather return the post at the old site, or return some 404 page on the new site?

Probably you'd want the post on the old site to come up. So I'm guessing your final pref would be to keep indexing/returning the posts on your old site, but to return your new site for [robert scoble], yes?

Matt: I want Google to bring up the most relevant page for whatever search people are doing.

So, if they are looking for "Corporate Weblog Manifesto" I'd expect that Google will continue pointing here: http://scoble.weblogs.com/2003/02/26.html since that's where I originally linked to it.

But, if people are looking for "Robert Scoble" well, then, Google should sense that they are looking for a person. The fact that there's a "person" named Robert Scoble who hasn't updated his Web site for more than a year should tell Google that that site is less relevant for names than, say, the "new" Scobleizer site that HAS been updated recently.

Relevancy is hard. And, no, I don't want Google to remove my old stuff. Just, maybe, make it a little less relevant than newer stuff, particularly when it comes to names.

John:>> If you’re asking that Google’s algorithm weight newer links more highly, then say so. But inflammatory linkbait like “fix google” just draws attention, to useless effect, and cuts the value readers may hope to find in your words in the future.

Good point. But, when it comes to a bad relevancy result, which is what this is, since when you're looking for someone you want to find the most recently-updated site that that person has made, I think it's just fine to be a little sensationalistic in the headline. Why? Cause it gets your attention, which got you to participate in this conversation. That helps us all when it comes to figuring out how Google works and how we can urge those who control its algorithms to tweak such.

"But, if people are looking for “Robert Scoble” well, then, Google should sense that they are looking for a person. The fact that there’s a “person” named Robert Scoble"

"I think it’s just fine to be a little sensationalistic in the headline. Why? Cause it gets your attention"

Well then, titles like "Google Eats Little Children" are more sensationalistic. They also don't eat up other readers' time in the process.

Summary: Robert wants Google's results ranking to assign his old links' weight to his new URL.
(No discussion on whether this would be a good and safe general thing for search engines to do -- discussion is just about the one case.)

Matt, I think you are taking a simplistic approach. There are various kinds of website. For some old if gold and for others new is better.

So ideally a search engine must sense that the old website of Robert is a Blog. A blog which is not updated for a year is a bit less relevant than a blog which is updated a week ago. It is immaterial whether both these Robert Scoble are the same. It is also immaterial who has control on the old website. It is also immaterial whether Robert wants to do something to make Google work easier by putting a link structure of Google's preference. Search is Google business and not Robert's. Since Windows Live Search can figure out a mechanism to put Robert's new blog over old, it proves that there is a way, except Google has to work a little harder to find that way for itself. The discussion here could be helpful.

To Robert, I find it funny that you need to spend time arguing those who oppose common sense. It reveals how many developer types are commenting...

To John, the sensationalism is just not the case. If you read my blog, you'll quickly see I am not shy to say WTF when I discover something fishy that could impact a ton of people. If Google were to implement a mood detection factor (hint, hint), that would be more obvious.

To Matt Cutts, thanks for answering again but I still disagree. Here are my points,

1) With that line of reasoning, where you justify it's better if Robert does the "heavy lifting" for Google, in the name of Google, then there is still a flaw : Google.fr returns the right top result.
What I am pointing out is not only that "top result = most relevant result", it's also the inconsistency depending on your IP address.

2) The case of multiple "Matt Cutts" does not work. It is perfectly fine that all Matt Cutts of the world compete in the top results, and that the order changes according to popularity over time. That's not what I am talking about. Google makes Robert Scoble makes compete with himself : that's a bug, period.

3) IP address and country language : it appears the user agent string is not used by Google. Or the weight is so low that it has little impact.

4) As for the lack of visual cues, I am sure that if someone is US does a search for football, and YOU are doing some culture filtering on his behalf, then you may want to select the radio option that makes it clear that you are selecting a subset of results of your choice. This radio option cannot be "web pages" obviously. Because if that were the case, then the user would have no way to force an unfiltered search. That's what I want as a user. I don't think selecting the right radio option below the search box clutters up the page, since there is nothing added.

>>Robert wants Google’s results ranking to assign his old links’ weight to his new URL.

No. I want the most relevant result to come to the top of the page. Searching for "Robert Scoble" should give you my most recently updated page. At least it should give you a page that's been updated more recently than a year and a half ago.

What's ironic is that Google's appreciation for old pages more than new ones is keeping me (and other old school bloggers) in power positions where we get most of the traffic.

What I'm asking Google to do would make it easier for new bloggers to kick me off of the A list.

Think about that one for a while.

It is so simple to improve the results for "Robert Scoble", and increase the search traffic your old posts receive.

All these things actually add to your reader experience. Google's No.1 advice to webmasters is design your website for readers, not search engines.

Add monthly archives to the sidebar
Add related posts to single pages

Create an "about me" page and link from that page to Wikipedia

Remove the link to your old blogroll, and create a new on on this domain (it is giving significant relevance to the old domain)

Link through to your Photo Blog with a redirect such that it isn't followed.

Think about the TechMeMe link - are they reciprocating "in kind" - it seems to me they don't carry every one of your stories

I don't advocate making Google the same as MSN or Yahoo. I love the fact that each search engine gives me different results.

> unfortunately my old blog is on a UserLand site and I no longer have control of it.

This is indeed the problem with hosted services.

Surely someone here could write a sitescraper that outputs some sort of Wordpress-format import, to put all the content (including comments) into your Wordpress system?

Robert, how about you ask people to do that for you, with a prize for the first person?

Stephane said "To John, the sensationalism is just not the case."

Now you're arguing about Robert's trivial reason for asking everyone to invest their time in this discussion. This is another example of how lengthy-yet-vague messages encourage each reader to go off on different tangents.

Just get to the point, and get your core idea across while reducing reader costs. That's the recurring theme here. Applies even more to video than text.

Robert wrote: "No. I want the most relevant result to come to the top of the page."

That's why *YOU* should summarize your comments yourself. I tried, but didn't quite get your message. It helps if you define your comments in functional terms -- right now readers still need to decrypt your intended sense of "most relevant". You haven't yet attempted to check whether your current mystery wish is appropriate to the world at large... you're talking about your links, your content, instead of weblogs in general, distinguishing "weblog" from "newspaper", and so on.

Editing for the reader's benefit... that's still the chokepoint here.

"What’s ironic is that Google’s appreciation for old pages more than new ones is keeping me (and other old school bloggers) in power positions where we get most of the traffic."

This is another non sequitur. All older links to you just oppress all the other "robert scoble" people in the world, and do not affect other people writing publicly. But perhaps you meant "when someone searches on 'tech weblog' they shouldn't see me", which is another digression on an unsummarized tangent.

It's fixed now