MSFT beta testing too hard?

Rob La Gesse wrote me a very constructive email today about how difficult beta testing Microsoft's products is. Instead of emailing it around I asked him "could you blog it?" Why? Cause I thought it would start a good conversation and to tell the truth I don't know who to email it to anyway. Putting things like these in public usually gets to the right people faster than if I try using email. It's weird that way. Even if the right people don't read my blog, usually someone who does forwards my posts along. I've seen that happen hundreds of times already, including several times just this week. (I had a really great conversation with David Webster, the guy who runs all the naming programs at Microsoft, but more on that later).

Anyway, thanks Rob for starting your blog!

To everyone else, this is a good chance to talk about what betas you'd like to be added to and/or give the folks who run the beta programs some constructive criticism about how to run them better. Thanks for the help. 

Comments

  1. Well, I guess I’ll talk about how I’d love to have some Vista action for over the summer to test on my tablet PC (really curious what it’s gonna look like on integrated Intel 915 graphics–no glass I assume). I’m currently having a blast with the Live Messenger beta, which I was REALLY pumped to get into. It’s been great being able to give all my friends here at college invites to the program, and being able to request features/report bugs for them. Thanks for the opportunities!

  2. Well, I guess I’ll talk about how I’d love to have some Vista action for over the summer to test on my tablet PC (really curious what it’s gonna look like on integrated Intel 915 graphics–no glass I assume). I’m currently having a blast with the Live Messenger beta, which I was REALLY pumped to get into. It’s been great being able to give all my friends here at college invites to the program, and being able to request features/report bugs for them. Thanks for the opportunities!

  3. Sure there are a lot of issues when beta testing Microsoft products, but to clarify a few things.

    Giving feedback isn’t nearly as difficult as Rob makes out. Nearly all managed beta programs have active newsgroups, so if you can’t find anywhere else to give feedback, use them. Addressing Mail Desktop specifically because Rob mentions it, it has newsgroups too and is on Microsoft Connect, both of which allow for giving easy feedback to the beta team.

    For Windows Live betas, ideas.live.com is converting a lot of the “informal” testers to official beta users, a definite step in the right direction. There are also centralised sites at feedback.live.com and support.live.com for those users who don’t have access to the newsgroups or Connect.

  4. Sure there are a lot of issues when beta testing Microsoft products, but to clarify a few things.

    Giving feedback isn’t nearly as difficult as Rob makes out. Nearly all managed beta programs have active newsgroups, so if you can’t find anywhere else to give feedback, use them. Addressing Mail Desktop specifically because Rob mentions it, it has newsgroups too and is on Microsoft Connect, both of which allow for giving easy feedback to the beta team.

    For Windows Live betas, ideas.live.com is converting a lot of the “informal” testers to official beta users, a definite step in the right direction. There are also centralised sites at feedback.live.com and support.live.com for those users who don’t have access to the newsgroups or Connect.

  5. Putting things like these in public usually gets to the right people faster than if I try using email.

    thanks for bringing this up. A lot of companies are afraid of blogs airing internal dirty laundry, but I believe that it helps out with good PR in that is shows the company cares about their users. The alternative is to wait till something is 100% fixed and then publish a webpage on it. Well in the months that that takes you’ve probably just lost a couple of consumers that are fed up with your old style of communication.

  6. Putting things like these in public usually gets to the right people faster than if I try using email.

    thanks for bringing this up. A lot of companies are afraid of blogs airing internal dirty laundry, but I believe that it helps out with good PR in that is shows the company cares about their users. The alternative is to wait till something is 100% fixed and then publish a webpage on it. Well in the months that that takes you’ve probably just lost a couple of consumers that are fed up with your old style of communication.

  7. Chris,

    I agree that it’s not impossible to find the right places to provide feedback for MS beta products – but it is also true that it isn’t as easy or as standardized as it could/should be. I think every MS beta product shoulod have a “feedback” button on the primary toolbar – just like that “Easy” button on the commercials. After all, most beta testers are working as free quality testers for Microsoft, and in return Microsoft should just make it “as easy as possible” to help them. Same goes for downloads, polls, notifications of new releases, etc. ALL of that should be done in a standard way from a single point of focus. Don’t make me work to help you, help me help you work!

  8. Chris,

    I agree that it’s not impossible to find the right places to provide feedback for MS beta products – but it is also true that it isn’t as easy or as standardized as it could/should be. I think every MS beta product shoulod have a “feedback” button on the primary toolbar – just like that “Easy” button on the commercials. After all, most beta testers are working as free quality testers for Microsoft, and in return Microsoft should just make it “as easy as possible” to help them. Same goes for downloads, polls, notifications of new releases, etc. ALL of that should be done in a standard way from a single point of focus. Don’t make me work to help you, help me help you work!

  9. Oh, and Robert – about Betas I would like to be added to – Vista, and Outlook 2007 would be cool. My MSDN subscription finally ran out (I guess my previous employer finally remembered not to renew it!)

  10. Oh, and Robert – about Betas I would like to be added to – Vista, and Outlook 2007 would be cool. My MSDN subscription finally ran out (I guess my previous employer finally remembered not to renew it!)

  11. Robert I find the newsgroups do most of what you said in your comment, but obviously some betas are not always the same. As for drop down boxes via menu’s etc, its a nice idea but if you’re testing a product that crashes constantly, it might be a bit difficult to use :)

    Here’s a suggestion though, instead of testing 13 products and finding it hard to keep up, why not test less and enjoy them more? Sure its cool to get all the early stuff, but Vista and Office 12 will just give you even more work to do!

  12. Robert I find the newsgroups do most of what you said in your comment, but obviously some betas are not always the same. As for drop down boxes via menu’s etc, its a nice idea but if you’re testing a product that crashes constantly, it might be a bit difficult to use :)

    Here’s a suggestion though, instead of testing 13 products and finding it hard to keep up, why not test less and enjoy them more? Sure its cool to get all the early stuff, but Vista and Office 12 will just give you even more work to do!