Over a million developers have joined DZone.
Platinum Partner

To Beta, or not to Beta - That is the question

· Agile Zone

The Agile Zone is brought to you in partnership with Hewlett Packard Enterprise. Discover how HP Agile enterprise solutions can help you achieve high predictability and quality in your development processes by knowing the status of your projects at any point in time.

"Whether 'tis nobler in the mind to suffer the slings and arrows of outrageous bugs, Or to take arms against a sea of defects, And by opposing end them." (with props to Bill S.)

In the past few years it's become more and more acceptable to release "Beta" software to the public - almost as if it was a production release.

The reasons for that I believe are manifold but boil down to
1) Gain user feedback
2) Release early to gain "mindshare" and/or get first to market.
3) Your QA process or team isn't that great (or your unwilling to spend money on them) and you'd rather have users do your testing for you.
4) You need to gain the confidence of your customers / investors in what you're building.

The number one proponent of the Beta, I must say is Google. I've been a Google Docs user for well over a year now and I love it - but it's still in Beta. The one thing I don't love is the FREQUENT UI changes in direction though - but hey it's a Beta and it hasn't lost a Document or Spreadsheet of mine yet.

But it's clear from using what Google call a "Beta" that their goals are really predominantly #1 (user feedback) and probably to a lesser extent #2 (mindshare). It's been very well tested (internally) before it hits the public. You can read more of what Google think about Betas here.

So the quality of their Betas is quite high (see also gmail) and a Beta for them is probably a production release for 90% of everyone else.

Here's the problem though - in making Beta's "popular" I think they (and others) have lowered the bar at least insofar as management can now look at it and say "Well if Google did it - why can't we?". So there's a tendency now to ship lower quality software and at the last minute people decide to slap a "Beta" label on it if it's not up to snuff so that they can declare victory. Then they hope people will use it as if it was production quality.

Personally I think software professionals should be using a public "Beta" ONLY as a last resort. If you need to gain user feedback - try not to be cheap about it. Get a bunch of potential users in a room and have them use your product - try not to prime them - just watch them. Video them. Understand what are they trying to do - how does that align with your product? Can you make your application so easy to use it doesn't need a manual? That should be the goal (although that might be ultimately unattainable any progress in making your application easy to use is good).

If the reason is #2 (to be first to market) then I think you don't understand technology trends. Neither Google or Internet Explorer (or Firefox) were first to market - look where Yahoo! and Netscape are now.

I guess I never really "got" the whole first-mover advantage thing. Think about it:
  • Apple vs. Windows (and back again)
  • Ford vs. Toyota
  • Betamax vs. VHS
  • Atari vs. Nintendo
  • WordPerfect / WordStar vs. Office
  • Friendster vs. Facebook vs. MySpace
The anecdotal lesson I got from these cases was to NOT be first mover and learn from the other guy and where he fails.

Anyway if you're in the position of #3 (you're not investing in your QA team/process) then that's a sorry state of affairs. Although we live in a world of finite resources - finite dollars and hard-to-find tech experts so I guess it can be excusable sometimes. But if you just punted to force your users to do the bulk of your testing work then shame on you.

If you're in position #4 - releasing a Beta to the public to gain confidence well it's hard to argue too much with that given the sorry state of "production" software these days (*cough* Vista *cough*) people's expectations are sadly low so its understandable people (especially investors, customers) want some reassurance.

Don't get me wrong on this though - I'm not the type of guy to wait until the product is perfect. It's more a question of what are your motivations and are you being lazy in releasing a Beta? If you want user feedback - GET USERS and GET THEIR FEEDBACK! Don't ship junk!

After doing some Googling I came across a blog article with similar sentiments
A call for revolution against Beta Culture

"I'm mostly tired about the fact that it seems that we all have given up. Tired because . . . . in reality, it's laziness and a poor job on the manufacturer part that we have accepted without questioning. Instead of calling foul play and refusing to participate, we keep buying."

"That's the key: We have surrendered in the name of progress and marketing and product cycles and consumerism."

Right on man - it just doesn't feel right. It feels lazy and short-sighted.

Here are some other thoughts by other folks

So here's my question to you, dear reader, what is your opinion on Betas? Useful software engineering technique or cop-out?

The Agile Zone is brought to you in partnership with Hewlett Packard Enterprise. Learn more about driving business innovation by leveraging Agile quality lifecycle strategies.


{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}