Before the days of the Web, a company would send out pre- release versions of software to a select set of technically adept testers who would run it through its paces, feeding back on bugs and potential feature alterations.
Today, beta software is regularly offered to anyone who asks, and in some instances, the tests are open ended – particularly in online consumer applications. Witness Google’s Google Groups Web site, which has now been in beta for half a decade.
The line between beta and ready to ship appears to be blurring in the corporate world, too. For example, when Microsoft releases a product, it issues multiple patches and updates after the event. Some call this patch management. Others say it’s in-the-field software development.
None of this surprises Carmi Levy, senior research analyst at London, Ont.-based Info-Tech Research, who argues that software has become increasingly complex thanks to increasing interdependencies between operating system, hardware drivers, middleware services and applications.
The increasing complexity of software is also the reason vendors rely on broader beta programs. It gives them a way of testing the software on machines with more combinations of different software and hardware components.
But it also makes testing more difficult, says Levy, and it’s forcing vendors to foist half-baked code onto customers earlier in the development process.
“We’re hearing from customers that they’re scared to death of beta code for that reason,” he says. “It’s not in the production environment, but they’re still scared that IT staff will have to devote so much more time to the beta code evaluation process.”
Support for bugs found during the beta process is far from a foregone conclusion, warns Nigel Fortlage, vice-president of IT at Winnipeg-based brokerage GHY International. “A lot of programmers want you to run out and report bugs, but their job isn’t to go out and fix it,” he says.
Instead, for almost a decade Fortlage has relied increasingly on open source software to run large swaths of his business. Unlike commercial developers of proprietary software, open source developers are not driven by quarterly revenue statements that dictate when they release product, he says, and this can improve software quality.
“Because the products are always so good and so stable, we continue to remain on that beta train, pushing the envelope and looking for new and better ways to evolve everything we have developed over the last nine years to the next level,” he says.
For software development to mature, the whole model must change, says Watts Humphrey, a Fellow at Carnegie Mellon’s Software Engineering Institute.
“The current strategy of building software and slapping it out as quickly as you can and fixing it in test is archaic and old fashioned. No other technology works that way other than software,” he says.
