by Larry Richman
January 6, 2007 11:48 AM

Windows Vista RTM Lab Session 2 --

Paul Donnelley, Test Manager, is the man many beta testers know via his communications on the Vista Beta Tester Newsgroups. Windows Vista would not exist, and certainly wouldn't be coming out in 23 days, without beta testing. And the essence of beta testing is bug reporting. The first beta versions of Vista had the Customer Experience Improvement Program turned on by default. This sent non-personally identifiable data to Microsoft, which allowed them to see how many copies were being installed and which versions. This was the first time they were ever able to do this. They also issued different ranges of product keys for different groups so they would know which audiences were installing which versions.

Immediately, any device which showed "driver not installed" was sent to them, and as soon as they could do it the drivers started showing up on Windows Update. By the time of Beta 2, many more drivers were included in box.

Some bugs are relatively easy to fix. Others require a lot of work by large groups. Each day, the daily shipper meeting would allow the team to share information and try to come up with solutions. Much more attention was paid to customer feedback than with previous Windows. Some bugs were not so much feature-related as they were serious problems, such as the inability to close Task Manager. That's an example of something being just wrong that needed to be fixed. It was a complicated problem that needed a lot of time and work to do.

Other examples of "bugs" are not so much "problems" as much as features which were either non-existent or wrong that were fixed because of testers' comments:

1) List view in Explorer was originally left out. It was put back in because of our feedback.

2) The ability to mute the Windows Vista startup sound was a BIG complaint among testers, It took awhile, and they resisted it, but finally gave in and allowed it.

3) The name of the "Documents" folder originally allowed no customization. Now it does.

4) UAC -- we all know about that. Probably the #1 complaint among testers through the early versions of Windows Vista was the ubiquitous UAC prompt. We wrote extensively about it here and lobbied actively to try and reduce the obstacles users had to overcome to get Windows Vista to work without hassles. We took it on as a cause here at PROnetworks, and we achieved results.

These are a few of the features in Vista which were added or changed in direct response to the feedback of the beta testers, many of you here at PRO.

As time went on, MS was able to begin to sort bugs by quality, in terms of who was filing them and the kinds of bugs being filed. For example, large numbers of the "general public" (non-geeks) filing bugs that simply said "Vista will not install" might not be as useful as bugs filed by, say, PROnet members like you who are able to file a "better quality" bug, with more details and information that is useful to the developers. This allowed them to sort bugs by groups of people who were filing them. Users who report good quality bugs will likely continue to do so. This way, MS may simply target this group in the future for a smaller beta program.