1 reply to this topic

[atsaunier]

If Paul Flessner and his SQL Server team have their way, traditional beta releases may soon be a thing of the past. And they aren't the only Softies who are pushing to overhaul the way Microsoft builds software to more closely emulate the open source process.



Consider the beta.
Beta testing has been the cornerstone of the software development process for Microsoft and most other commercial software makers for as long as they've been writing software. But if certain powers-that-be in Redmond have their way, betas may soon be a thing of the past for Microsoft, its partners and its customers.

What's behind the shift?

Beta builds are predictable milestones by which external and internal testers have measured their progress toward the delivery of final code. Betas are typically few and far between, and feedback on them tends to be late and sometimes lame. But they are nonetheless a known quantity, and one that software developers, OEM partners and customers of all sizes rely on in gauging when and how to support a new product.

Open source turned the traditional software development paradigm on its head. In the open source world, testers receive frequent builds of products under development. Their recommendations and suggestions typically find their way more quickly into developing products. And the developer community is considered as important to writing quality code as are the "experts" shepherding the process.

Until fairly recently, Microsoft championed the way it "made the sausage" as the best way. But in the last year or so, there's been a noticeable change in Redmond's rhetoric. Top Microsoft brass have been wondering aloud whether big-bang betas are the best way to develop and test software. Might there be a better alternative?

Microsoft's Engineering Excellence unit, headed by 20-year company veteran Jon DeVaan, is overseeing a project to reengineer the way software is developed inside Microsoft. DeVaan, a member of Microsoft's central Business Leadership Team, manages the engineering standards used to create Microsoft's software products.

With feedback from the Windows Core Operating Systems Division (COSD), Office, development tool and applications units, DeVaan is spearheading a fundamental change to its engineering process. As part of this initiative, Microsoft's major product units will do more "virtual teaming" across division lines, bringing together developers from multiple units to build a new product or subsystem, and then disassembling them again once the project is completed. At the same time, these teams will rely on CTPs (or whatever Microsoft ultimately ends up calling the more frequent test builds) as opposed to betas, to speed up the product development and testing cycles.

In January, Jim Allchin, co-president of Microsoft's Platform Products & Services Division, hinted at this change. Allchin acknowledged that the Vista team was relying on more frequent CTP test builds, as opposed to traditional betas, in its final development stages for Windows Vista.

The SQL Server team also favored CTPs over traditional beta releases in the final throes of SQL Server 2005 testing and development, said Paul Flessner, Microsoft senior vice president of Server Applications. In fact, the SQL team decided to forego Beta 3 of SQL Server 2005 because developers felt the CTP feedback they were receiving was solid and rapid enough to push product out the door, Flessner said.

Going forward, if Flessner prevails with what he describes as the "SQL Server Reengineering Initiative," it's going to be a beta-free world for SQL Server developers and testers.

"If I had my way, there'd never be another (SQL Server) beta," Flessner told Microsoft Watch. "The boss doesn't always get his way, but we are definitely leaning toward this."

This change of heart isn't coming out of the blue. The open source development process has influenced Microsoft's thinking, Flessner and other Microsoft officials have admitted. But there are other factors at work, as well.

First and foremost, Microsoft divisions that have piloted CTP programs – primarily the Windows, SQL Server and Visual Studio units -- and its testers have found CTPs to be a more efficient way to exchange with testers code and feedback on it.
In the last 18 months that SQL Server 2005 was in development, Flessner's team "really got in the CTP groove," he said.

"CTPs helped me keep the engineering team in a branch where we could ship," said Flessner. "It kept the team from checking things in until they were of high quality. We had a very sophisticated branching system and very high-quality reverse integrations that got integrated back into our main builds. That was a super-powerful thing."

With "Katmai," the next major version of SQL Server, Flessner is planning to move to a pure CTP/no beta model, he said.

Granted, what the SQL Server team labeled a "CTP" was different from what the Visual Studio or Windows team labeled a CTP. The Visual Studio team, which established the de facto definition of CTP as a "snapshot of a product at a given point in time," released CTP test builds that were not of beta quality, officials acknowledged. The Windows team complicated matters, by releasing December, Febraury (and soon, April) CTPs of Vista that officials also described as Beta 2 builds. The SQL team, for its part, did full quality checks on each and every CTP it issued last year.
Because of the differences in terminology around CTPs, some kind of new nomenclature is likely for these test builds, Flessner said. Regardless of the label, however, the impact will be similar to that of CTPs as they exist today, he said.

When a feature is ready, it will go into the product," Flessner explained. "I'd like to be in a situation where we can ship (a new release or update) in 120 days, due to market requirements, the team being tired or whatever."

Flessner said he expects the new engineering process to have an immediate and beneficial effect on product ship cycles for everything from the next version of SQL Server, to WinFS, the Windows File System code.

"I'll be able to go to a 24- to 36-month (new release) cycle," he said. "I couldn't do it without these engineering changes."

Based on that timetable, SQL Server Katmai would be a 2007 or 2008 deliverable. And SQL Server 2005 Service Pack 1 – which will include the data-mirroring technology that Microsoft cut at the last minute from the SQL Server 2005 product due to quality concerns – will also come to market more quickly. Microsoft released this week its first CTP for Service Pack 1. Flessner is estimating the final service pack code should be ready in April – many months quicker than the typical service pack is developed and tested.

Flessner said the team is aware there are refinements to make to the process. Customers have reported trouble attempting to mix and match different test builds of Microsoft products.

The ability to install the latest CTP atop the most recent one is a priority, as well. "Customers need to be able to upgrade. When you can't upgrade from one CTP to the next, that is a definite problem," Flessner admitted.

At the same time, Microsoft is thinking beyond how and when to deliver the bits and is also reworking its processes, Flessner emphasized.

"There's the whole question of what constitutes a milestone any more," Flessner said. "It's very much like how Boeing is building planes using sub-assembly teams that are multi-disciplinary."

But there's one key difference, Flessner admitted. "We get to fly each thing with each CTP. That's is our advantage."
Source: Microsoft Watch

[Neon]

Very interesting!

So long as it increases the quality of the product, i like it