Shipping Software

This is a great interview with Tim Marsland who is a distinguished engineer and CTO for the Operating Platforms Organization in Sun Microsystems.  In the interview he talks about the approaches Sun takes to the process of test, integration, compatibility and shipping Solaris.  It gives some great insights.  I particularly liked these sections.

First they use daily builds and eating your own dog food (but don’t use that term) ,  I really like the daily build concept as I have described in previous posts:

We also use the previous night’s build to build today’s system, and every two weeks we put the latest build on the shared file- and mail-server for a large proportion of the operating system development group. We deliberately hold our own feet to the fire. Developers quickly got used to running their stuff on their own desktops and their build machines before integration.

Next a section to use when your manager asks you “why do we need to install the beta”

A few months later, we came up with another interesting idea: the Platinum Beta program. This is an idea where we say that we think our beta software is good enough for the beta tester—not just to kick the tires, but to put it into a production setting. This seems a gigantic risk for a customer to take; the way we made that possible is by giving the customer the support of a development engineer 24/7. Customers in the program get an engineer’s pager number and can call any time and rapidly get workarounds and fixes for their problems. The great thing is that there are customers who are keen on working with us that way because they value the relationship highly and welcome the ability to interact with the engineering group at that level.

This section is great,  it talks about the fact that “testing” in a controlled environment never finds all of the issues, and it also demonstrates the value of making the development team use their own software or the system they are designing to support themselves.  In Sun’s terms the development team become a sort of internal Platinum Beta programme.  I particularly like to make sure that the project and programme managers use the systems they are managing the development of.

When customers do that, they find a completely different set of problems than we do. Obviously, one of the things we’re concerned about is that all of our internal alpha usage is about the things that we do. If you ask “normal” beta customers to test things, very few of them deliberately put it into a place where they’re relying on it, and once they find their first bug, they often give up. The people who are in the platinum program are willing to press on and discover more, particularly if we can fix their problems quickly. The program isn’t very large, but those customers do a production deployment and the engineering team learns an enormous amount from the experience.

Of course when you do lots of real use testing with dedicated users who will help you track down the bugs quality goes way up, a great advantage that the Open Source development community have when developing tools that they actually use.

Once we started down that path, we realized that we were producing “beta releases” that were of equivalent quality to other companies’ production software. We didn’t arrive at that opinion by ourselves. Our customers told us that.

I also like the idea of incremental delivery,  very similar to the daily build concept of incremental development build delivery.  Both have the advantage that a large system evolves in small increments that work before the next increment is added.  At any point in the process you can stop and what you have works, or very nearly with just the last few small changes needing to be debugged.  If you take the big band approach you end up with hundreds of components that nearly work all having to come together at the same time and be made to work, MUCH MUCH more difficult.

That was the genesis of Solaris Express, which is the near-continuous delivery of new stuff. We argued that if we’re using the software internally as part of our production environment, then why not allow our customers who are interested in getting new technology to have access to that software, too.

The picture is of the ‘Shell’ sculpture on Cleveleys beach.  I have it as my wallpaper and it reminds me to relax when dealing with issues relating to ‘shipping software’ that’s not ready.