NT versus Linux
If you've been paying attention to the press about the Mindcraft survey, recently, you may have been confused about test results comparing Microsoft NT with Linux. The first test performed in April of 1999 by Mindcraft came under enormous criticism from the press as well as from the Linux community. The Microsoft-funded report found NT significantly faster on a number of tasks than Linux was.
Criticisms of improper tuning and failure to involve Linux experts in the testing process came from all quarters. The Linux community proclaimed this test as yet another example of Microsoft's FUD (Fear Uncertainty and Doubt) campaign against Linux. But, in June of 1999, Ziff Davis Labs reran the test (this time not funded by Microsoft), and the results were woefully similar. NT appeared to smoke Linux on several meaningful comparisons. Can this be true? How could Linux come off second to the NT ``sloth'' we Linux zealots so adore to criticize? How could NT possibly best Linux at anything? And yet, Linux experts were available and in attendance at the Ziff Davis Labs comparison.
What Happened? As usual, Microsoft continues in the P. T. Barnum tradition of never giving a sucker an even break. In the Microsoft worldview, all is competition and winning is all there is. ``Losing'' is not an option. Most likely, Microsoft studied the Linux kernel in depth and created a test expressly designed to exploit its weaknesses.
Linux was never designed to be the fastest kernel in existence. The goal for the Linux kernel has been ``high availability.'' It has to keep working, no matter what. If you have to stop and fix (reboot) a broken kernel every so often, that will offset any short-term time gains you make in raw speed of execution. So, the original tests were all designed to be ``sprints.'' They were not designed to be ``marathons'' as you would find in the real world. Nor were they designed to be ``World's Strongest Man'' competitions, as you might find in the real world. The tests were for raw speed over a relatively short period of time on very expensive hardware.
The ZD Labs results were not as favorable for NT as the original Mindcraft study's. The ZD Labs recreation did employ the expertise of various Linux gurus who had taken the time to ``tweak'' the Linux system so that near maximum performance was attained for the platform. But Linux still came off second best. The details of the Mindcraft and ZD studies discoveries can be found at http://www.kegel.com/mindcraft_redux.html
The Aftermath. Several areas for performance improvement have been identified. Most notably, the bottlenecks in the Linux TCP/IP stack that Microsoft dutifully found and designed the test to exploit can now be fixed. Optimizations for high-performance hardware that Linux developers have normally have less access to than Microsoft kernel developers might now be implemented. This has been a chance for Linux developers to test their theories.
The Linux community cannot dismiss these latest results. In a fair contest, ZD labs found NT to be faster under certain conditions. Whether those conditions are realistic ultimately depends on how a customer might use their systems. If they had an application that needed short-term speed on high-end hardware with the need to scale to more than 160 clients, NT might be the appropriate choice at this point in time. An honest and competent consultant would make that recommendation. However, in all real-world applications that I've personally been involved with, endurance has been more important than short-term speed. ``High availability'' has meant more to me than raw speed. The Mindcraft and ZD Labs tests carefully avoided comparing Linux and NT on any of these criteria.
The genuine beauty of the Open Source development model will become apparent in the near future as the bottlenecks in the Linux kernel get fixed. One advantage of the Open Source development model is that the source tree must be maintained every day so that developers who are not in close physical proximity with each other can use it efficiently. Closed source development models like Microsoft's usually have a much harder time matching the turnaround time achieved by Open Source models like Linux's. So, Microsoft caught us once, but it's doubtful they will catch us again on the same problem. In a way, we should consider that they did us a favor.
Other Factors. Expensive hardware isn't always the norm for small- and medium-sized businesses. IT departments don't always have unlimited hardware budgets, so running servers on older hardware often takes place in the real world. Linux will run on a 386sx or better. NT wouldn't have a prayer of doing that. But I wouldn't recommend anyone actually run a server on a 386 if you don't have to. That wouldn't actually be realistic. But it is possible. There are lots of P5's with 32MB RAM doing the daily work of the Enterprise out there that no one hears about because they just work and don't break. You can run Linux printservers, dialup servers, name servers, mp3 servers, or what-have-you, on very modest hardware and get away with it.
Also, we must remember that Microsoft is master of marketing and illusion. As the Bill Gates (played by Anthony Michael Hall) character in Pirates of Silicon Valley said to Steve Jobs (played by Noah Wiley), when Jobs said Windows would never be as good as the Mac: ``that doesn't matter.'' The customers who buy operating systems largely wouldn't understand which is a better choice. They will make decisions based on less than fully-rational criteria. Unlike the Open Source community, Microsoft is not expected to be fair or impartial. They don't have to try and tell the truth. They don't have to try and do what's best for the customer.
As Nicholas Pretreley, publisher of Linux World magazine (http://www.linuxworld.com) admonished, the Linux community should not try to compete on Microsoft's terms. Contests that Microsoft invites Linux to participate in will never be stacked in Linux's favor. But, if genuine and meaningful data can be harvested from the experience that Linux developers might not otherwise be able to achieve, it might be worth it to participate in these apparent spider and fly invitations with Microsoft. There might be enough in it for us. So long as the price we pay in media coverage is not too dear. So far, the price doesn't appear to be beyond what we can afford.