There Is No Open-Source Community…
… and what to do about it. A few thoughts I’ve finally gotten down.
Professor Jerry Mechling invited me to be a guest instructor in his “Leadership for a Digital World” course at Harvard’s Kennedy School of Government today. As part of the class, we’re interviewing Massachusetts CIO Peter Quinn on the state’s priorities for IT-enabled initiatives. Peter is a well-known proponent of open-source software alternatives for the public sector, and a primary force behind the Government Open Code Collaborative.
To help myself prepare for the class, I’ve put down some observations on open-source which I hope will be helpful to others for whom open-source is unfamiliar but potentially important. In the past, I’ve been a user of open-source software, a senior executive in both open- and closed-source software firms and at an impartial integrator, a board member of an open-source software consortium, and a sponsor of mission-critical application development efforts that use open-source software in major organizations. So hopefully my experiences will be helpful too.
A few words of explanation:
There are two ways of distributing software. One way is called a “binary” distribution. This means that what you get is a bunch of ones and zeros that a computer can make sense of, but which humans can’t. Another way is to distribute the “source code”, or the code in which the program was originally written. This allows users of the software to modify the source code for their purposes as they see fit.
Binaries can be distributed free of charge (this is sometimes called “freeware” or “shareware”), and they can be copied. They just can’t be modified. In commercial software, getting binaries to work requires a license key, a unique code that unlocks the binary. You can copy commercial software if permitted (for example, to make a backup copy), but you need that license key to make it work. Sharing the license key is usually not kosher.
With open-source though, you can view, copy, edit, and redistribute the “secret sauce”. However, the terms of permissible redistribution vary widely according to the specific license under which a user originally obtains open-source software. Some licenses like the GNU (GNU’s Not Unix) Public License, or GPL, require that any modifications made to the original source code must be redistributed openly (that is, as open-source, not binary format) if they are redistributed along with the original source code (which itself must of course be redistributed as open-source). Other licenses, such as the Mozilla Public License, or MPL, allow open- or “closed-source” modifications to piggyback on re-distributions of open-source software, at the discretion of the people or organization who write the modifications or extensions to the base code.
The proliferation of open-source software licenses — check out http://opensource.org — illustrates both the vitality of the open-source world, but also the myriad and widely-varying interests and strategies being pursued by its participants. Which brings me to the principal point of this essay, namely, that there is no such thing as “the open-source” community.
Why does this matter?
I meet people all the time, both folks who work in open-source software projects as well as people who are current or potential users or sponsors of applications that use open-source software, who make assumptions about what open-source means, how it will work, and how its providers will behave, that are based on really bad assumptions about things like quality, cost, and process. Many believe generally that open-source is either inferior or superior to closed-source alternatives. A better perspective is that “it depends.”
The first rule of software selection is to have a clear idea of what your requirements are. A common mistake is to stop at defining the features you need, and to ignore performance (including scalability of this performance to however many users you need to support), usability, extensibility (how sophisticated and well-documented are the API’s provided), usage and support, total/ lifetime cost of ownership, and degree of legal risk, among others.
The most helpful thing you can do when considering open- and closed-source alternatives to meet whatever needs you have is to forget the labels “open-source” and “closed-source”. For any given set of requirements, there are open-source alternatives that are far superior to most if not all closed-source alternatives, and the reverse can be equally true.
So how do you figure out what software makes sense? Unfortunately it’s getting harder. While I claim that there is no open-source community (certainly not any more, though there may have been one in the past), there are many open-source communities. Each of these has its own unique and often competing interests. In the past, this divergence of interests has been masked by the existence of a common “enemy” — usually Microsoft — and smaller, less mature markets. But, as the markets for open-source software expand and the communities become increasingly commercialized, competition intensifies. With this competition comes, via human nature, a lot of hype and FUD-based (Fear, Uncertainty, Doubt) marketing. Each project’s proponents make claims about the universal value and applicability of their own hacks, in many cases ahead of any reality. In the commercial / closed-source world, the defense is opacity — you can’t see the code to evaluate whether or not these claims are real. In the open-source world, the defense is extensibility — if you don’t like what you see, you have the freedom to extend it. The latter defense is better than the former, but not by much.
(Related to this is a problematic expectation for how new open-source code gets developed. Many people assume that if they wait around, “the open-source community” will sense their needs and develop what’s required, for free. I suppose this is statistically possible, in the same way that Shakespeare’s monkeys will eventually get around to finishing things. But as a practical matter, most open-source software development is funded by someone. And even if someone else builds what you need, you have to be mutually aware and willing to engage. Also, it’s especially nice if your paths are roughly parallel and not just the momentary crossing of significantly different development vectors, and this takes ongoing coordination, which itself requires funding. And it’s even more important that there not be two of you, but lots more users, and committed users at that. Naivete about this is not unique to users of open-source software. Plenty of people “in the community” have expectations for altruism that get disappointed all the time. A better assumption is that people might share what they have already (funded and) developed themselves, if it’s in their interest to do so.)
What can users do to help themselves?
I’ve been thinking, not originally, that the best way for users to approach open-source software projects is to “open-source” their requirements, and work to reconcile their differences into as few specifications for starting-point solutions as practical. Then let software providers, both open- and closed-source, compete based on the best combination of answers to the different dimensions I described above. (Good answers come with proof points — real users.)
In the private sector this is harder, because those requirements often reflect proprietary trade secrets (implied processes, etc.). But it seems to me that in the public sector, and in education as well, this secrecy constraint does not exist or even make sense. I suppose these requirements are already public somewhere, but they sure are hard to find. Further, they are most certainly not reconciled across all of the different entities whose needs are sufficiently overlapped to make it worthwhile to do so. Perhaps there is a role here for intermediary associations. An enterprising community that leads this “standards-setting” effort on behalf of potential users, which also benefit from intimacy with them, might make a good partner.