Tim Maroney ([info]tim_maroney) wrote,
@ 2003-05-26 12:56:00
Previous Entry  Add to memories!  Tell a Friend!  Next Entry
open source as unpleasant subculture
A /. story led me to the archives of the Linux kernel mailing list. In this I am reminded of one personal reason I have never been attracted to the Free Software or Open Source ideology. It requires a major lifestyle commitment which involves the company of unpleasant people, as a quick browse through those archives will demonstrate.

The promise of making the source code for software freely available is easy modification. That is, given that software often needs to be customized to meet the requirements of a particular project, a wealth of free software should make accomplishing any project's goals easier, because one can modify something that is already close to the requirements instead of starting from scratch.

The reality is different. When combining small free tools for simple functions, like adding simple forms to a web site, and given a good initial tool selection, then one can often apply this method and quickly integrate some simple command-line-based tools, or perl or Python modules, without a great deal of startup time or lifestyle commitment to the project. Modifying the large open source projects where customization presents interesting possibilities, such as the Linux or Darwin kernel, the GCC compiler, or the Mozilla browser, however, is a very different story. They are not particularly penetrable without help, and it can be very misleading to try to understand them by reading the code oneself or using whatever sparse documentation may be available. They require a lifestyle commitment and subculture participation.

Large open source systems are sprawling and Frankensteinian, and one is going to need the guidance of the mailing list and chat room to get anywhere with them. For instance, there may be a paper on memory allocation in the system (a basic service required by almost all software programs, but often different from program to program). This document might well describe only a hoped-for future direction, which was never implemented after the person left the project. Going through the source and looking for memory allocation might fail to reveal that there are actually five different memory allocators, all representing partially tuned attempts to deal with different aspects of the problem, and scattered throughout the code controlled by a set of macro expansions. Of course, the community knows this already, and therein lies the problem.

Popping into a chat room to get an answer is really not that odious a responsibility, in theory. In practice, though, one has to actually integrate into the community, spending long hours bartering for help, and trying to make the best of dealing with unpleasant and undersocialized people who often have little social status or acceptance outside the community. The level of discourse ranges from abusive to actionable. Simply getting straight answers is difficult. Ordinary technical issues become the objects of gargantuan philosophical and religious debates, usually uninformed by any reading in the relevant computer science literature. Heaven forfend that someone should drop in with a well-meaning technical question that has become a hot-button issue for the insular software-project subculture, or wishing to help solve a problem that has been vehemently denied for years.

Every free project community I've encountered so far is run as a fiefdom and partakes of a highly polarized insider-outsider dynamic. Abuse is taken for granted and encouraged toward newbies, as a kind of hazing that establishes loyalty to the group. The prize is the sanctioned ability to give more abuse than you receive. These seats are highly coveted and it is assumed that newbies will be contenders, so abusing newbies and driving them away serves a function beyond initiatory bonding as well. The best way to take them is to be willing to soak up a lot of abuse and to spend sixteen hours a day glued to every relevant chat room, mailing list, bug database and cvs archive, waiting for some justification for your next tantrum. Take another look at the Linux Kernel mailing list, reportedly one of the most functional communities in open source. It runs about 250 messages a day. This is all these people do.

Between the time drain and the unpleasant company, making a significant contribution to free software and open source would require a kind of lifestyle commitment that I am not prepared to make. Having an ordinary software engineering job already leaves my life revolving around software more than I would like it to. I don't have time to invest in an online apprenticeship in a "free" project that wants to eat months of my life before it will give me the time of day. In my profession, I don't have to. I'm not a newbie at more than twenty years experience. The people hiring me know my references and background and they treat me with professional respect and collegiality. If I have to spend a large portion of my life on software, I'd like to do it in an atmosphere of decorum, not in a chat room getting jeered at by sixteen-year-old aspirants to a retro cyberculture dream world that never existed. I'm a software professional, not a software nerd. The center of my life is elsewhere, and status in such an insular and unpleasant community has no appeal for me.



(Post a new comment)


[info]isomeme
2003-05-26 02:42 pm UTC (link)
I've definitely run into the same dynamic on many open source projects. At a previous job, we were using an open project as a component in our product, and needed to get a code modification integrated into the open project in order to do what we needed without forking the codebase. Now, this was a very beneficial change to the code, which subsequently has been used by many people; but we still had to pass through a "hazing" period unrelated to the merits of our submission before we were judged worthy of having our code committed to CVS.

Not every open project is like this, though; there are huge cultural variations from community to community. For example, I'm currently working with Apache Cocoon, which is sparsely documented and fiendishly complex, but is supported by a developer/user community unusually willing to provide straight, pertinent answers to well-phrased technical questions on the project mailing lists.

(Reply to this) (Thread)


[info]tim_maroney
2003-05-26 09:32 pm UTC (link)
Apache does seem way above the mean for open source software. many of the side projects are quite impressive, and I'm glad to hear at least some of them place value on collegiality.

Thanks for the tip on Cocoon. The separation of functions to reflect different core competencies is particularly intriguing -- whether this particular model works out or not, it reflects much more concern for the authoring process than most tools.

I was sufficiently interested to start installing it, but I found the 13-page install instructions daunting, and that doesn't include the other tools I would have to install like Tomcat and Ant. Even the relatively good open source projects still have the Frankenstein problem, or more specifically, painful installation and administration costs, which I would say result from deferring integration costs onto the user. As a programmer and designer I want to concentrate on development tasks, not on system-integration tasks.

(Reply to this) (Parent)(Thread)


[info]isomeme
2003-05-26 09:57 pm UTC (link)
Yes, you're definitely right that the Apache projects in general are the cream of the crop in terms of both attention to architectural and authoring issues and in how they treat newcomers.

If you already have a servlet container running (Tomcat, for example, but anything 2.2-compliant should work), then installing the basic Cocoon package is as easy as building or obtaining its warfile and deploying it. Most of those 13 pages are about setting up the tools needed to build (Ant) and host (Tomcat) Cocoon, rather than Cocoon itself.

That being said, getting beyond the default install and actually doing your own stuff with Cocoon is a bit daunting, hence my forays onto cocoon-dev and various wikis in recent weeks. The "Frankenstein" image definitely applies here. I'm hoping the payoff in separation of concerns in our View layer will pay for my investment of time in climbing this learning curve, but alas there's no way to be sure a priori.

(Reply to this) (Parent)(Thread)


[info]tim_maroney
2003-05-29 10:56 pm UTC (link)
Tomcat installation on OS X from source was giving me major headaches (kind of surprising since I've successful installed it before on Solaris). I backed off for a while, then did a Google search for better instructions and found this, which made it much easier. A successful install is pleasing, of course, but at the same time, this free software experience always leaves me scratching my head and asking, is there any reason just installing something needs to be this hard?

The question itself is politicized and subject to the status dynamics of the open source subculture, which is to say, asking it is punished in a variety of ways. Gee, I didn't have any trouble, wonder what's wrong with you is the most common one; a more tolerant approach simply complains in an exasperated way why this person is objecting to the same kind of thing we all have to do every day. A cynical approach welcomes the difficulty as a guarantor of job security and lashes out at the threat to the established order.

My point as an application programmer on Mac and Windows is that to achieve results in that kind of programming, I don't have to constantly jump through this kind of system administrator or system integrator hoop. There's no good reason the server-side software needs to be written this way. I view it as a kind of slackness I wouldn't get away with in my software world. In client-server software, though, it's business as usual.

(Reply to this) (Parent)

(Reply from suspended user)

[info]isomeme
2003-05-26 03:04 pm UTC (link)
I can understand the risk to a developer of trusting GPL to protect their work against commercial exploitation, but what is the risk to a company which is willing to work within the fairly well-known boundaries of legitimate commercial use of GPL software?

(Reply to this) (Parent)(Thread)

(Reply from suspended user)

[info]isomeme
2003-05-26 06:47 pm UTC (link)
Interesting point. Of course, the vast majority of software licenses and contracts are never tested in court; GPL just happens to be (a) especially well known and (b) ideologically driven and couched in unusual language. Those latter factors may make it a more attractive or likely target, but it should not be forgotten that nearly all licenses and contracts remain untested in the courts without ill effect.

(Reply to this) (Parent)(Thread)

(Reply from suspended user)

[info]tim_maroney
2003-05-26 09:41 pm UTC (link)
The GPL is particularly vague on one key point, which is when two pieces of software interact so intimately that the GPL infection carries between them. A proprietary driver might considered too intimate with a GPL kernel due to the intricate messages sent between them at runtime, for instance. The GPL FAQ explanation of "mere aggregation" acknowledges its own vagueness and that the question would be settled by the courts.

(Reply to this) (Parent)(Thread)

(Reply from suspended user)
OSS teams
[info]maxomai
2003-05-26 03:15 pm UTC (link)
In a more engineering-oriented note, I'm personally of the opinion (consonant with your description of memory-allocation implementations) that 10,000 substance-abusing college students do not an effective engineering team constitute....

It's a cute remark, but it doesn't reflect the reality of most open-source projects. In fact, most projects, including some of he most important projects, consist of no more than a dozen people. The Linux kernel and GCC are notable exceptions, because everybody wants to make an important contribution there (and because almost nobody does.) Eventually, the really critical projects (like the Linux kernel, KDE, GNOME, etc.) turn into full-time professional pursuits for the core people.



(Reply to this) (Parent)(Thread)

(Reply from suspended user)
Re: OSS teams
[info]maxomai
2003-05-26 03:50 pm UTC (link)
Your "exceptions" were precisely the elements we were investigating...

And I agree with Tim's assessment of the culture surrounding the large projects. It reflects both my experience and Eric Raymond's more recent experiences.

I'd take the position that a dozen substance-abusing computer science majors don't constitute an engineering team, either, for what it's worth.

This is an unfair, inaccurate and malicious characterization of many teams that are doing high-quality work.

(Reply to this) (Parent)(Thread)

(Reply from suspended user)
Re: OSS teams
[info]tim_maroney
2003-05-26 09:56 pm UTC (link)
I took you as referring to a naive view of open source that is common among certain Raymond adherents -- that opening a project will create thousands of free resources to work on the core code. I recall Mitchell Baker citing this in an interview as a false expectation they had at the start of Mozilla. The many-eyeballs principle is one of the most important parts of the ESR model, and it is not borne out in practice. The more eyeballs one has on a project, the more distractions one has to manage. Casual third-party bug fixes in complex code bases are usually carried out without understanding of correct (much less best) practices in that source code, so they introduce problems more often than not. Non-committed and non-core contributors are rightly viewed with suspicion since their contributions are likely to be destabilizing. This is why successful free projects usually have a small number of core contributors and are insular -- it's not that open source can't work, especially with corporate contributions, but that it does not live up to the narrative open source has of itself.

(Reply to this) (Parent)


[info]phygelus
2003-05-26 03:09 pm UTC (link)
instead, you post to slashdot.

(Reply to this) (Thread)


[info]tim_maroney
2003-05-26 09:47 pm UTC (link)
Fortunately, kibitizing is easy!

Still, I've been gratified that by keeping my concerns civil and reasonable, I've been able to help make more space on /. for positions more complex than the formerly de rigeur "d00d, you need to read The Cathedral and the Bazaar again." I'm not the only poster who's helped to legitimize discussions of problems with open source and free software by any means, but I do have some feeling of accomplishment.

I've also tried to work the issue from the contribution side, but that's where I acquired the experiences I summarize above. I could do it but it would require more lifestyle commitment than I am willing to give.

(Reply to this) (Parent)(Thread)


[info]phygelus
2003-05-27 02:01 am UTC (link)
Oh, kibitzing can be lots of fun, and sure, contrary to folk wisdom, sometimes one can even accomplish things in the process.

I mostly intended a snide comment on the nature of the slashdot "community". I nearly wrote "Instead, you post to slashdot and join occult orders"...

(Reply to this) (Parent)


Create an Account
Forgot your login?
Login w/ OpenID
English • Español • Deutsch • Русский…