WebMink

On Java and Openness

[Background: There's been some discussion over on Groklaw triggered by the eWeek article I refer to below. As well as a large discussion on Groklaw (where I have been told I'm not welcome & should stay on my own site), there's a long comment thread to my posting which has grown too long to be readable so I'm continuing the discussion with a fresh posting - my apologies to those of you coming in on the middle of the thread. These are my personal opinions written only by me and don't necessarily represent Sun's view - the terms on the front page apply.]

OK, so I've slept on it, let's see what happens :-)

First, I agree with most of what you say. I believe that freedom is the goal and that open source development delivers that to the developer. I believe that the standardisation of sufficiently proven technologies delivers freedom to implementations to compete on their merits (early standardisation is another matter). I believe provable interoperability and portability - let's call that 'compatibility' - delivers freedom to deployers. That's a simplification of course, but consequently a system that delivers the most freedom to the most people will at best involve:
open standards that were developed in an open forum using restriction-free technologies
open, community-based, innovative open-source-licensed development of those standards
platform-independent deployment and demonstrably interoperable function
That's a kind of freedom manifesto for software - no single part is enough to deliver freedom to all, it takes the set. No-one is delivering on the whole manifesto at present.

You talk about Java and open source:
Sun has resisted calls for making Java open source, opening up its development to outside organizations who would have more incentive to contribute to a project which did not require them to relinquish all rights to their contributions. ... A combination of open standards for Java along with an open source Java development model might be exactly what Java needs to more successfully compete with its primary competition, much as Linux has been able to achieve with a similar approach in operating systems.
I'd probably agree with that. The issue though is not if but how. Java already has open standards at least as effective as any other in the world of software, but creating an open source development community without trashing license-enforced compatibility with those standards is a conundrum. This is the question I could use some help with.

I can hear you're not impressed with Java, but I actually think that Java has done a pretty good job of delivering on that software freedom manifesto, as evidenced by the scale and diversity of the Java community. You say:
Sun has published specifications for Java which in theory allow for multiple implementations, but to date the only viable products available are all derived from Sun's reference implementation. In this sense, the "open standards" approach has failed to free Java from being a one-vendor, one size fits all offering.
Let's lay aside for a moment your assertion that it's Sun that's published these specs - IBM and the other JCP members would disagree vigorously with you - and look at your diversity accusation. It's not clear which Java specification you're referring to. There are many implementations of most significant Java specifications, including open source. I really have no idea how you can even think of calling it one-vendor, one-size.

Many implementers have decided to license the reference implementation from the respective spec lead (not always Sun) to save redeveloping the whole system from scratch, but their implementations are by no means the same so I regard your characterisation as unfair. There are various VMs, various incremental compilers, various tools suites, various library implementations. This applies to each of the Java editions and the result is a range of choice that's stunning with a consistency and compatibility that's remarkable. Just on J2EE there's a choice of something like 30 viable implementations, including open source. The result is so far from 'one vendor, one size fits all' that I really can't see what you're talking about.

You talk about Java as 'proprietary'; let's pick the Jargon File definition of the word because that's probably closest to your intent:
In the language of hackers and users, inferior; implies a product not conforming to open-systems standards, and thus one that puts the customer at the mercy of a vendor able to gouge freely on service and upgrade charges after the initial sale has locked the customer in. Often used in the phrase "proprietary crap".
Interestingly, that's pretty resonant with Schwartz's definition. So you say:
In a battle between two strictly proprietary frameworks, Sun will likely lose out to their competition because of the large advantage in market penetration that Microsoft enjoys with their Windows platform.
I can't agree for a moment with that characterisation, which is insulting to the huge number of experts who have been involved in specifying and implementing the various Java specifications. It ignores the massive adoption of Java (on Windows as well as on the various Unix flavours), and implies that all that matters is the desktop. It also views Java as all about Sun - in fact IBM, Oracle, BEA and pretty much every vendor apart from Microsoft depends on and contributes to Java, and Java is at the heart of a vast range of open source projects too.

For reasons I'd not presume to try to articulate, when the open source definition was written at the end of the 90s (after the Java platforms had been available for several years) it left no room of any kind for licensing to be conditional upon some freedom-promoting form of compatibility. I personally believe this to be an error. If it had, I have no doubt that many Java specifications would already be implemented under OSD-compatible licenses. The consequence is that there's no way to write a single OSD-compatible license and maintain compatibility-dependent licensing, which is a requirement if the Java community is to remain free from monopolisation.

Plenty of people with an almost religious faith in the power of unbounded free markets have argued that license-enforced compatibility isn't necessary. That may be right or it may be wrong, but the Java market is too big a responsibility to mess with solely on the basis of dogmatism. That's been the reason the necessary changes to the JCP that have been carried out over the last few years have been so carefully deliberated, much to the frustration of those with 'easy answers'. When change happens, it had better be right because there will be no second chance.

Given the aggressively competitive nature of the Java marketplace and the number of large companies involved, I'm not persuaded that the high level of compatibility across implementations that deployers find in Java today can be maintained just by good will and market forces. For example, a large player could easily start to expect use of a class library that, for excellent technical reasons, was not compliant with the J2SE specification and their sheer weight of customers would make it the de facto standard. This would 'break' the market for all the smaller players and rapidly lead to monopolisation.

Apache, Sun and others have worked hard to get this far with harmonising the approach of the JCP and OSI (JCP 2.6 seems to me to remove all the remaining obstacles to F/OSS Java implementations) but there are no easy answers and people who pretend there are haven't been paying attention (sometimes wilfully). Working on these problems I'm afraid means leaving aside the question of taking actions that aren't in the interests of the existing Java community, and the vast majority of that existing community believes compatibility to be the 'crown jewels' of Java.

posted at 3:10 PM (UK) | |

Comments:

I agree 100%, as I noted at http://www.geoffarnold.com/mt-archives/000107.html .
I'd go further, and note that while open source may offer freedom from monopoly of delivery (lots of folks can provide distros), it relies upon monopoly of source control to avoid fragmentation. It's a monoculture.

# posted by

Geoff Arnold : 6:07 AM

Don't sweat it Simon. Groklaw is only slightly above Slashdot in the requirements to post and have opinions that match the group. Actually after reading that particular thread, I'd say that Groklaw isn't even above Slashdot.

It's the fanboy approach I detest, where I would gather that most of the posters have never used Java, wouldn't know anything about it other than what Eric Raymond has said in his last missive.

I used to have some respect for Eric, but after his completely incorrect outpourings on Java, well, frankly he should stop believing his own press releases.

Remember for most of these posters if it's not written in LAMP, then it's not real 733t OpenSource (tm) and anything that a company produces is a result of "da man" and is inherently evil.

I work with large companies on a daily basis, and the adoption of Java and J2EE in particular is going through the roof, and at the end of the day, they are the ones that will matter, not what some weenies posting crap on a bulletin board in some weird open source love-fest.

My favourite quote for this kinda stuff is;

"The right to be heard does not automatically include to the right to be taken seriously".

# posted by

Anonymous : 8:49 AM

OK, I read your reply and have to thank you for taking the time to think things over and give such a thoughtful response. I think either I may have given the wrong impression that I do not give Java much credit, or you have misread my intent. In any case, let's move along.

My main feeling when thinking about Java, where it is today and where it could have been, is disappointment. When Java was first introduced, there was tremendous excitement and energy over all the ways Java was going to revolutionize software development and deployment. The Java movement had a lot of momentum behind it. There have been, as you pointed out, very significant positive developments and widespread adoption of Java by a number of organizations over the years. But where is the excitement today?

You missed my point entirely when you say my comments about Java vs. .NET "implies that all that matters is the desktop". The battleground that Microsoft is fighting for is not just the desktop, but all products everywhere that software will a play a part in (which means in practice virtually every electronic device of the future). .NET will be the ubiquitous foundation for Microsoft branded software. Java, in combination with the highly portable Linux kernel, could have been (and could still be) a logical challenger to Microsoft's offering as a universal platform for integrated and embedded software of all types. What holds it back is the resistance many open-source and free software advocates have for the restrictions and hurdles Sun places in their path for creating a Java development environment they can feel comfortable using. Java has accomplished successes in the enterprise software market, and will continue to be a strong competitor in that arena. But they now have to fight the perception, if not the reality, that .NET is a new and improved implementation of the original Java ideas. In the larger context of the software market, Sun appears to have missed its mark with Java.

The link you posted for the article about Apache's Open Source J2EE project helps illustrate some of my points. Because Sun has not open-sourced their conformant J2EE reference implementation, organizations like Apache and JBoss must spend months of effort to reinvent Sun's wheel. Despite all these efforts, it will still be a long while before any open source implementation can claim to provide all of the J2EE components given in the specification. While this strategy may be economically beneficial for Sun in the short run, in the long run it simply gives competitors like Microsoft greater lead time before a truly conformant open-source J2EE is a reality.

We seem to agree that Java's licensing model is incompatible with the open source licensing models available today, and that this situation has had a negative impact on Java's acceptance and growth in a large segment of the open source community. You call compatibility the "crown jewels" of Java, and state a fear that "a large player could easily start to expect use of a class library that, for excellent technical reasons, was not compliant with the J2SE specification and their sheer weight of customers would make it the de facto standard". What you fail to recognize is that license-enforced Java compatibility has not prevented the situation you fear from happening. Microsoft has already in essence created not just a new class library, but an entirely new framework, in an effort to create a new de facto standard. Instead of simply worrying about slight incompatibilies in Java implementations which could have made cross-vendor software integration more difficult but still doable, Sun now faces the prospect of increasing numbers of potential customers being locked into a completely incompatible system.

You write "The consequence is that there's no way to write a single OSD-compatible license and maintain compatibility-dependent licensing, which is a requirement if the Java community is to remain free from monopolisation." That statement may be true, but the goal is stated incorrectly. A more correct restatement would be to say "there's no way to write a single OSD-compatible license and maintain Java compatibility, which is a requirement if the Java community is to remain free." Substituting the need for compatibility-dependent licensing (which is an approach and not a goal) with the need for compatibility (which is what people really want assurances about) leaves a statement which has not been proved. Leaving aside Sun's insistence on license-enforced compatibility opens up possibilities for finding a resolution for compatibility concerns under an open source umbrella. The discussion should be centered more on how to resolve the problems and not on why it can't be done.

You point out that, "When change happens, it had better be right because there will be no second chance." I would like to add that when a time for change has come and gone, it doesn't really matter what your first or second choices were. For Sun, the clock is ticking.

Still anonymous by choice.

# posted by

Anonymous : 10:10 AM

Heya, since I don't like Blogger's Terms and Conditions I prefer to post anonymous, and since you don't like anonymous posters apparently, I'm Dalibor Topic from Kaffe.org. Unfortunately, I'm not the person that's been making all those well-written, almost educational posts on freedom all over your blog, though I appreciate that person's wit and style.

Anyway, I came here by the way of Groklaw, and found your replies to the GrokLaw discussion to be well-written, too, and recognized the unique Simon Phipps style that I came across today already via Bruno's Java.net blog. Clicking on one of the S. links got me here, so I thought I might as well comment on some of the issues you raise, since I'm one the guys implementing a free as in speech runtime for applications written in Java.

I won't go into predicting the outcome of the Microsoft's .net vs. Sun's Java battle between non-free platform. It doesn't matter which non-free toolkit some people may use.

I've taken a close look at the SCSL in my own weblog on advogato and taken it apart with respect to the open source definition. Just adding a 'compatibility-dependant licensing' clause to the OSD wouldn't fix the non-free status of SCSL covered code. As it stands now, SCSL meets 2 out of 10 criteria for open source software. Making it 3 out of 11 means it still wouldn't be open source. You'd have to mutilate the OSD completely to make the SCSL pass the OSD.

And it would be a very bad idea to add such a clause anyway. I fear that the motivation behind the compatibility argument is the desire to control the platform. Control of a platform and freedom to use the code as you wish don't mix. It doesn't make sense to try to mix them in a license that calls itself open-source.

Instead, if compatibility for compatibilities sake is what Sun wants for Java, that would be quite simple to achieve by following the path that Sun has already taken. One of things that I found revolutionarily great about Java was the free beer availability of the language specification and the VM spec, allowing many people to evaluate the language and the platform. Sun made a bold step ahead back then, by putting the 'industry standard' into the hands of everyone who wanted a copy. Compare getting the Java Language specification gratis to handing out a lot of cash for a copy of the ISO C++ standard. That energized a lot of people into using Java for research and development.

The next step, if you care about the compatibility, is to make the test suites for determining compatibility available under a free beer license, at least. Then everyone who cares about compatibility can download it, run it on their platform of choice against their runtime of choice and determine if it is compatible or not.

So if people care about compatibility, they'll chose the compatible runtimes, and have the means to decide which runtimes are compatible with the specification. If they don't care about comaptibility, well, then the whole 'crown jewels' argument is pointless anyway. And if Sun believes that the crown jewels of Java are in compatibility, then they should be willing to show everyone just how compatible they are and the mythical incompatible large player is not. Give every user the means for evaluating compatibility, and you'd make a lot of people happy.

Moreover, I invite Sun to participate in Free Software Foundation's Mauve project, which is working on a vendor independant compatibility test suite for Java API specifications.

You see, the trouble is that even Sun's own Java implementations are in parts incompatible between each other. Sun broke JDBC for example, by adding methods to interfaces. So you can not recompile an old JDBC driver on a new JDK. Or take a look at the Release notes for JDK releases: almost every release breaks binary compatibility with the previous one in subtle, but fortunately documented ways. One of my all-time favorites was changing the values of some finals (mouse-event related, to make room for the wheel) between JDK releases.

For another funny bit, Sun knowingly released JDK 1.4.2 with a compatibility kit failure, according to http://java.sun.com/j2se/1.4.2/compatibility.html#incompatibilities : "In J2SE 1.4.2, there is a known JCK failure. When running with the -enablesystemassertions flag, in rare cases, converting a floating-point number to a string can throw an assertion error.

In J2SE 1.4.2_01, this incompatibility is removed and the associated bug is fixed. The bug report associated with this incompatibility is 4905011. ".

Now, if I'm an enterprise user of JDK 1.4.2, don't you think I may appreciate being able to find out how well the JDK works precisely on my setup from today, instead of the one Sun's engineers tested 9 months ago? The JDK releases from Sun have a long history of being broken on anything that's not among the handful of the certified Linux distributions, for example.

For example, if I switch from 'some-call-it-proprietary' Red Hat to 'definitely-non-proprietary' debian, how can I be sure that the JDK actually works according to the spec? Sun never tested the JDK on Debian.

# posted by

Anonymous : 7:25 AM

Just to set the record straight, you were asked by one poster on Groklaw not to post there if you were only interested in drawing visitors to your own site. His request came not because of any opinions you were expressing there, but rather your request to move further discussion off of the Groklaw site and onto your own. It makes you sound more sympathetic if you give the impression you were banned from Groklaw for being a rebel, but in reality it was just one person objecting to what he considered crass commercialism.

# posted by

Anonymous : 4:30 PM

Just to set the record straighter for the AC who tried to do so above, two posters on Groklaw felt that they no longer wanted me to post on Groklaw. They did this when I asked that comments about the posting I made here be made here instead of on Groklaw where I have trouble noticing them. As no-one sprang to my defence I assumed these two to be representative.

# posted by

webmink : 5:31 PM

If you're going to set the record straight, it helps to be truthful. This is the full text of the Groklaw thread with comments by the two posters you refer to:

"I've got my own comments system and would be happy to discuss if you want to take it over there." by you

"So you post this just to atract visitors to your own site? You are not planning on discussing it here? Hmm. Strange and a bit smelly." by NemesisNL

"The poor sap didn't even have the courtesy to provide a URL! ;) Go figure..." by EnragedBeaver

"Don't know what you two are talking about, I am participating in the conversation here and provided a link to my site twice around here including in the root comment. I just don't want two sets of conversation about the same topic as there's only one of me." by you

"Stay on your own site then. Trying to get people from Groklaw to your own site to increase trafic is something I find a bit smelly, as I said." by NemesisNL

"Not what I'm up to mate, just trying to survive. Try trust and welcome, it's hard enough being in discussions here as it is without this sort of hostiity." by you

I count ONE poster asking you to stay on your own site, and only when you explicitly asked that a discussion be moved here. Thinking that only one person's views might constitute the views of an entire online community is pretty shallow thinking. If you don't want to feel hostility from members of that community, it would help if you didn't intentionally disparage its members with comments such as the ones the posted below.

"I am amazed by how rabid the mistrust of Sun is in this place. There are so many comments here that seem to be based on misrepresentations and half-understood truths or on - I supposed I shouldn't be surprised - legalism."

"I just don't have an account and don't want one - this hasn't been a fun place to hang out so far because people seem quick to shoot & slow to listen."

So it seems your primary dissatisfaction with, and reason for leaving, the Groklaw discussion was that you couldn't find enough people there who agreed with your views. Even the slow listeners could pick up on that.

# posted by

Anonymous : 8:41 PM

So our interpretation differs. You missed my parent comment in your resposting, so you're spinning too - it is impractical to post the counter-examples as there as just too many in that thread. You are welcome to your views and I hear the undercurrent loud and clear, as I expect do most others.

While you are very welcome here Mr or Ms AC I'll not accept further anonymous comments in this thread and will delete them. "anonymous" means not leaving me sufficent identity detail to confirm your identity via e-mail or web search - I'm not requiring people to register.

Just so everyone knows, the "anonymous" feature I have left enabled here is to allow those not agreeing with the Blogger ToS to still post despite the brokenness of the Blogger comments system, not so they can troll. The alternative is to require registration (there's a setting for it) and I am reluctant to do that as I know those who read licenses will decide not to register.

# posted by

webmink : 2:24 AM