Tuesday, January 13, 2009

A problem with feature requests

Among many of my secret lives, I personally follow several mailing lists, newsgroups, and technology updates for reasons beyond the scope of this posting. I follow the WHATWG mailings list somewhat closely. One issue that has been raging quite ferociously recently was whether or not to include RDFa, eRDF, RDF, or whatever into HTML 5.

Many of my readers, especially those for whom RDF is immediately associated with cumbersome, inflexible APIs that should be rid from programs, are probably wondering why one would want to include such a specification into HTML 5. Don't worry, you're not alone. Even Ian Hickson, the editor of the HTML 5 spec, is having problems trying to figure out why. To whit:

One of the outstanding issues for HTML5 is the question of whether HTML5 should solve the problem that RDFa solves, e.g. by embedding RDFa straight into HTML5, or by some other method. Before I can determine whether we should solve this problem, and before I can evaluate proposals for solving this problem, I need to learn what the problem is.

A bit of context: earlier in 2008, there was a previous thread about RDF measuring well in excess of 100 messages. Being a good editor, Hixie asked in this message about what problems it was trying to solve. The response? Seventy-three email messages, most of which promptly ignored the issue. Many sub-discussions centered around things such as quality of search engines, how it should be implemented, etc. The idea of trying to figure out why people should use it got lost in the wind.

And to me, this signifies a problem. One of the most important questions when deciding whether or not to include a feature is why. And it seems to me that this question is the one that is least pondered by proponents of new features. The answer is often some variant of "it's obvious" or a description of what the feature does. The last bit is like trying to answer a question of "Why do you want to put a door in the wall here?" with an answer "Because we can have quicker access to the other side of the wall." At first glance, it's acceptable, but in reality, it doesn't justify the feature (why then, do you want quicker access?).

There are more instances where I've seen this. One of them that aggravates me the most is the proposal to include closures in Java. There are several conflating issues in the entire controversy, so here's some background. There are three proposals for closures: CICE (usually with ARM included), which really isn't a closures proposal, more of a "let's decrease anonymous inner class verbosity;" FCM, a "lite" version of closures which basically just makes methods first-class objects, and BGGA, which is full-blown closures support. By now, when people refer to a closure proposals, it's the full BGGA closure; FCM has all but disappeared, and CICE+ARM is generally only mentioned as a compromise opportunity.

The BGGA proposal can be viewed as roughly comprising three parts. The first is the idea of function pointers, the second is the ability to convert a function pointer to an interface so long as the interface has a single method and the function signatures match, and the third is (in a nutshell) the ability to create control structures. I have seen a lot of valid arguments on both sides for the first two portions, but the third portion still mystifies. Yet it is this third part which truly differentiated BGGA from FCM, and it is there where almost all of BGGA's complexity comes into play.

Just one catch: Why do you want or need the ability to create control structures? One control structure almost invariably pops out: the with construct, or (in other words) syntactic sugar for a try { ... } finally { ... } block. This is where the ARM portion of the CICE proposal comes in—it would add the only construct desired by any sizable amount of people. Okay then, that's one, how about another? And therein lies the problem. It's hard to come up with other examples. Proponents always mention the with and assume that everything else is evident. For something that is definitely going to increase complexity and difficulty in programming greatly (e.g., return 5 would behave differently, in some cases, than return 5;), you better have more than one, easily manually-addable use case.

Another problem is underestimation of what it takes to include a feature. A 16 MB extension for a 6 MB program is completely untenable, so let's include it into the 6MB program! How about introducing multiple tokens as a CSS unit token (think about it for a moment...)?

The final thing that irks me with feature requests is the importance with which people attach. The news that Java 7 will definitely not be containing closures (adding a complex, controversial feature to a specification already behind schedule isn't exactly tenable) seems to have been treated with proclamations that Java is dead or that it will die as a result. I somehow can't imagine that millions of Java programmers will suddenly switch because Java won't have closures—indeed, no programming languages ever in the Top 5 have ever had closures.

Similarly, the announcement that JavaScript had been disabled in trunk Thunderbird builds for email was met with a few vocal opponents complaining (I don't know of any non-Mozilla product that ever had support for JavaScript in email to begin with). And most of my readers are no doubt aware of the furor that removing MNG support caused.


Anonymous said...

Just to correct/inform you on your:

(I don't know of any non-Mozilla product that ever had support for JavaScript in email to begin with)

It's only the most used mail client - Microsoft's O/OE that has had it along with Netscape Messenger.

Wladimir Palant said...

Yes, I have been frequently rejecting/ignoring feature requests where the proponents couldn't explain why that feature is necessary. Or, when they managed to explain, the answer often was: "Let's see, I think there is a much better way to solve this". That's unfortunately a very common problem, but it is one you learn to deal with.

Ian M said...

The difference with MNG is that it was widely used in Asia.

Perhaps Asia wasn't terribly interesting to Mozilla at the time.

Havvy said...

Wonderful post, but since you only touched up on one aspect (explaining why) of giving feature requests (a huge part of my brief history on the web), I shall give the full form you should be able to answer in a feature request.

1. State the current situation. What is the area that you want to effect? This adds familiarity to the request, and thus a basis to start from.

2. State was is wrong with it. Be specific. Don't state generalities and don't state other problems not associated that your current feature request will not solve. That just muddles the suggestion, and will create false hopes for it.

3. State the general change you wish is made. Going into nitty gritty details in not a good idea. Treat it as a summary point for the next section.

4. State specifics. Use numbers, and if you have decided to create an image showing how the change would look, this is where it goes. If your suggestion is good, this is where your debating points will be when you 'defend' against it. If there is anything that could be unclear when this feature replaces (if it does replace) an old feature, make a note of it here.

5. Summarize the other four points in one final paragraph, and wrap it up. This lets it all melt together, and can show how the changes work.

This seems like a lot of work, but in reality, it offers ways for people to find bugs before it is implemented. Of course, this is only for major changes, and minor quick fixes don't need this much time spent on it.

Neal Gafter said...

Actually, we have dozens of examples of control-construct-like methods that would be worth adding to the JDK once BGGA closures were added, such as a version of for-each loop API for iterating over maps, one that makes the index available inside the loop, the entire concurrent-array framework from jsr166, APIs for working with the SQL framework while having connections, statements, etc automatically closed, a version of the synchronized statement that uses java.util.concurrent.locks.Lock, and many others. Just as important are control-like API patterns that arise in non-JDK APIs, where using such APIs currently results in too much boilerplate. APIs give you much more flexibility than language features. For example, it is no coincidence that the ARM-block proposal glosses over the handling of exceptions; there is no good policy that handles most of the use cases.

Neal Gafter said...

One more: the concurrent loop API described here http://gafter.blogspot.com/2006/10/concurrent-loops-using-java-closures.html and implemented here http://markmahieu.blogspot.com/2008/08/for-eachconcurrently.html