Bugs and trends

As anyone who hangs out on IRC, subscribes to Mozilla's newsgroups, or reads key mailco blogs on a regular basis knows by now, Thunderbird has dipped under 3,000 UNCO bugs. The key chart on the matter now shows quite clearly that we have beaten that number, the first time in about a year. But this one metric does not reveal all.

First on my list of things to point out is the trends. The UNCO count peaked at 3500 and a bit earlier this year. Extrapolating is not the easiest thing here, but it seems that the overarching trend before the recent freefall would have pushed the UNCO count as high as 3700. Meanwhile, one might fret at the uptick in NEW bugs. However, the overarching trend may have been changing before the bugdays (very hard to tell); worst-case, it's only 200 above where the trend line falls. So 700 bugs have been "confirmed" in some sense at the expense of 200 of them staying valid. Impressive work.

Second is to point out that Thunderbird bugs don't represent the entire picture. Combining the bug counts from the Core mailnews components (Mailnews: * and Networking {IMAP, POP, News, SMTP}) and the mailnews side of Seamonkey as well, the aggregate UNCO count peters out to 3268 as of this writing (which is in the middle of a bugday, albeit a rather tame one). All open bugs come out to 10913.

Some components are "healthier" than others. I would consider a component healthy if the number of UNCO bugs is low with respect to NEW bugs, and the NEW bug count is low with respect to all bugs. For core components at least, I would consider both thresholds to be about 10%. The component I follow most closely (Networking: News) satisfies the first metric but not the second (7/143/897). I've not worked out numbers for the Thunderbird side, but a 30%/10% does not seem irrational. Surprisingly enough, Thunderbird does have a NEW count of about 10% of all bugs (2187/20403); the only component not close to this is RSS (106/658).

Time to jump back into bugday and nurse a sick component back to health!

Threading in mailnews

I was cleaning up my WHATWG mailing list folder—a task which mostly involves looking at the subject of a message and deciding whether or not I cared to keep this piece of correspondence—when I thought about how threading interacted. If you haven't subscribed to this mailing list (which I doubt most readers have), the main WHATWG author (Hixie) writes a message which is a reply to several messages at once.

A brief aside, if I may: for completely unrelated reasons (responding to a new bug which turned out to be a dupe, stupid me for checking validity for duplication), I was perusing RFC 2822, specifically the In-Reply-To field (§ 3.4.6). Interestingly enough, the case of a message having multiple parents is quite well-defined in the spec (and Hixie violates the spec on this point). I did a brief check of the code on this point, and the code will handle the theoretically correct case fine (using In-Reply-To in lieu of References, which is not quite correct, but works for the purposes of threading).

Anyways, the thing that caught me the most was that I often cared more about Hixie's catch-all reply than the earlier message to which the reply had been attached. In essence, I wished for the ability to reroot threads. I thought a little more, and listed other threading enhancements I wanted. But there already is a mammoth chart of threaded view issues—see bug 236849 for a sublist of many of these.

At the core of threading, one can distinguish several levels of threading. The basest is none at all; this is represented by turning threaded view off. Second is relying on subject: one can only tell that two messages are related by this methods, but not which is a reply to the other. Third is typical threading, relying on In-Reply-To and References, which works well. Fourth is what I like to think of as über-threading: parsing the message text to determine the quoted replies and use that to determine the parent of a message. Fifth, and highest, is the ability to redefine threading as the user sees fit. Note that most of these are orthogonal, so that one can have a combination of the inner three to determine a thread's parent.

The utility of redefining your own threading is hard to over-state. How many of you have received email where people blithely hit "Reply to All" and start a new message like that, but others in the same category legitimately use reply features? I myself have one thread like that composing 20 different real threads. Other times you hit those cases where someone one a borked client (*cough*Yahoo!*cough*) and someone changes the thread subject, or a confluence of mailing lists and forwarding and replies (four threads where one is warranted, again in my inbox).

There are touchier areas with respect to threading. For example, the notion of subthreads is powerful (there are RFEs to implement practically every "Apply xxx to whole thread" as also an "Apply xxx to subthread"), but it is a pain in the backend, not least of which is the fact that we have some other bugs inducing loops into the thread hierarchy there. Similarly, the question of what do with multiple parenting (both how to represent it and how to generate it) can be touchy on the UX end. A final thorn I would like to specifically direct your attention to is the idea of dummy thread headers, as referred to in jwz's algorithm, the seminal work on the matter (ignore his anti-NS 4 rant, however, he lives in the glory days of NS 2).

On the other hand, don't expect me to implement any of these improvements soon, nor anyone else for the matter. I merely wanted to express my opinions as Thunderbird drivers debate UI on a higher level, with a tendency that seems to be somewhat towards ignoring some of the finer aspects of good message threading. Ah well....

Documentation in Mozilla

Having worked in depths of poorly documented, just plain undocumented, or, worse yet, misdocumented code, I have started taking some initiatives on documenting code. Working with db48x, we have improved some of Mozilla's documentation framework (achievable with make documentation). I'm still polishing the fine edges of bug 433206, but what's in there should be sufficient to make spiffy documentation. The other important component of fix comes from doxygen bug 535379, a simple fix that handles Mozilla's IDLs better.

There's still more to go. There should probably be an official documentation guide for mozilla or at least the components. Someone patching up SVG and dot in doxygen would be helpful, especially the annoying URI mistake.

But the important part is how to document code. At the moment, the class list is provided in a 5-by-several hundred line table containing every IDL file and all exported headers. Wondering about how to do some IO foo, but don't know where to look? Right now, your only choice is to go through this entire list, guessing at names that would produce the right magic. Ideally, however, the documentation would include separate modules that make querying easier. However, before I make a commitment, I need to investigate how namespaces interact with doxygen for best results.

So, the important question is basic documentation. Doxygen's manual is a good starting point, but I'll brush up on basics. Documentation is signified by, alternatively, /*!, /**, /*<, ///, //!. or /*<. The ones with < in the names are used for post-documentation. A comment consists of a brief description (one-sentence, punctuated by a period), followed by potentially several paragraphs of almost-HTML code (doesn't have all HTML tags). Interspersed, though typically at the end, are doxygen tags, denoted by your preference of @ or \ (the majority of code uses @, just to warn you).

To describe all the tags would be arduous and pointless. Common ones are exception (the nsresult values), note, param, and return. See should probably be more common as well. Links to other documentation can be generated by providing the fully-qualified member name, e.g. nsMsgFolderFlags::Directory. Code can be further grouped by using the name tag and @{...@}. The latter signifies a group; one can also distribute comments across multiple members using the format.

More advanced documentation that might be helpful: lists (you can use HTML tags or -, #, and indentation, to represent unordered lists, ordered lists, and nested lists in said order). Formulas can be specified in LaTeX format if you really need it. Message sequence charts can also be generated, as well as generic dot diagrams as well, in addition to the ones doxygen generates for you. But the documentation pages can never be better than the sources from which they are derived...

Updates

So, as May draws to a close, I have some updates on my mozilla work. First off is the pain I have been enduring as I work on listarchive. As of right now, I can create an account through the account wizard, and have it show up in the folder list without anything blowing up (which is not trivial). It turns out that the lack of the folder being initialized through RDF was that I retrieved it from a direct create instance instead of retrieving it from the RDF service. With a few more functions, I could probably avoid the RDF service, but until jminta finishes his war on RDF, it's just as well to use RDF and have things work magically. Of course, listarchive still requires some modification of sources to work, and don't even think about asking me to port it to Thunderbird 2, it's bad enough on trunk!

In addition, I have put some quality time in with my documentation of creating new account types. A lot more exists in my local work than exists on said page in part because I want to have working prototypes before committing to a guide, and in part because I need to make sure I'm not relying on deep-level unreliable code before posting.

Part 3 of work concerns documentation. If you rely heavily on mxr, you may have noticed that over the past week or so, some idl files got some nifty diagrams. These diagrams are created by doxygen (db48x is so nicely hosting them). You can generate these diagrams and doxygen documentation by typing make documentation from the root makefile. It is helpful to include this update to mozilla, and this update to doxygen. The mozilla update squelches some problems and changes some options to make documentation look better. The doxygen update makes doxygen like mozilla's IDL files better. If you want more tips on documentation, this site is a good starting place. (Writing some mozilla documentation guides is creeping up on my todo list as well).

Various other small things: I've been working on the address book rewrite some more. I have some updates to fakeserver related to writing some tests for old news subscription bugs, and to fix some bugs in code that was never tested. Updating morkreader, doing some pref analyses of de-mork of panacea.dat. I've even found time to start maintaining an account manager rewrite proposal.

Break time's over, here weI go!

Visual Studio and Mozilla

Visual Studio and Mozilla, sitting in a tree, K-I-S-S-I-N-G… Okay, maybe they're not kissing, but I'm getting them to like each other a whole lot more. I have a quick-and-dirty script that creates the filter list for the .vcproj from the mozilla directory. It appears, however, that mozilla has something on the order of 23000 .cpp, .c, .idl, and .h files, which makes loading the class viewer a bit slow. It also takes quite a bit of time to load the project in the first place.

Right now, my script only loads .cpp, .c, .idl, and .h files into the project, and does so dumbly. Version 1.0 will probably read the Makefile.in files to determine which directories we shouldn't look into, based on a simple heuristic of GNU_CXX being undefined, and OS_ARCH equalling WINNT; I may make some broad-based assumptions about platforms as well (don't include MOZ_THUNDERBIRD unless we're thunderbird, e.g.). It will also prune "empty" directories; in this regard, perhaps generating the folder tree from the output would be easier.

Still to do is to set up options such that clicking "Build" or "Debug" does The Right Thing™. I'd also like to investigate some performance problems, but that's a long ways away. Eventually, I'd like to hook up the project file to a customized add-in that adds mercurial checkout support and adds/removes the files as happens in the hg repo (but it won't run those commands: I'm assuming people are using shared-source trees here).

Announcing Start of Listarchive!

Last night, I started work on the first of my triad of extensions detailed in this earlier post, listarchive. So I can't come up with a good name. The most up-to-date information will be available at this web page, but I will summarize some key points here.

listarchive is an extension that first and foremost aims to provide sane access to mailing list archives as if they were a simple folder in Thunderbird (and probably Seamonkey as well). From a certain user request, it is likely I will touch on other aspects of mailing lists as well in this extension, e.g., the desired "Reply to List" feature. In addition, however, I will also be using my experience with developing this extension to write a guide to developing extensions involving more complex operations in Thunderbird.

The extension, however, is still very much in the design phases. I would very much welcome any feedback to be sent to Pidgeot18+listarchive@gmail.com (with the "+listarchive" component, it helps me triage replies). Specific pieces of information I am looking for at this point:

• Mailing list archive URIs; I ask in advance to not send any more mailman URIs (unless it happens to be in a different language than English), as I have several already. I am most interested in accumulating a diverse supply of mailing list implementations and international versions to best determine what impact internationalization plays on list archives.
• Ways in which you think a mailing list archive should tie into regular email. For example, I believe it should translate a mailing list URI reference (e.g., http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2008-May/014646.html) into a link to open the message up as a TB email message.
• Any questions or suggestions for UI (this comes more into play once I have some prototypes working).
• Any other question you may have (related to listarchive). I'm quite willing to answer them.

Thanks in advance for any comments you send me. I'm also hoping that this will work out well (no reason it shouldn't).

An Update on Life

So I am quite happy today—then again, I'm always happy, but I'm happy for a specific reason today. Over the weekend, I finally got my new laptop. Now, it's not about to start running Crysis anytime soon, but it's quite good for a mid-range laptop, especially now that I'm determined to do things right.

Step one was to install Linux; I use Debian more for historical reasons than anything else, but it works. This time, I didn't manage to accidentally corrupt the Windows' NTFS tree thereby requiring a reinstallation, nor did I try to netinstall off of a wireless connection. However, wireless has proved as daunting a task to solve as it did the last time I tried to do it. After much finessing and going everywhere for information, the irksome driver finally worked under ndiswrapper randomly today. Still requires a lot of manual intervention, but at least I know how to get it to work.

Also fixed today were two irksome problems. My resolution is now back up at the comfortable 1280x800 instead of an annoyingly fat 1024x768. The mouse is now configured to give me a middle click when I click both left and right buttons, a boon for the touchpad. Finally, the most irksome of my problems was the inability to use <Alt>-# to switch screens in irssi; even that's been fixed.

I don't think I did too badly on those two exams I took today that I should have been studying for instead of setting up my laptop, so that's reassuring. Mozilla's building fine right now (I'm waiting for libgklayout.so to link so I can laugh in its face!); scp'ing my data from my old machine seems to be progressing smoothly.

That said, I've still got things to do. I'm planning on finally getting around to requesting CVS privs today. fluxbox still demands some configuration, and laying down on the carpet for lack of desk space is beginning to aggravate my elbows. Hey, libgklayout.so finished building, and I didn't even notice it. There's also the Vista thing of sides to shore up. So far, the only good thing about Vista that I can find is that it comes with better games than XP. I haven't tested the games I've actually paid for yet, though...