Saturday, October 16, 2010

Updated code coverage

It's been about seven months since I last ran code coverage tools on Thunderbird, so I thought I would do it again. In these intervening seven months, Thunderbird has moved to libxul, and the mozmill tests have become more important, which means I get to change my methodology slightly.

Problematic caveats first, then. For various reasons, my build for code-coverage results runs on a 64-bit machine which I have to ssh to get to and on which I lack administrative privileges. Previously, the build setup caused gcov to fail to work properly for the mozilla-central code, causing my build scripts to require hacks to only cover the comm-central code. It seems that something in either the environment, the code, or the building of libxul caused it to be fixed.

On the other hand, the environment prevented me from running the mozmill tests correctly. On this computer, the user accounts are set up via LDAP, so the gtk initialization code tried to get user information from libc, which got it from NSS, which got it from LDAP, which tried to get it from another LDAP library. Unfortunately, it chose to use the directory/c-sdk ldap instead of the system ldap causing a crash. The only solution was to disable ldap, which required a few tricks to work correctly. Oh, and somehow the tests failed if I didn't enable libxul.

My last issue was with mozmill. I had a plan to use Xvfb for the display for mozmill (the intent being that I could automate mozmill tests on several computers overnight via screen). Turns out that it someone complained about needing Xrandr, so I got to run all of the mozmill tests via twice-forwarded X connections.

Anyways, the results are in. These are the results before mozmill tests were run, and these are the results including mozmill tests. By comparison, these are the results from my last run (which do not include mozmill tests). For completeness sake, 41 xpcshell-tests failed and 10 mozmill tests failed. I do not have the record of which ones failed, however.

Finally, here is the HD view of the code-coverage treemap results:

By comparison, the old code-coverage treemap results:

I hope you enjoy the results!