Building mozilla with clang LTO

Ingredients:

• 1 (one) copy of clang, preferably on the new side
• 1 (one) source tree containing mozilla-central
• 4 (four) GiB (that's 2³⁰, not 10⁹) of memory
• Linux (these instructions not tested on OS X, so it may or may not work for you)

Instructions:

For the sake of example, I have clang installed to /usr/local; substitute as appropriate for your environment. Configure mozilla-central with the following mozconfig:

# -O4 tells clang to do LTO
ac_add_options --disable-debug --enable-optimize='-O4'
# Exclude this line, and your will use over 10x the amount of memory.
ac_add_options --disable-debug-symbols
# Build with clang
export CC=/usr/local/bin/clang
export CXX=/usr/local/bin/clang++
# These flags are necessary to get elfhack to compile
export LDFLAGS='-Wl,-z,norelro -flto'
# This flag is necessary to link properly
export RANLIB='ar -s --plugin /usr/local/lib/LLVMgold.so'

You may season with other configure options as desired. Configure and let build for a few hours. Admire the build process as most individual files get compiled much quicker, but be displeased when link times explode. Enjoy!

JS code coverage

My most recent adventure has been an extended foray into the depths of getting good JavaScript code coverage. The tool that I had been pointed to a few years ago was JSCoverage, which took the approach of "reserialize all JS code with implementation details," which doesn't work quite so well with the ever-shifting landscapes of Mozilla JS implementation. It also used SpiderMonkey's internal-only parser API, which, from experience, is a bloody edge to ride on. I've dithered on and off about using the reflection API to use this, but making sure that data can get extracted out again is a bit of a pain.

Now it's 2012 and the JS engine has better hooks that allow you to use debugging APIs without causing the code to come to a screeching halt in terms of performance (with the proliferation of setTimeout-based tests, drastically slowing down JS execution causes a surprising amount of tests to fail). So I started by using a modified version of Gregory Szorc's patch for development, but that immediately ran into several bugs with the JS debugging API. It also requires major contortions due to current limitations of the API. After complaining enough in #jsapi, I was pointed to some bytecode counter hooks enabled in the JS API. After finding and temporarily working my way around more bugs (I've been told that most of this stuff isn't necessary with compartment-per-global, but no one has been ripping this stuff out yet…), I produced a patch (see the previous linked bug) which hooks into xpconnect and automatically does code coverage for all JS run on the xpconnect runtime.

Since the pc counts just dumps out raw counts of hits, I need further processing to make it work; this comes in the form of post-processing script. There is also related issues that counts can't be generated for scripts it never sees, so there's also logic in my ever-growing code-coverage building script to compute all functions that get run. This is the most fragile part of the process right now, as many things can cause it to break. In addition, I ended up writing a partial replacement of lcov in python since I was getting extremely tired of its poor performance. This goes so far as actually reading the gcno and gcda files output by gcc itself, since gcov has problems reading one of them.

So the final result is posted now, and, like usual, I have a treemapified version as well. In that, I went ahead and implemented a feature which lets the URI specify the directory to start in, so you don't have to bear with several long script dialog warnings to view into a specific directory.