For realistic testing, I grabbed a real database and instrumented all of the accesses in real use cases (i.e., startup and shutdown). Since the real data is what I love to call my Profile from Hell™, there's enough data that I can get some reasonable statistical accuracy in timing. I'm also testing what is a path of note in Thunderbird startup, so I made sure to test cold startup, which, incidentally, is very annoying. A 1 second test takes several seconds to reload xpcshell.
The first performance results were very surprising to me. I expected that not touching the knobs would probably result in a mild performance regression (mork is fast by virtue of choosing the "space" option in the space/time tradeoff). What I wasn't expecting was something that was an order of magnitude slower. I eventually got off my butt and hooked up jprof to see what was going wrong. Unfortunately, I forgot that jprof's visualization of output results sucks, so I traipsed back to my very old post to pull down my script to turn the dumb HTML file into a more manageable dot file (which needed fixing, since the only commit to jprof in 3 years changed the output syntax on me).
Clearly, the output file implicates JS as being the hotpath. Since this is xpcshell and I had to hack the jprof initialization in, I decided to move the jprof initialization to the execution point to eliminate the effect of parsing my ~20K-line input file (I did say it loads a lot of data). When I last used jprof for serious performance tracking, I noted that the main regression appeared to be caused by XPCThrower::ThrowBadResult (about 60%, apparently). So here, the main regression appears to be... you guessed it, JS exceptions (the method names have changed in 3 years, though).
That is truly unexpected, since I'm not supposed to be actually changing any API. After looking back at the code and inspecting much more deeply, I found out that I actually was changing API. You see, the nsIMsgFolderCacheElement has two ways to access properties: as strings or as integers. Accessing integers clearly throws an error if the value isn't there; accessing strings also appears to throw an error. Assuming this to be the case, I explicitly threw an error in both cases in my new implementation. It turns out that the current implementation actually doesn't throw for strings (I'm not sure if it's intended to or not). Fixing that greatly improved the times. A brief summary of the real results is that mork is faster on startup, even on general property access, and slower on shutdown, only the last of which is not surprising.
In summary: throwing JS exceptions, at least via XPCOM, is very, very, very slow. If that code is a potential hotpath, just say no to exceptions.
4 comments:
LevelDB indeed seems like a great mork alternative. Specifically, the I/O patterns and built-in compaction are great.
The caveat is that most things written with mork do still assume everything is in memory once loads complete. Unless you force LevelDB to load everything into memory or fix the consuming code to issue asynchronous queries, this seems very likely to introduce significant performance regressions for those cases where everything is not already fully cached into memory by the OS. (And a performance test that does not take into account the potential for cold I/O is accordingly not really a representative performance comparison.)
Why are you surprised that Mork is faster (except on shutdown)?
>In summary: throwing JS exceptions, at least via XPCOM, is very, very, very slow.
Is this supposed to be the case? Worth filing a perf bug on this?
Neil: Mork being slower on shutdown was what was surprising, I guess I just wrote that sentence incorrectly
James: I don't think it's supposed to be quite so bad, but I goofed and ran these numbers on a debug build, and I haven't had time to run tests on an opt build to see if the speed penalty is still insanely high.
Post a Comment