On Wed, Aug 18, 2010 at 10:12 AM, Michael Snoyman
While working on optimizing Hamlet I started playing around with the BigTable benchmark. I wrote two blog posts on the topic:
http://www.snoyman.com/blog/entry/bigtable-benchmarks/ http://www.snoyman.com/blog/entry/optimizing-hamlet/
Even though your benchmark didn't explicitly come up in this thread, Johan and I spent some time improving the performance of Text for it. As a result, in darcs HEAD, Text is faster than String, but slower than ByteString. I'd certainly like to close that gap more aggressively. If the other contributors to this thread took just one minute to craft a benchmark they cared about for every ten minutes they spend producing hot air, we'd be a lot better off.
It could be that these were flaws in text that are correctable and have nothing to do with UTF-16;
Since the internal representation used by text is completely opaque, we could of course change it if necessary, with no user-visible consequences. I've yet to see any data that suggests that it's specifically UTF-16 that is related to any performance shortfalls, however. Some people have been floating the idea of multiple text packages. I
personally would *not* want to go down that road, but it might be the only approach that allows top performance for all use cases.
I'd be surprised if that proves necessary.