On Wed, Aug 18, 2010 at 2:12 AM, John Meacham <john@repetae.net> wrote:
<ranty thing to follow>
That said, there is never a reason to use UTF-16, it is a vestigial
remanent from the brief period when it was thought 16 bits would be
enough for the unicode standard, any defense of it nowadays is after the
fact justification for having accidentally standardized on it back in
the day.

This is false. Text uses UTF-16 internally as early benchmarks indicated that it was faster. See Tom Harper's response to the other thread that was spawned of this thread by Ketil.

Text continues to be UTF-16 today because

    * no one has written a benchmark that shows that UTF-8 would be faster *for use in Data.Text*, and
    * no one has written a patch that converts Text to use UTF-8 internally.

I'm quite frustrated by this whole discussion; there's lots of talking, no coding, and only a little benchmarking (of web sites, not code). This will get us nowhere.

Cheers,
Johan