[ANN] text 0.7, fast Unicode text

Bryan O'Sullivan

15 Dec 2009 15 Dec '09

8:11 a.m.

The new 0.7 release of the text packagehttp://hackage.haskell.org/package/textadds support for Unicode I/O, using either the new locale-aware Handle code in 6.12 or a fallback on older releases. Details: http://www.serpentine.com/blog/2009/12/15/data-text-0-7-gains-io-support/

Attachments:

attachment.html (text/html — 409 bytes)

Show replies by date

Don Stewart

15 Dec 15 Dec

8:18 a.m.

bos:

...

The new 0.7 release of the text package adds support for Unicode I/O, using either the new locale-aware Handle code in 6.12 or a fallback on older releases.

Details: http://www.serpentine.com/blog/2009/12/15/ data-text-0-7-gains-io-support/

That was ridculously fast. -- Don

Ivan Lazar Miljenovic

11:15 a.m.

"Bryan O'Sullivan" writes:

...

using either the new locale-aware Handle code in 6.12 or a fallback on older releases.

How do you manage this? Any chance of abstracting something like this out (ala extensible-exceptions)? -- Ivan Lazar Miljenovic Ivan.Miljenovic@gmail.com IvanMiljenovic.wordpress.com

Bryan O'Sullivan

6:12 p.m.

On Tue, Dec 15, 2009 at 3:15 AM, Ivan Lazar Miljenovic < ivan.miljenovic@gmail.com> wrote:

...

"Bryan O'Sullivan" writes:

...
using either the new locale-aware Handle code in 6.12 or a fallback on older releases.

How do you manage this? Any chance of abstracting something like this out (ala extensible-exceptions)?

The fallback is very simple: on versions of GHC older than 6.12, you get UTF-8 codecs and your platform's native newline conversion, and that's it :-) There really isn't anything to abstract.

Ivan Lazar Miljenovic

10:33 p.m.

"Bryan O'Sullivan" writes:

...

The fallback is very simple: on versions of GHC older than 6.12, you get UTF-8 codecs and your platform's native newline conversion, and that's it :-) There really isn't anything to abstract.

Aha, you use Text's own decoding/encoding functions, so it isn't very useful for other libraries that don't use Text :( -- Ivan Lazar Miljenovic Ivan.Miljenovic@gmail.com IvanMiljenovic.wordpress.com

Bryan O'Sullivan

11:08 p.m.

On Tue, Dec 15, 2009 at 2:33 PM, Ivan Lazar Miljenovic < ivan.miljenovic@gmail.com> wrote:

...

Aha, you use Text's own decoding/encoding functions, so it isn't very useful for other libraries that don't use Text :(

Huh? I am confused. The point of the text I/O code is to support I/O on the Text type. What else would I be wanting there?

Ivan Lazar Miljenovic

11:18 p.m.

"Bryan O'Sullivan" writes:

...

Huh? I am confused. The point of the text I/O code is to support I/O on the Text type. What else would I be wanting there?

Yes, but I thought you were using utf8-string or something for it (that is, something not Text-specific). -- Ivan Lazar Miljenovic Ivan.Miljenovic@gmail.com IvanMiljenovic.wordpress.com

Ashley Yakeley

16 Dec 16 Dec

4:03 a.m.

Bryan O'Sullivan wrote:

...

The new 0.7 release of the text package http://hackage.haskell.org/package/text adds support for Unicode I/O, using either the new locale-aware Handle code in 6.12 or a fallback on older releases.

Details: http://www.serpentine.com/blog/2009/12/15/data-text-0-7-gains-io-support/

How do you pack Unicode codepoints into Word16? Do you use UTF-16? Supposing - s = "\x010A60\x010A61" -- Old South Arabian script t = pack s Is (unpack t) the same as s? What is (length t)? -- Ashley Yakeley

Bryan O'Sullivan

5:22 a.m.

On Tue, Dec 15, 2009 at 8:03 PM, Ashley Yakeley wrote:

...

How do you pack Unicode codepoints into Word16? Do you use UTF-16?

You don't get to find out as a user of the Text API, but yes.

...

Supposing -

s = "\x010A60\x010A61" -- Old South Arabian script t = pack s

Is (unpack t) the same as s? What is (length t)?

Yes, and 2. Any other answer would clearly be wrong (I'm lookin' at you, Java and C#) :-)

Cetin Sert

5:54 a.m.

http://corsis.sourceforge.net/img/csharp-6.png http://corsis.sourceforge.net/img/csharp-6.pngo__O!? 2009/12/16 Bryan O'Sullivan

...

On Tue, Dec 15, 2009 at 8:03 PM, Ashley Yakeley wrote:

...
How do you pack Unicode codepoints into Word16? Do you use UTF-16?

You don't get to find out as a user of the Text API, but yes.

...
Supposing -

s = "\x010A60\x010A61" -- Old South Arabian script t = pack s

Is (unpack t) the same as s? What is (length t)?

Yes, and 2. Any other answer would clearly be wrong (I'm lookin' at you, Java and C#) :-)

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe