August 2002 - Haskell-i18n

RE: [Haskell-i18n] Code
by Sven Moritz Hallberg 23 Aug '02

23 Aug '02

On Fri, 2002-08-23 at 12:00, Simon Marlow wrote: > > > Yes, for the record, I have a working UTF-8 module now. It > > operates on > > > lists of Chars/Word8s but can also en/decode single > > characters. It is, > > > AFAICS, conformant to Unicode 3.2. > > > > It'd be nice to have a look, at least. Perhaps it could be put on the > > web somewhere, and links posted here (and/or linked from the Wiki)? > > An SF project is probably the best way to go. We could use > cvs.haskell.org, but new accounts on there have to go through the > maintainer (Jeff Lewis), and to make Jeff's life easier I think it would > be better all round if the project could be managed directly by those > involved. So go for SF to begin with, and move to cvs.haskell.org when > there is code that needs to be included with a compiler release. I'll be happy to share my code, just point me somewhere to upload it. If you create an SF project, I'm registered as pesco there. Sven Moritz

1 0

Re: [Haskell-i18n] Code
by Ashley Yakeley 23 Aug '02

23 Aug '02

At 2002-08-23 00:10, Ketil Z. Malde wrote: >Is it much hassle creating a SF project? Not really... it's filling out a form and waiting for a day or two. What license do the Haskell libraries use? -- Ashley Yakeley, Seattle WA

2 1

RE: [Haskell-i18n] Code
by Simon Marlow 23 Aug '02

23 Aug '02

> What license do the Haskell libraries use? The details are not precisely clear, but to all intents and purposes libraries/base is under a BSD-style license. That is the intention, anyway. libraries/network, libraries/haskell-src and libraries/haskell98 are also BSD-licensed. It is expected that we will have a package for GPL or LGPL licensed libraries at some point in the future. I'd argue strongly in favour of putting all code generated by the i18n project under nothing more restrictive than a BSD license, and preferably the same license as the Haskell report and FFI addendum. (I hope the eventual outcome of this discussion will be a Haskell 98 i18n addendum). Here's the Haskell report license: The authors intend this Report to belong to the entire Haskell community, and so we grant permission to copy and distribute it for any purpose, provided that it is reproduced in its entirety, including this Notice. Modified versions of this Report may also be copied and distributed for any purpose, provided that the modified version is clearly presented as such, and that it does not claim to be a definition of the Haskell 98 Language. Cheers, Simon

1 0

Re: [Haskell-i18n] Unicode in source
by Ashley Yakeley 23 Aug '02

23 Aug '02

At 2002-08-22 23:59, Ketil Z. Malde wrote: >Otherwise I fear that we spend an inordinate amount of time and >complexity of solving 'could-be' problems that in reality won't affect >anybody. Right. Anyway, there's always {;}... -- Ashley Yakeley, Seattle WA

1 0

RE: [Haskell-i18n] Code
by Simon Marlow 23 Aug '02

23 Aug '02

> Sven Moritz Hallberg <pesco(a)gmx.de> writes: > > > On Thu, 2002-08-22 at 02:39, Ashley Yakeley wrote: > > > I have some code for Unicode properties and the like I'd like to > > > contribute, I just need a place to put it. Is it worth creating a > > > SourceForge project for the internationalisation effort? > I think other > > > people have come up with stuff too, like UTF-8 decoders etc. > > Is it much hassle creating a SF project? > > > Yes, for the record, I have a working UTF-8 module now. It > operates on > > lists of Chars/Word8s but can also en/decode single > characters. It is, > > AFAICS, conformant to Unicode 3.2. > > It'd be nice to have a look, at least. Perhaps it could be put on the > web somewhere, and links posted here (and/or linked from the Wiki)? An SF project is probably the best way to go. We could use cvs.haskell.org, but new accounts on there have to go through the maintainer (Jeff Lewis), and to make Jeff's life easier I think it would be better all round if the project could be managed directly by those involved. So go for SF to begin with, and move to cvs.haskell.org when there is code that needs to be included with a compiler release. Cheers, Simon

1 0

RE: [Haskell-i18n] Unicode in source
by Ashley Yakeley 22 Aug '02

22 Aug '02

At 2002-08-22 16:12, Ashley Yakeley wrote: >Some characters are considered "half-width", see UTR #11. ><http://www.unicode.org/unicode/reports/tr11/> UAX #11. Whatever. -- Ashley Yakeley, Seattle WA

1 0

RE: [Haskell-i18n] Unicode in source
by Ashley Yakeley 22 Aug '02

22 Aug '02

At 2002-08-22 08:49, Sven Moritz Hallberg wrote: >Ashley, do your property tools include something that can handle >composition? I have decomposition functions: decomposeCanonical :: String -> String; decomposeCompatibility :: String -> String; See <http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/hbase/Source/HBa se/Text/Unicode.hs?rev=HEAD&content-type=text/plain>. Both fully decompose text according to the two kinds of decomposition: "canonical" and "compatibility". These don't do "canonical ordering" of things like multiple accent modifiers, so they don't count as normalisation forms D and KD. I might be able to write the code to do this, however. I don't have composition functions. And it won't help you for layout, since not all characters are composable as single codepoints. See UAX #15 for normalisation. <http://www.unicode.org/unicode/reports/tr15/> -- Ashley Yakeley, Seattle WA

1 0

Re: [Haskell-i18n] Unicode in source
by Marcin 'Qrczak' Kowalczyk 22 Aug '02

22 Aug '02

22 Aug 2002 18:18:28 +0100, Alastair Reid <alastair(a)reid-consulting-uk.ltd.uk> pisze: >>> columns) of a given wide character (wchar_t) in the current locale. > ^^^^^^^^^^^^^^^^^^^^^ > > Can you clarify whether this is because the mapping of wchar_ts onto > Unicode code points depends on the locale or whether the width of a > Unicode code point depends on the locale. Both. But some people argue that this is evil and wcwidth should have a generally accepted standard meaning, as long as wchar_t is Unicode, because for terminal emulators etc. it must agree on remote systems in order to be useful. http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c > [I'm trying to understand whether we can be sure that the number of > columns is portable so that different machines will apply the layout > rule consistently (assuming the translation into UTF8 is consistent).] In theory no. In practice maybe yes, provided that we use a fixed table instead of relying on system wcwidth. -- __("< Marcin Kowalczyk \__/ qrczak(a)knm.org.pl ^^ http://qrnik.knm.org.pl/~qrczak/

1 0

RE: [Haskell-i18n] Unicode in source
by Sven Moritz Hallberg 22 Aug '02

22 Aug '02

On Wed, 2002-08-21 at 12:02, Simon Marlow wrote: > > > Apparently, this isn't quite supported by GHC: > > > > Prelude> map Char.ord "\74\749\7490" > > [74,237,66] > > > > which is, of course, the values modulo 256. > > I think you've found a bug. [...] Oh, oops. :) > (aside: aren't there problems with Unicode not being a fixed-width > character set? Some characters are expected to combine with others to > form a glyph, there are multiple versions of some characters with > different widths, there are several widths of space, etc.) I think (...) these issues should not pose a problem. variable-width characters: Unicode specifically doesn't say anything about the glyph representation of the characters. So it is reasonable to assume there will be fixed-width unicode character sets. Remember that even our latin alphabet has characters of different width (i vs. w) which we just somehow manage to fit into glyphs of the same width. If one's editor would really use a variable-width font he'll already have the problem with ASCII. composition characters: I think we should interpret each character in the source as exactly one and leave any possible composition to the level of editing tools. The way I imagine the use of these composition characters is, for instance, as keyboard input to an editor which then composes them into a single char before writing anything to a file. I'd say this issue belongs to the domain of text processing. Regards, Sven Moritz

5 8

Drafting a unicode proposal
by Martin Norbäck 21 Aug '02

21 Aug '02

Could we use the Haskell wiki to draft a Unicode proposal? I know that traditionally this has been more formal committee work, but we could try. I've made a first page: http://www.haskell.org/wiki/wiki?UnicodeIssues There are limitations in this wiki, so it may be an idea to switch to a more featureful wiki. See my comments in http://www.haskell.org/wiki/wiki?ReportingProblems Regards, Martin -- Martin Norbäck d95mback(a)dtek.chalmers.se Kapplandsgatan 40 +46 (0)708 26 33 60 S-414 78 GÖTEBORG http://www.dtek.chalmers.se/~d95mback/ SWEDEN OpenPGP ID: 3FA8580B

3 3