August 2002 - Haskell

Re: UTF-8 library
by Ashley Yakeley 08 Aug '02

08 Aug '02

At 2002-08-07 02:43, Axel Simon wrote: >But the point was that C might have different sized characters and that >these functions would still be portable even if the size of CChar changes. Text encoded with ISO 8859-1 or UTF-8 is octets. If you want to use CChars, you should then subsequently convert the Word8s into CChars. -- Ashley Yakeley, Seattle WA

6 10

Re: UTF-8 library
by Ashley Yakeley 08 Aug '02

08 Aug '02

At 2002-08-06 05:38, John Meacham wrote: >One major nit I have with this is the type signature of >decodeUTF8 and encodeUTF8 >a String should always represent a string of characters, not a byte >stream, the signatures should be > >decodeUTF8 :: String -> [Word8] >encodeUTF8 :: [Word8] -> String I think you mean encodeUTF8 :: String -> [Word8] decodeUTF8 :: [Word8] -> String ...or even decodeUTF8 :: [Word8] -> Maybe String It might also be useful to have stream functions. Decoding UTF8 octets is a kind of parsing, after all. But yes, you're right. A Char is a Unicode codepoint, nothing else, and certainly not a C 'char'. A C char is _usually_ a Word8 or an Int8, but not necessarily IIRC. I've always thought it a bit odd that the well-specified types Word8, Int8 etc. are hidden away in a package while the machine-dependent Int type, which I avoid in all my code, is in the Prelude. -- Ashley Yakeley, Seattle WA

2 1

Re: UTF-8 library
by Ashley Yakeley 08 Aug '02

08 Aug '02

At 2002-08-08 02:56, Patryk Zadarnowski wrote: >1990 ANSI/ISO C requires chars to be at least 8 bits wide in section >5.2.4.2.1. This extends to ANSI/ISO C++, which cites ISO C for its >definition of <limits.h>. Haven't got C'99 handy, but it'll be a >similar story - char is *at least* 8 bits wide. > >Hope that satisfies the pedants. But there's no guarantee that char is exactly 8 bits wide, is there? So it's appropriate to have separate types, CChar/CSChar/CUChar and Word8/Int8. I submit that interfaces to network handles should all use Word8, because the standards all speak unambiguously of "octets". For files it's debatable perhaps, but I strongly lean towards Word8 for any "binary" (i.e. direct) file access. CChar/CSChar/CUChar are for FFI to C and C++. -- Ashley Yakeley, Seattle WA

2 1

Re: UTF-8 library
by Ashley Yakeley 08 Aug '02

08 Aug '02

At 2002-08-08 03:16, Patryk Zadarnowski wrote: >Although, it's beyond me why the C standard should dictate anything >about Haskell, especially the arcane and obscure dark corners of it >such as the sizes of integer types. No Haskell compilers compile to >pure ISO C anyway. We need the types precisely for FFI to C, and IMO no other reason. Even then, FFI to functions using fixed-size types should use WordN/IntN types. The rest of Haskell shouldn't depend on it... although I suspect the "Int" type (which I avoid) corresponds to C "int" -- something far more likely to vary than "char". -- Ashley Yakeley, Seattle WA

1 0

Re: Text in Haskell: A PROPOSAL
by Ashley Yakeley 08 Aug '02

08 Aug '02

At 2002-08-07 11:05, Ken Shan wrote: >Let me clarify my understanding of this point a bit further. On the one >hand, GHC uses Char to mean a 32-bit value like a Unicode code point. No, GHC uses Char to mean a Unicode codepoint. These are not 32-bit. It only allows the 17 pages i.e. values in the range '\x0' to '\x10FFFF'. This is the Right Thing as per Unicode 3.1 and later (current is 3.2.0). >On the other hand, GHC uses Char to mean what files store and sockets >transmit and foreign functions process under the C type "char". Right, and this is a very bad idea. The file IO functions should be using Word8s >These two uses are inconsistent, and must be separated. I agree. At 2002-08-07 09:53, Ken Shan wrote: >I have a stake in using Haskell for international text processing: In >particular, I have been writing Haskell code that typeset international >text. Let me summarize what I think are the basic types of data that >need to be distinguished and processed *somehow* within a Haskell >program: > > (1) chars in C (perhaps distinguishing between unsigned, signed, and > default) > (2) 8-bit integers (i.e., signed) and words (i.e., unsigned) > (3) Unicode code values (16-bit) I think the whole 16-bit code value thing was dropped as of 3.1. UTF-16 uses 16-bit values to represent text just as UTF-8 uses 8-bit values. >The conflict in the present discussion arises from two desires: One, to >use Char as 1 above, for FFI convenience and quick-and-dirty code. Two, >to use Char as 4 above, for international text processing and conceptual >correctness. > >I believe that we need library functions to: > > (a) Convert between 1 and 2, or more generally, convert between 1 and > Integral types; > (b) Convert between 2 and 4, under a specified encoding such as > ISO-8859-1 or UTF-8; > (c) Convert between 3 and 4, according to the Unicode standard. You mean according to UTF-16. >My proposal involves the following types: > > (1) Represent char in C as Char, and zero-terminated strings (char*) > in C as CString. We already have the CChar type that means that. > (2) Represent 8-bit integers and words as Int8 and Word8. Agreed. > (3) Represent Unicode code values as Word16, or a new Haskell type > CodeValue. I don't think it's appropriate to have a new type. UTF-16 is a way of representing codepoints as 16-bit integers. The UTF-16 functions should use Word16. > (4) Represent Unicode code points as Word32, or a new Haskell type > CodePoint. We already have the Char type that means that -- in GHC, at least. String and character literals in programs should use it. -- Ashley Yakeley, Seattle WA

6 6

Re: Text in Haskell: A PROPOSAL
by Ashley Yakeley 08 Aug '02

08 Aug '02

At 2002-08-07 17:37, Sven Moritz Hallberg wrote: >Hm, what about encodeCharUTF16? Would that return Word16s? UTF-16 may represent a single Char as one or two Word16s. encodeCharUTF16 :: Char -> [Word16]; or encodeCharUTF16 :: Char -> (Word16,Maybe Word16); or encodeCharUTF16 :: Char -> Either Word16 (Word16,Word16); >Hrm. But then, how to write that to a file? Depends on what order you want the halves of each Word16. Unicode 3.0 defines four character encoding forms/schemes: UTF-8, UTF-16, UTF-16LE, and UTF-16BE. UTF-16 encodes as 16-bit units, the other three encode as 8-bit units. So you might have something like this: encodeUTF8 :: String -> [Word8]; encodeUTF16 :: String -> [Word16]; encodeUTF16LE :: String -> [Word8]; encodeUTF16BE :: String -> [Word8]; The authority here is Unicode Technical Report 17, which is part of the Unicode Standard. <http://www.unicode.org/unicode/reports/tr17/> But watch out... I've noticed a certain amount of incoherence in the Unicode standards, for instance Unicode 3.0 sec. 2.3 refers to UTF-16 one of four "Character Encoding Schemes" which is "an encoding form plus byte serialization" even though UTF-16 by itself doesn't include byte serialization. -- Ashley Yakeley, Seattle WA

1 0

Re: Text in Haskell: A PROPOSAL
by Ashley Yakeley 08 Aug '02

08 Aug '02

At 2002-08-07 21:26, Ken Shan wrote: >One thing I am concerned about is that getChar should be IO CChar, not >IO Char. No, it should be IO Word8, because strictly speaking a C char doesn't have to be an octet. CChar should be the machine-dependent type. Network handles should certainly be octets. And I think we can assume file handles are too... -- Ashley Yakeley, Seattle WA

1 0

RE: OT: haskell, .net, microsoft and vs
by Andrew Kennedy 08 Aug '02

08 Aug '02

> -----Original Message----- > From: Nigel Perry [mailto:NPerry@mac.com] > Sent: Tuesday, August 06, 2002 7:44 PM > To: Haskell Mailing List > Subject: Re: OT: haskell, .net, microsoft and vs ... > As part of Project 7, a Microsoft project with various language > developers during the development of .NET a number of languages added > VS support, ranging from simple syntax colouring and "makefile" > project support to full integration. Mondrian, the simple functional > scripting language, was one of these which I undertook, it's at the > simple end. However I did not also add support for Haskell (I did a > partial port [not speed optimised and many libraries missing] of GHC > to .NET using Mondrian as a backend). Other languages integrated > include Fortran (Salford), Cobol (Fujitsu) and Component Pascal (QUT). > Work-in-progress: VS integration of SML.NET (http://www.cl.cam.ac.uk/Research/TSG/SMLNET/) We use COM interop and a bootstrapped (subset) compiler to get syntax highlighting, incremental parsing and type inference, intellisense of SML and .NET library functions, and build support. Also some support for debugging (breakpoints, stack trace, locals). We hope to release this implementation at some point. - Andrew.

3 2

Fw: one C compiler
by Scott J. 07 Aug '02

07 Aug '02

Hi, I still have another silly question. I read that the numeric functions like "sinus" etc. were brought in by Cnu-code to Haskell (gnumeric..?) How can Haskell than be compiled by the Haskell compiler? Thx Scott ----- Original Message ----- From: "Scott J." <jscott(a)planetinternet.be> To: "Lars Lundgren" <d95lars(a)dtek.chalmers.se> Cc: <haskell(a)haskell.org> Sent: Wednesday, August 07, 2002 7:52 PM Subject: Re: one C compiler > Yes I want to make Code written in C available to Haskell. > > But I worry bow about mixing different C compilers. > > E.g. for Windows there is a retargettable C compiler made by Jacob Navia > available for a low price (it is even free). > > See http://www.q-software-solutions.com/ .And on the other hand Microsoft > bundles all his compilers in Visual Studio. net . I don't know if I can use > code from C# available to Haskell. > > Regards > > Scott > ----- Original Message ----- > From: "Lars Lundgren" <d95lars(a)dtek.chalmers.se> > To: "Scott J." <jscott(a)planetinternet.be> > Sent: Wednesday, August 07, 2002 10:14 AM > Subject: Re: one C compiler > > > > On Tue, 6 Aug 2002, Scott J. wrote: > > > > > Hi, > > > > > > I want to know if I have to use the same c compiler with which GHC was > > > ported to windows or may I use other C compilers for poting C to GHC? > > > > > > > It is very hard to understand what it is you are asking. GHC is a compiler > > written in Haskell (Not C). What is it you are trying to do? Make some > > code written in C available to some code written in Haskell, or what? > > > > > > /Lars L > > > > > > > > _______________________________________________ > Haskell mailing list > Haskell(a)haskell.org > http://www.haskell.org/mailman/listinfo/haskell >

1 0

Re: UTF-8 library
by Ashley Yakeley 07 Aug '02

07 Aug '02

At 2002-08-07 14:49, Alastair Reid wrote: >I think a proper working group is called for. I agree. -- Ashley Yakeley, Seattle WA

1 0