August 2002 - Haskell

GHC bug,or Hugs feature?
by Arthur Baars 09 Aug '02

09 Aug '02

One of our students had a program that successfully compiled with Hugs but failed to type check in GHC. After adding some type signatures GHC could compile the program as well. The definitions below are not supposed to make sense, (f and g are just identity functions) but illustrate the situation in the students code: 1) the functions f and g are mutually recursive 2) there is a sort of (indirect) polymorphic recursion - the polymorphic types of f and g are instantiated to (Char->Char) f :: a -> a f a | True = a | g 'a' == 'a' = g a g :: a -> a g a | True = a | f 'a' == 'a' = f a The code above is successfully type-checked by both Hugs and GHC. If we remove either the type signature of f or the type signature of g, however, GHC fails to type check the code. Hugs still infers: f,g :: a->a When removing both type signatures GHC and Hugs both infer f,g::Char->Char, as expected. So this code failed in GHC: f :: a -> a f a | True = a | g 'a' == 'a' = g a g a | True = a | f 'a' == 'a' = f a Is this a bug in GHC or a feature of Hugs? I did not check what NHC and other Haskell compilers say about this. Can someone try? Cheers, Arthur

3 4

Text in Haskell pedantry
by George Russell 09 Aug '02

09 Aug '02

Ashley wrote [quote]No, a file is always a list of octets. Nothing else (ignoring metadata, forks etc.). [/quote] On MVS at least a file is a list of list of octets, because record boundaries are not handled by a "record boundary character" but by other means. There are still more horrible details of MVS access methods no-one here will want to know about. I think it would be more correct to say a file is always a list of C characters, since while this may not be true, at least every system in the foreseeable future is going to make it possible to pretend it is, at least for the sort of files people are likely to want to process using Haskell.

1 0

AlphaBeta (chess) in Haskell
by Mario Lang 09 Aug '02

09 Aug '02

Hello. Does anyone know of a AlphaBeta/Minimax module for haskell, and / or a chess module? I've found several references on the web, but no code. There are little examples in whyfp paper and in some other Haskell related paper, but both are far from complete. Also, I've found a reference to a working mate-problem solver which was used for HAT testing, but also, no code. Maybe someone has something already working which could be used as a basis. My plan is to provide a generic AlphaBeta module which could be used for different kind of two-player games. -- Thanks, Mario

3 2

RE: Text in Haskell: a second proposal
by Simon Marlow 09 Aug '02

09 Aug '02

Here's my take on the Unicode issue. Summary: unless there's a very good reason, I don't think we should decouple encoding/decoding from I/O, at least for the standard I/O library. Firstly, types. We already have all the necessary types: - Char, a Unicode code point - Word8, an octet - CChar, a type representing the C 'char' type The latter two are defined by the FFI addendum. Taking hGetChar as an example: hGetChar :: Handle -> IO Char This combines, IMO, two operations: reading some data from the file, and decoding enough of it to yield a Char. Underneath the hood, the Handle has a particular encoding associated with it. In GHC, currently we have two encodings, ISO8859 (aka binary, but we shouldn't use that term because the I/O library works in terms of Char) and MS-DOS text. We could easily extend the set of encodings to include UTF-8 and others. Seeking only works on Handles with a 1-1 correspondence between handle positions and characters (i.e. in the ISO encoding). Why combine I/O and {en,de}coding? Firstly, efficiency. Secondly, because it's convenient: if we were to express encodings as stream transformers, eg: decodeUTF8 :: [Word8] -> [Char] Then we would have to do all our I/O using lazy streams. You can't write hGetChar in terms of hGetWord8 using this: you need the non-stream version which in general looks something like decode :: Word8 -> DecodingState -> (Maybe [Char], DecodingState) for UTF-8 you can get away with something simpler, but AFAIK that's not true in general. You might want to use compression as an encoding, for example. So in general you need to store not only the DecodingState but also some cached characters between invocations of hGetChar. It's highly unlikely that automatic optimisations will be able to do anything useful with code written using the above interface, but we can write efficient code if the encoder/decoder can work on the I/O buffer directly. There's no reason why we shouldn't provide encoders/decoders as a separate library *as well*, and we should definitely also provide low-level I/O that works with Word8. Cheers, Simon

1 0

Re: Text in Haskell: a second proposal
by Ashley Yakeley 09 Aug '02

09 Aug '02

At 2002-08-09 01:19, Sven Moritz Hallberg wrote: >> Whether or not the old Char-based ones should be deprecated, or whatever, >> I don't know. > >I think any notion of treating the _raw_ contents of a file as Chars >must go, because it is simply incorrect. Right. Certainly we need to come up with _correct_ Word8-based file functions, and separately, text-encoding functions. After that we need to consider what is to be done with the existing "expedient" (and conceptually ugly) Char-based file functions. Should they be deprecated, or should we fix them with a particular encoding scheme such as UTF-8 or ISO 8859-1, or what? What about newline handling? etc. -- Ashley Yakeley, Seattle WA

1 0

Text in Haskell: a second proposal
by Ken Shan 09 Aug '02

09 Aug '02

Thanks to the discussion on this list, I now see that there are not four but five types to be distinguished in text processing: 1. Octets. 2. C "char". 3. Unicode code points. 4. Unicode code values, useful only for UTF-16, which is seldom used. 5. "What handles handle". The new entry in the list is the last one. Wolfgang Jeltsch mentioned "Stream_Element" in Ada 95, which I don't know specifically about but sounds like the same idea. I suggest that the following Haskell types be used for the five items above: 1. Word8 2. CChar 3. CodePoint 4. Word16 5. Char On most machines, Char will be a wrapper around Word8. (This contradicts the present language standard.) Let me elaborate. Files are funny because the information units they contain can be treated as both numbers and characters. Treating these units as numbers, we can convert them to say octets and use them to serialize higher-level data structures such as directed acyclic graphs, bitstrings, and Unicode text. Treating the same units as characters, we can concatenate text like "cat" does, strip out or replace selected characters like "tr" does, or compute the number of units in a stream like "wc -c" does. When we treat a file as containing characters (rather than numbers), we are in effect using a funky character encoding. This encoding is not UTF-8, not ISO-8859-1, and not ASCII. The specifics of this encoding depends on the machine. On mine, it supports 256 characters, mapped to the numbers 0 through 255. Between 0 and 127 is ASCII. From 128 to 255 are 128 additional characters -- let's call them "#128" through "#255". These characters are completely new. For example, "á" is not "#225", just like the integer "225" and the unique complete graph with 225 vertices are not the character "#225", either. Many people will never worry about the characters #128 through #255. This is akin to the fact that the characters "a" through "z" does not concern a program that reads two numbers (in decimal, textual form) from standard input, adds them up, and prints the result to standard out (again in decimal, textual form). As long as the set of characters handled by standard input and standard output includes the decimal digits, the period, and whitespace, the program will work. What do the five Haskell types proposed above mean for the practical programmer? The types in the Haskell IO library will not change; for instance, the only way to read an information unit from a handle is hGetChar :: Handle -> IO Char As I mentioned above, Char under the present proposal is not the type of a Unicode character, but the type of an information unit handled by handles, contra the current language standard. There should, however, be a basic guarantee on how big this information unit is; a reasonable one to make is that it contains at least 8 bits. In other words, ord (chr i) == i for all i such that 0 <= i <= 255. CChar will be a synonym for Char on most systems, just like CInt is a synonym for Int on most systems. I don't know if the sockets library should use Word8 or Char, but it should be one of the two. Now for the tricky issue of converting and defaulting and guessing encodings. In short: Encodings should be handled separately from files. Encoding conversion should be in a library separate from the I/O library. Without involving I/O, I should be able to write a program to answer the question "how many 3-byte sequences are valid UTF-8 text?". We will want stuff in the library like data Encoding text code = Encoding { encode :: [text] -> Maybe [code] , decode :: [code] -> Maybe [text] } utf8 :: Encoding CodePoint Word8 iso88591 :: Encoding CodePoint Word8 as well as char :: Encoding Char Word8 so that UTF-8 conversion from [Char] to [CodePoint] is (>>= decode utf8) . encode char :: [Char] -> Maybe [CodePoint] I am sure many complexities of character encodings are not considered here, but that is in part the point: I would like to see Char in the language standard dissociated from Unicode and return to the more abstract concept of an information unit. -- Edit this signature at http://www.digitas.harvard.edu/cgi-bin/ken/sig http://www.ethnologue.com/

2 1

Re: UTF-8 library
by George Russell 09 Aug '02

09 Aug '02

Axel wrote [snip] > I guess that is a good point, but due to backwards compatibility this is > propably not acceptable: The C interface of the FFI has the string > functions: > peekCString :: CString -> IO String > newCString :: String -> IO CString > > which should really be > > peekCString :: CString -> IO [Word8] > newCString :: [Word8] -> IO CString > > Unless that changes, there is really no point to give the encode and > decode functions that type. [snip] Such a change would be annoying, since I have already used peekCString and newCString quite a lot. (They are a great improvement on what we had before!) Converting CStrings to [Word8] is probably a bad idea anyway, since there is absolutely no reason to assume a C character will be only 8 bits long, and under some implementations it isn't. A better suggestion would be to provide ALTERNATIVE functions which got from CString/CStringLen and friends to [CChar], and make your UTF8 converters go between [CChar] and String. However we should not be forced to do this every time we want to construct a CString from a String (a very common need when calling C functions) so the existing functions should remain with their existing semantics.

4 7

Storable tuples and what is 'alignment'?
by Hal Daume III 09 Aug '02

09 Aug '02

Two things: 1) Why aren't there built in Storable instances for tuples? Writing my own is a pain and in keeping with the 'up to 7tuple' limit, I think built in instances would be nice 2) Could someone tell me what 'alignment' is and if this is the correct definition of a storable tuple: instance (Storable a, Storable b) => Storable (a,b) where sizeOf (a,b) = sizeOf a + sizeOf b alignment (a,b) = alignment a + alignment b peek ptr = do a <- peek (castPtr ptr) b <- peek (castPtr ptr `plusPtr` sizeOf a) return (a,b) poke ptr (a,b) = do poke (castPtr ptr) a poke (castPtr ptr `plusPtr` sizeOf a) b -- Hal Daume III "Computer science is no more about computers | hdaume(a)isi.edu than astronomy is about telescopes." -Dijkstra | www.isi.edu/~hdaume

5 4

8 bit characters?
by George Russell 09 Aug '02

09 Aug '02

Patryk Zadarnowski wrote [snip] > > > > But there's no guarantee that char is exactly 8 bits wide, is there? So > > it's appropriate to have separate types, CChar/CSChar/CUChar and > > Word8/Int8. > > No, and neither are there any guarantees about there being *any* > specific-width integer types at all! If you're worried about char not > being 8 bit, it's likely that the C compiler won't provide an 8 bit > integer, either, so Word8/Int8 doesn't solve the problem. [snip] There is no reason why a Haskell implementation should not provide Word8 or Int8 even if that is not supported by the underlying hardware. Actually I remember doing something like this for MLj (ML for Java). 8 bit words are not provided by the Java Virtual Machine (or at least, weren't then). However they can be provided fairly cheaply. I think I stored them as chars, but for operations treated them as shorts or ints with extra high junk bits. You need to put in extra work to clear the bits when comparing, or doing right shifts, or dividing, but not for addition, subtraction, multiplication, other bit operations, or loading or storing. So it's pretty cheap. Manual wrote [snip] > As I understand it, in ANSI C, the only freedom that an > implementation has in choosing a concrete representation for > "char" is to decide whether it is signed or unsigned. In > any case, it is going to be an 8 bit entity. [snip] This is false. Look at section 5.2.4.2.1 http://std.dkuug.dk/JTC1/SC22/WG14/www/docs/n843.htm This says only "Their implementation-defined values shall be equal or greater in magnitude (absolute value) to those shown, with the same sign." Thus CHAR_BIT must be at least 8. It may be greater than 8. Also see section 5.2.1. This is the C revision, but I am very sure it was the same with ANSI C.

2 1

Re: Text in Haskell: A PROPOSAL
by Ashley Yakeley 08 Aug '02

08 Aug '02

At 2002-08-08 02:54, Ketil Z. Malde wrote: >Would it be sufficient to have "raw" socket/file functions using >[Word8], This would certainly be a good thing. >and let the "standard" functions (e.g. readFile) convert to >[Char] according to current locale settings? The notion of "current locale settings" (including newline conventions) bothers me. I'd like my Haskell program to do the same thing regardless of which machine executes it -- particularly these days when files get shared around a lot. Do we really need "text mode" anymore? > With, perhaps, UTF-8 as a reasonable default? Perhaps it should _always_ be UTF-8? Or is that too slow in some cases? It certainly raises "seek" issues as one Char codepoint may be represented by several octets. >(And of course, en/decoding functions readily available for manual >use) Yes. -- Ashley Yakeley, Seattle WA

4 3