February 2009 - Glasgow-haskell-users

Re: Ready for testing: Unicode support for Handle I/O
by Paolo Losi 04 Feb '09

04 Feb '09

Max Vasin wrote: > Wouldn't it be more correct to separate binary IO, which > return [Word8] (or ByteString) and text IO which return > [Char] and deal with text encoding? IIRC that was done in > Bulat Ziganshin's streams library. That's exactly what I meant. Text IO could be then implemented on top of binary IO. Would it be possible to envision a solution that enables composition of IO low level strategies vs binary <-> text conversion strategies? Pao

1 0

Ready for testing: Unicode support for Handle I/O
by Simon Marlow 04 Feb '09

04 Feb '09

I've been working on adding proper Unicode support to Handle I/O in GHC, and I finally have something that's ready for testing. I've put a patchset here: http://www.haskell.org/~simonmar/base-unicode.tar.gz That is a set of patches against a GHC repo tree: unpack the tarball, and say 'sh apply /path/to/ghc/repo' to apply all the patches. Then clean your tree and build it from scratch (or if you're using the new GHC build system, just say 'make' ;-). It should validate, bar one or two minor failures. Oh, it doesn't work on Windows yet. That's the major thing left to do. If anyone else felt like tackling this I'd be delighted: all you have to do is implement a Win32 equivalent of the module GHC.IO.Encoding.Iconv (see below), everything else should work unchanged. Depending on whether any further changes are required, I may amend-record some of these patches, so treat them as temporary patches for testing only. Below is what will be the patch description in the patch for libraries/base. Comments/discussion please! Cheers, Simon Unicode-aware Handles ~~~~~~~~~~~~~~~~~~~~~ This is a significant restructuring of the Handle implementation with the primary goal of supporting Unicode character encodings. The only change to the existing behaviour is that by default, text IO is done in the prevailing encoding of the system. Handles created by openBinaryFile use the Latin-1 encoding, as do Handles placed in binary mode using hSetBinaryMode. We provide a way to change the encoding for an existing Handle: hSetEncoding :: Handle -> TextEncoding -> IO () and various encodings: latin1, utf8, utf16, utf16le, utf16be, utf32, utf32le, utf32be, localeEncoding, and a way to lookup other encodings: mkTextEncoding :: String -> IO TextEncoding (it's system-dependent whether the requested encoding will be available). Currently hSetEncoding is availble from GHC.IO.Handle, and the encodings are available from GHC.IO.Encoding. We may want to export these from somewhere more permanent; that's something for a library proposal. Thanks to suggestions from Duncan Coutts, it's possible to call hSetEncoding even on buffered read Handles, and the right thing happens. So we can read from text streams that include multiple encodings, such as an HTTP response or email message, without having to turn buffering off (though there is a penalty for switching encodings on a buffered Handle, as the IO system has to do some re-decoding to figure out where it should start reading from again). If there is a decoding error, it is reported when an attempt is made to read the offending character from the Handle, as you would expect. Performance is about 30% slower on "hGetContents >>= putStr" than before. I've profiled it, and about 25% of this is in doing the actual encoding/decoding, the rest is accounted for by the fact that we're shuffling around 32-bit chars rather than bytes in the Handle buffer, so there's not much we can do to improve this. IO library restructuring ~~~~~~~~~~~~~~~~~~~~~~~~ The major change here is that the implementation of the Handle operations is separated from the underlying IO device, using type classes. File descriptors are just one IO provider; I have also implemented memory-mapped files (good for random-access read/write) and a Handle that pipes output to a Chan (useful for testing code that writes to a Handle). New kinds of Handle can be implemented outside the base package, for instance someone could write bytestringToHandle. A Handle is made using mkFileHandle: -- | makes a new 'Handle' mkFileHandle :: (IODevice dev, BufferedIO dev, Typeable dev) => dev -- ^ the underlying IO device, which must support -- 'IODevice', 'BufferedIO' and 'Typeable' -> FilePath -- ^ a string describing the 'Handle', e.g. the file -- path for a file. Used in error messages. -> IOMode -- ^ The mode in which the 'Handle' is to be used -> Maybe TextEncoding -- ^ text encoding to use, if any -> IO Handle This also means that someone can write a completely new IO implementation on Windows based on native Win32 HANDLEs, and distribute it as a separate package (I really hope somebody does this!). This restructuring isn't as radical as previous designs. I haven't made any attempt to make a separate binary I/O layer, for example (although hGetBuf/hPutBuf do bypass the text encoding). The main goal here was to get Unicode support in, and to allow others to experiment with making new kinds of Handle. We could split up the layers further later. API changes and Module structure ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ NB. GHC.IOBase and GHC.Handle are now DEPRECATED (they are still present, but are just re-exporting things from other modules now). For 6.12 we'll want to bump base to version 5 and add a base4-compat. For now I'm using #if __GLASGOW_HASKEL__ >= 611 to avoid deprecated warnings. I split modules into smaller parts in many places. For example, we now have GHC.IORef, GHC.MVar and GHC.IOArray containing the implementations of IORef, MVar and IOArray respectively. This was necessary for untangling dependencies, but it also makes things easier to follow. The new module structurue for the IO-relatied parts of the base package is: GHC.IO Implementation of the IO monad; unsafe*; throw/catch GHC.IO.IOMode The IOMode type GHC.IO.Buffer Buffers and operations on them GHC.IO.Device The IODevice and RawIO classes. GHC.IO.BufferedIO The BufferedIO class. GHC.IO.FD The FD type, with instances of IODevice, RawIO and BufferedIO. GHC.IO.Exception IO-related Exceptions GHC.IO.Encoding The TextEncoding type; built-in TextEncodings; mkTextEncoding GHC.IO.Encoding.Types GHC.IO.Encoding.Iconv Implementation internals for GHC.IO.Encoding GHC.IO.Handle The main API for GHC's Handle implementation, provides all the Handle operations + mkFileHandle + hSetEncoding. GHC.IO.Handle.Types GHC.IO.Handle.Internals GHC.IO.Handle.Text Implementation of Handles and operations. GHC.IO.Handle.FD Parts of the Handle API implemented by file-descriptors: openFile, stdin, stdout, stderr, fdToHandle etc.

4 6

Segmentation fault trying to build ghc 6.10.1 using macports, Mac OS X 10.5, PPC
by David Menendez 03 Feb '09

03 Feb '09

I'm trying to upgrade GHC to 6.10.1 using macports on a PowerBook G4 running OS X 10.5.5. From what I can tell, I'm getting a segmentation fault from cabal-bin. This is possibly related to <http://trac.macports.org/ticket/15142> and <http://hackage.haskell.org/trac/ghc/ticket/2380>. cd extensible-exceptions && /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_ghc/work/ghc-6.10.1/libraries/cabal-bin /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_ghc/work/ghc-bootstrap/bin/ghc /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_ghc/work/ghc-6.10.1/libraries/bootstrapping.conf configure --distpref=dist-bootstrapping --with-compiler=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_ghc/work/ghc-bootstrap/bin/ghc --with-hc-pkg=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_ghc/work/ghc-bootstrap/bin/ghc-pkg --package-db=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_ghc/work/ghc-6.10.1/libraries/bootstrapping.conf.tmp /bin/sh: line 1: 19203 Segmentation fault /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_ghc/work/ghc-6.10.1/libraries/cabal-bin /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_ghc/work/ghc-bootstrap/bin/ghc /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_ghc/work/ghc-6.10.1/libraries/bootstrapping.conf configure --distpref=dist-bootstrapping --with-compiler=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_ghc/work/ghc-bootstrap/bin/ghc --with-hc-pkg=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_ghc/work/ghc-bootstrap/bin/ghc-pkg --package-db=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_ghc/work/ghc-6.10.1/libraries/bootstrapping.conf.tmp make[1]: *** [bootstrapping.conf] Error 139 make: *** [stage1] Error 2 -- Dave Menendez <dave(a)zednenem.com> <http://www.eyrie.org/~zednenem/>

3 3

ST monad and monad tranformers
by Tyson Whitehead 03 Feb '09

03 Feb '09

I have a situation in which I believe I need a parameterizable version of the strict ST monad. My computation type is "StateT s' (STT s (ErrorT e m)) a" (i.e., fails or succeeds and has an internal state involving a state thread). The STT type above is a version of ST like the ReaderT, StateT, etc. types. newtype STT s m a = STT ( State# s -> m (STTBox s a) ) data STTBox s a = STTBox {-#UNPACK#-} !(State# s) {-#UNPACK#-} !a (I'm guessing on the UNPACK paragmas here) with runSTT :: (Monad m) => (forall s. STT s m a) -> m a runSTT m = case m of STT m' -> do STTBox _ x <- m' realWorld# return x (writing this as "runSTT (STT m') = ..." doesn't typecheck with ghc 6.8.2) instance Monad m => Monad (STT s m) where return x = STT $ \s -> return $ STTBox s x (STT m) >>= k = STT $ \s -> do STTBox s' x <- m s case k x of STT k' -> k' s' plus all the assorted instances for Functor, MonadPlus, MonadFix, MonadTrans, MonadReader, MonadState, etc. For example, instance MonadWriter w m => MonadWriter w (STT s m) where tell = lift . tell listen (STT m) = STT $ \s -> do (STTBox s' x,w) <- listen $ m s return $ STTBox s' (x,w) pass (STT m) = STT $ \s -> pass $ do STTBox s' (x,f) <- m s return (STTBox s' x,f) I was looking for any comments, wondering if there is a reason for this not existing in the library already, and what I should do in terms of paragmas and such for speed? I see the GHC-ST file has a mix of INLINE and NOINLINE. http://www.haskell.org/ghc/dist/current/docs/libraries/base/src/GHC-ST.html In particular, return, >>=, >>, and runST are marked INLINE, but there is a "regrettably delicate" comment that goes with the runST method. Also, what about the Functor, MonadPlus, MonadFix, MonadTrans, MonadReader, etc. methods? Thanks! -Tyson PS: I would be happy to provide the whole works to be added to the library if it is something that should be there.

3 5

Surprised to type in (1+n) as a type index!
by Ahn, Ki Yung 03 Feb '09

03 Feb '09

I was writing a document in lhs form and typed in something like this. > data Exp n where > Atom :: Exp () > List :: [Exp n] -> Exp (1+n) I wasn't expecting this to actually load up on GHCi, but when I load this up with the magical -fglasgow-exts option, it just worked !!! *Main> :t Atom Atom :: Exp () *Main> :t List [Atom, Atom] List [Atom, Atom] :: forall (+ :: * -> * -> *). Exp (+ GHC.Generics.Unit ()) It seems GHC is treating + as just a type variable for binary type constructor of kind * -> * -> *. *Main> :t List [Atom, Atom] List [Atom, Atom] :: forall (+ :: * -> * -> *). Exp (+ GHC.Generics.Unit ()) *Main> :t List [List [Atom, Atom], List []] List [List [Atom, Atom], List []] :: forall (+ :: * -> * -> *) (+1 :: * -> * -> *). Exp (+1 GHC.Generics.Unit (+ GHC.Generics.Unit ())) *Main> :t List [List [List [Atom, Atom], List []]] List [List [List [Atom, Atom], List []]] :: forall (+ :: * -> * -> *) (+1 :: * -> * -> *) (+2 :: * -> * -> *). Exp (+2 GHC.Generics.Unit (+1 GHC.Generics.Unit (+ GHC.Generics.Unit ()))) But, what is this GHC.Generics.Unit thing and where is this documented? Thanks, Ahn, Ki Yung

1 0

Core questions
by Matthijs Kooijman 02 Feb '09

02 Feb '09

(I've previously sent this mail to haskell-cafe, but I guess this list is more appropriate) Hi all, for my Master's thesis, I'm looking into functional hardware descriptions, and in particular to translate haskell code into VHDL that can be programmed into an FPGA. For this, I'm using the GHC API to load the haskell source and give me (simplified) core representation. I then walk over the core tree and translate it to VHDL. On the whole the GHC API has been very useful to me. It took me some time to get used to all the (deeply) nested (algebraic) types (I didn't have any previous haskell experience when I started a month or so ago), but things are working out by now. However, there are two issues bothering me still. The first is that the Core types (in particular CoreExpr) are not instances of Show. They are instances of Outputable, which allows them to be pretty printed. However, this pretty printing is good to view the structure of the expression that the CoreExpr represents, but doesn't show the structure of the CoreExpr itself. For example, tuple construction is printed simply as (a, b), while the actual core expression is a nested application of two types, and a and b to the GHC.Tuple.(,) function (or datacon?). Also, the exact constructors used are not quite clear, which made it harder to work with the Core language for me. Instead of looking at the structure of the CoreExpr and write the appropriate patterns, I had to resort to a lot of trial and error. I tried deriving show for CoreExpr myself, but that required me to derive Show for a dozen other types and resulted in type errors beyond my understanding. Is there any compelling reason that CoreExprs are not instances of Show? My second question concerns typle construction. Since tuple types are not primitive types, but dependent types defined in various places (GHC.Tuple and Base IIRC), code working with tuples is not fundamentally different from code working with other (user defined) dependent types and thus not trivial to recognize. I've found that there are some predicate functions that can tell me if a type is a tuple type, but I've had no such luck for the actual tuple construction. In particular, when an expression creates a tuple of two Ints (a, b), this is represented in Core as the application of the element types and the element values to the (,) function (or data constructor? Not sure about this, in Core it's just a Var and the isDataConName [IIRC] predicate crashes when run on the Var). For now, I've manually matched the Var's name to GHC.Tuple.(,) and removed all type arguments to get at the actual values in the tuple. This is of course completely non-portable, to tuples with more than two elements, or unboxed tuples, etc. Is there some predicate function that can do this for me in a more portable way? Gr. Matthijs

6 6