- Libraries - Haskell.org

memory useage of data types in the time package
by Duncan Coutts 23 May '10

23 May '10

Hi Ashley (and other people interested in the time lib), I wonder if we could look more closely at the in-memory representations of the data types in the time library to see if we could make them a bit smaller. There are various sorts of programs that deal with a large quantity of data and often that includes some date/time data. It's a great shame that programs dealing with lots of time data might have to avoid using the time package and revert to things like newtype Seconds = Seconds Int32 simply because they take less space, and can be unpacked into other data structures. Looking at the representations in the time package, I think there are a number of places where the size could be reduced without affecting the behaviour or significantly affecting the range of values that can be represented. For example, data TimeOfDay = TimeOfDay { todHour :: Int, todMin :: Int, todSec :: Pico } This uses 40 bytes (or 80 bytes on 64bit) but it could be brought down to just 20 bytes (or 24 on 64bit). We could use strict fields here for the Ints and build the package with -funbox-strict-fields. This would save 4 words. The most expensive one here is Pico which uses an Integer underneath and thus cannot be unboxed. We could save a further two words if Pico were based on Int64 instead of Integer. In principle, the hour and minutes could be Int16 and perhaps in future ghc might be able to pack them into a single word inside the TimeOfDay record. If we find cannot change Pico (since it's in the base package), then we could change the time package to use some equivalent fixed point type. newtype Day = ModifiedJulianDay {toModifiedJulianDay :: Integer} This could use Int64 without significantly affecting the range. It would still let us represent dates trillions of years into the future or past. (Using Int32 would allow dates a mere few million years in the future). Again, the point of using Int64 is to save an indirection and to allow the type to be unboxed into other constructors such as: data UTCTime = UTCTime { utctDay :: Day, utctDayTime :: DiffTime } This takes 7 words, 28 or 56 bytes. It could be reduced to 20 or 24 bytes. This one is especially useful to make smaller since it is used as a timestamp in many applications, so there tend to be a lot of them. newtype DiffTime = MkDiffTime Pico Again, Pico based on Int64 rather than Integer would save space, save an indirection and allow further unboxing. In general, do any of the date/time record types need to have lazy fields? My guess is that they do not. Duncan

7 8

unix package patch to add support for setting effective uid/gid
by Eric Mertens 17 May '10

17 May '10

Hello, I believe that it would be useful for the unix package to support setting the effective user-id and group-id of a process. By setting the effective id instead of the real id you can toggle between real and set-user-id. It seems like a simple oversight that these functions have been left out of the unix package. I have attached the darcs patch for these function. This patch does not explicitly bump the version number as I thought it might be integrated with other patches before a new release (That could be wrong). -- Eric Mertens <emertens(a)galois.com> Galois, Inc.

2 1

turning ExitCode into an IO exception
by Henning Thielemann 11 May '10

11 May '10

Is there a prefered way to convert an ExitCode to an IO exception? When running a sequence of IO actions, it is certainly best to use the same way of showing failure for all of them. However running shell commands with functions from the 'process' package shows failure by ExitCode which is different from almost all other IO actions in 'base'.

2 1

Haskell 2010 libraries
by Simon Marlow 09 May '10

09 May '10

Hi Folks, I'm editing the Haskell 2010 report right now, and trying to decide what to do about the libraries. During the Haskell 2010 process the committee agreed that the libraries in the report should be updated, using the current hierarchical names, adding new functionality from the current base package, and dropping some of the H'98 library modules that now have better alternatives. In Haskell 2010 we're also adding the FFI modules. The FFI addendum used non-hierarchical names (CForeign, MarshalAlloc etc.) but these are usually known by their hierarchical names nowadays: e.g. Foreign.C, Foreign.Marshal.Alloc. It would seem strange to add the non-hierarchical names to the Haskell language report. So this is all fine from the point of view of the Haskell report - I can certainly update the report to use the hierarchical module names, but that presents us with one or two problems in the implementation. The obvious thing to do would be to make a haskell2010 package that re-exports the appropriate modules from base, providing a fixed API that people can depend on when they write Haskell 2010 code. However, what happens when someone wants to write some code that uses Haskell 2010 libraries, but also uses something else from base, say Control.Concurrent? The modules from haskell2010 overlap with those from base, so all the imports of Haskell 2010 modules will be ambiguous. The Prelude is a bit of a thorny issue too: currently it is in base, but we would have to move it to haskell2010. Bear in mind these goals: we want to a. support writing code that is Haskell 2010 only: it only uses Haskell 2010 language features and modules. b. not break existing code as far as possible c. whatever we do should extend smoothly when H'2011 makes further changes, and so on. Here are some non-options: 1. Not have a haskell2010 package. We lose (a) above, and we lose the ability to add or change the API for these modules, in base, since they have to conform to the H'2010 spec. If H'2011 makes any changes to these modules, we're really stuck. 2. As described above: you can either use haskell2010, or base, but not both. It would be painful to use haskell2010 in GHCi, none of the base modules would be available. Here are some options: 3. allow packages to shadow each other, so haskell2010 shadows base. This is a tantalising possibility, but I don't have any idea what it would look like, e.g. should the client or the package provider specify shadowing? 4. Provide a haskell2010 package and a base2010 package that re-exports all of base except the modules that overlap with haskell2010. You can either use haskell2010, haskell2010+base2010, or base. This is a bit like (1), but avoids the need for shadowing by using package re-exports, on the other hand confusion could well arise due to the strange base2010 package, and some people would surely try to use haskell2010 + base and run into difficulties. 5. Not have a haskell2010 package, but have the report say that implementations are allowed to add things to the standard libraries. Thoughts? Better ideas? Cheers, Simon

13 20

ANNOUNCE: network-protocol-xmpp 0.3
by John Millikin 07 May '10

07 May '10

XMPP[1] is a protocol for asynchronous communication via long-lived XML documents. It is most famous for underlying the Jabber and Wave protocols, but is also used for a variety of cases where security and extensibility are useful. My library, network-protocol-xmpp[2], is an implementation of most of RFC 3920 and a bit of RFC 3921. It supports opening client-to-server and component-to-server sessions, which is useful for implementing XMPP-based clients. This library's interface is very simple: clients may start a session with 'runClient' or 'runComponent', send and receive stanzas, and resume stored sessions. Later, I intend to add additional modules to support features such as MUC or file transmission, but for now I'd like to make sure the core library works. There's an example XMPP client, echo.hs[3], in the Darcs repository. Server-to-server sessions and accepting connection requests are not yet supported, so it's not usable as the foundation of an XMPP server yet. However, given that XMPP prefers to place features on the servers rather than clients, it's likely that writing an XMPP server in Haskell would take a *lot* of effort anyway. You're probably better off just using ejabberd or OpenFire. My thanks go to Stephan Maka, who let me know that this library actually has users, and who contributed most of the Component connection implementation. ---------- Additionally, I'm trying an experiment intended to break the gridlock of Haskell XML libraries. Currently, there are multiple options for XML handling (HXT, HaXml, libxml, tagsoup, xml, xml-basic) but each defines their own data types. This makes it very difficult to mix-and-match -- I can't easily use one library's parser with another's tree manipulation combinators. Simple use cases, such as using HXT for serializing, drag in dozens of dependencies on libraries like HTTP and QuickCheck. A library's poor data type definitions (such as HaXml's lack of support for namespaces) means that an entire set of potentially useful operations is unusable. And a library's choice propagates! In n-p-xmpp 0.2, which used HXT, clients were forced to figure out arrows before they could perform even rudimentary inspection of received stanzas. To fix this, I've created the xml-types[4] package. Its only dependency is the "text" library, and it defines full types for most XML structures[5]. My hope is that other libraries will replace their own types with these, so that (for example) I can parse events with libxml, manipulate them with HXT's arrow API, and write them back out using a custom serialiser without having to jump through type conversion hoops. It might take some time before any major XML libraries are transitioned to these types, but maybe in a year or two there'll be enough to be useful. [1] http://en.wikipedia.org/wiki/Extensible_Messaging_and_Presence_Protocol [2] http://hackage.haskell.org/package/network-protocol-xmpp [3] http://patch-tag.com/r/jmillikin/xmpp/snapshot/current/content/pretty/examp… [4] http://hackage.haskell.org/package/xml-types [5] I haven't included types for inner entity declarations in DTDs, out of laziness, because does anybody actually *use* those?

2 1

darcs patch: Correct haddock formatting.
by Adam Vogt 06 May '10

06 May '10

1 patch for repository http://darcs.haskell.org/packages/base: Thu Apr 22 22:21:03 EDT 2010 Adam Vogt <vogt.adam(a)gmail.com> * Correct haddock formatting.

2 1

Proposal: add System.IO.hGetBufSome
by Simon Marlow 06 May '10

06 May '10

http://hackage.haskell.org/trac/ghc/ticket/4046 I'd like to add the following function to `System.IO`, alongside `hGetBuf` and `hGetBufNonBlocking`: {{{ -- | 'hGetBufSome' @hdl buf count@ reads data from the handle @hdl@ -- into the buffer @buf@. If there is any data available to read, -- then 'hGetBufSome' returns it immediately; it only blocks if there -- is no data to be read. -- -- It returns the number of bytes actually read. This may be zero if -- EOF was reached before any data was read (or if @count@ is zero). -- -- 'hGetBufSome' never raises an EOF exception, instead it returns a value -- smaller than @count@. -- -- If the handle is a pipe or socket, and the writing end -- is closed, 'hGetBufSome' will behave as if EOF was reached. -- -- 'hGetBufSome' ignores the prevailing 'TextEncoding' and 'NewlineMode' -- on the 'Handle', and reads bytes directly. }}} We need it to properly fix #4041 and #3808. It corresponds more closely to the semantics of Unix `read()`, and I've found this to be the most useful low-level API for reading in practice. In the long term we'd like to provide a better binary I/O API, and we might even get around to doing that before 6.14.1, but if not we'll need this. Deadline: 2 weeks. Cheers, Simon

4 3

Asynchronous exception wormholes kill modularity
by Bas van Dijk 01 May '10

01 May '10

Dear all, (sorry for this long mail) When programming in the IO monad you have to be careful about asynchronous exceptions. These nasty little worms can be thrown to you at any point in your IO computation. You have to be extra careful when doing, what must be, an atomic transaction like: do old <- takeMVar m new <- f old `onException` putMVar m old putMVar m new If an asynchronous exception is thrown to you right after you have taken your MVar the putMVar will not be executed anymore and will leave your MVar in the empty state. This can possibly lead to dead-lock. The standard solution for this is to use a function like modifyMVar_: modifyMVar_ :: MVar a -> (a -> IO a) -> IO () modifyMVar_ m io = block $ do a <- takeMVar m a' <- unblock (io a) `onException` putMVar m a putMVar m a' As you can see this will first block asynchronous exceptions before taking the MVar. It is usually better to be in the blocked state as short as possible to ensure that asynchronous exceptions can be handled as soon as possible. This is why modifyMVar_ unblocks the the inner (io a). However now comes the problem I would like to talk about. What if I want to use modifyMVar_ as part of a bigger atomic transaction. As in: block $ do ... modifyMVar_ m f ... >From a quick glanse at this code it looks like asynchronous exceptions can't be thrown to this transaction because we block them. However the unblock in modifyMVar_ opens an asynchronous exception "wormhole" right into our blocked computation. This destroys modularity. Besides modifyMVar_ the following functions suffer the same problem: * Control.Exception.finally/bracket/bracketOnError * Control.Concurrent.MVar.withMVar/modifyMVar_/modifyMVar * Foreign.Marshal.Pool.withPool We can solve it by introducing two handy functions 'blockedApply' and 'blockedApply2' and wrapping each of the operations in them: > import Control.Exception > import Control.Concurrent.MVar > import Foreign.Marshal.Pool > import GHC.IO ( catchAny ) > blockedApply :: IO a -> (IO a -> IO b) -> IO b > blockedApply a f = do > b <- blocked > if b > then f a > else block $ f $ unblock a > blockedApply2 :: (c -> IO a) -> ((c -> IO a) -> IO b) -> IO b > blockedApply2 g f = do > b <- blocked > if b > then f g > else block $ f $ unblock . g Control.Exception: > finally :: IO a -> IO b -> IO a > a `finally` sequel = blockedApply a $ \a' -> do > r <- a' `onException` sequel > _ <- sequel > return r > bracket :: IO a-> (a -> IO b) -> (a -> IO c) -> IO c > bracket before after thing = blockedApply2 thing $ \thing' -> do > a <- before > r <- thing' a `onException` after a > _ <- after a > return r > bracketOnError :: IO a -> (a -> IO b) -> (a -> IO c) -> IO c > bracketOnError before after thing = blockedApply2 thing $ \thing' -> do > a <- before > thing' a `onException` after a Control.Concurrent.MVar: > withMVar :: MVar a -> (a -> IO b) -> IO b > withMVar m io = blockedApply2 io $ \io' -> do > a <- takeMVar m > b <- io' a `onException` putMVar m a > putMVar m a > return b > modifyMVar_ :: MVar a -> (a -> IO a) -> IO () > modifyMVar_ m io = blockedApply2 io $ \io' -> do > a <- takeMVar m > a' <- io' a `onException` putMVar m a > putMVar m a' > modifyMVar :: MVar a -> (a -> IO (a,b)) -> IO b > modifyMVar m io = blockedApply2 io $ \io' -> do > a <- takeMVar m > (a',b) <- io' a `onException` putMVar m a > putMVar m a' > return b Foreign.Marshal.Pool: > withPool :: (Pool -> IO b) -> IO b > withPool act = blockedApply2 act $ \act' -> do > pool <- newPool > val <- catchAny > (act' pool) > (\e -> do freePool pool; throw e) > freePool pool > return val I'm not proposing to make this change (yet) because I first would like to have some discussion on this. Thanks for reading this rather long mail, Bas

12 47

Add flipped fmap
by Bas van Dijk 01 May '10

01 May '10

Hello, I would like to propose adding a flipped variant of <$> to Data.Functor: (<$$>) ∷ Functor f ⇒ f α → (α → β) → f β (<$$>) = flip (<$>) Please see the ticket for motivation and patch: http://hackage.haskell.org/trac/ghc/ticket/3962 Discussion period: 2 weeks. regards, Bas

13 40

Deprecating haskell98 module aliases
by Gwern Branwen 29 Apr '10

29 Apr '10

See http://hackage.haskell.org/trac/hackage/ticket/640 It seems to me that a warning on using the 'haskell98' package wouldn't be a bad thing; those modules have since been split apart in better modules, the names are ever more unfamiliar, etc. But Duncan thinks it merits discussion. -- gwern

16 35