December 2008 - Libraries

There are too many error handling conventions used in library code!
by dons＠cse.unsw.edu.au 13 Aug '11

13 Aug '11

This article on the 8 different error handling strategies various common Haskell libs use: http://www.randomhacks.net/articles/2007/03/10/haskell-8-ways-to-report-err… got me thinking: we need to standardise/recommend a small set of methods for library error handling. The lack of consistent error reporting between libs results in verbose code, as we're not able to use a single error handling technique when gluing together code from different libs (i.e. we can't just use Maybe or Either/ErrorT). (It's always nice when you *can* use, say, a Maybe monad to glue code from a number of sources, but eventually you reach code with a different error system, and the Maybe monad breaks down). So, some questions I think we can tackle: * can we identify error handling strategies from the list that should not be used anymore? (throwDyn?) * can we move some of the outlying libraries to a more consistent error framework? * do we need more support for gluing together our different error systems? (it should be easier to glue code using , say, Maybe/Either). * what role does MonadError play here, as a generic error handler? * can we make precise recommendations about which error strategies to use? -- Don

11 16

Proposal #2629: Data.List: Replace nub; add nubOrd, nubInt, nubWith
by Bart Massey 24 Feb '10

24 Feb '10

http://hackage.haskell.org/trac/ghc/ticket/2629 Everyone always complains about nub, but nobody ever does anything about it except (map head . group . sort), which isn't lazy and isn't always faster. :-) I've implemented a new function nubWith that takes a "stop list" as an argument and filters its target list against the stop list. I've then re-implemented nub and implemented nubOrd and nubInt in terms of nubWith: the stop list is a typeclass, so these implementations are trivial and new implementations are easily added. nubBy is left alone, since there's nothing obvious to be done about it. All of the nubs are still fully lazy. Basic QuickCheck tests are provided, and pass. Performance benchmarking code is provided. The performance of my nub implementation is quite comparable to that of the standard one. My nubOrd and nubInt implementations are dramatically faster, since they use a Set and IntSet respectively for the stop list. In particular, they are performant on long lists with long nubs, unlike the basic nub. My implementation is available via git at git://svcs.cs.pdx.edu/git/nub.git or can be browsed at http://svcs.cs.pdx.edu/gitweb?p=nub.git;a=tree and has a maybe-outdated tarball at http://svcs.cs.pdx.edu/haskell/nub.tar.gz The Nub.hs file itself is attached to the proposal. If the proposal is accepted, I will prepare a patch against current GHC library top-of-tree, but for now it seems easier for everyone to just look at the bits in their current natural habitat.

15 31

export toDescList from Data.Map
by Evan Laforge 25 Nov '09

25 Nov '09

So, I mentioned this a long time ago but didn't get any responses and then I got distracted. So this time I added a ticket and patch and everything: 2580 Here's the text: It's even implemented, but not exported. Without this, there's apparently no way to iterate over a map from high to low, since foldl is also not exported. Also, I implemented a toDescList for IntMap, and could do the same for Set, and could add those patches, if people agree it's a good idea. In addition, is there any problem with exporting foldl aka descending for the three modules? I implemented a foldl for IntMap and it seems to work, but I'd want someone smarter to look for problems with it. In addition, Set and Map are inconsistent with folds: Set exports 'fold' and says "no order is guaranteed", but it's just defined to foldr, which is in ascending order. Set.foldr itself is not exported. Meanwhile, Map also exports fold, but heavily implies from the doc that it's guaranteed to be ascending. Would it restrict implementation changes too much to export a descending fold for all three? Other languages that have tree oriented general purpose maps all let you start anywhere and iterate in either direction. Anyway, is 2-3 weeks ok for discussion period? The library submission wiki doesn't say how long.

6 13

template-haskell depends on packedstring
by Gwern Branwen 18 May '09

18 May '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 So I was trying to compile lambdabot with cabal-install. Lambdabot uses Template Haskell for generating some boilerplate, and it was a failed dependency. Template Haskell uses packedstring for... something... and packedstring was a failed dependency: Data/PackedString.hs:83:7: Could not find module `Data.Data': it is a member of package base, which is hidden Now this is a problem, but it's not the problem I am interested in. I am interested in: why is template-haskell using packedstring? Wasn't packedstring obsoleted years ago? I downloaded the TH tarball to see what fascinating things it was using packedstring for, and I found that it's hardly doing anything - it apparently is just encoding module (ModName) and OccName (whatever that is), and then the other functions shift back from PackedString/ModName-OccName to String. (This is all in Language/Haskell/TH/Syntax.hs) I replaced all the PackedString stuff with normal ol' String and compiled fine, and indeed seems to work fine at Template-Haskelly stuff. So here's my suggestions: 1) Remove the packedstring dep and calls from the TH code. It is useless, it shackles the package to a long obsolete one, and so on. Even if we assume that the packedstring code is still faster than native String, and that there's no performance penalty for the conversions back and forth from String to PackedString, and that the possible size increase from linking in PackedString code is not worth worrying about, and we also assume that PackedString is being used on Names large enough to make up for any overhead - even if we assume all that, TH is a compile-time process and so it doesn't matter all that much if we shave a few microseconds off one small part of the compilation process. Installability and cleaner code would be worth that price. 2) If just String is not acceptable, then I understand from dons that the reason we can't use just plain ol' ByteString is because the Names may well be in UTF-8, and ByteString's [Word8]s apparently don't treat that well. Given that, we need some sort of UTF8 bytestring. As it happens, we do in fact have such a thing: Data.ByteString.UTF8 . It looks a bit incomplete, but utf8-string in general is pretty widely-used and reliable (also builds on 6.10): just about every XMonad user uses XMonadContrib which uses utf8-string, and I can count the number of complaints because utf8-string on one hand. Thoughts? I searched thoroughly through various bug-trackers and mailing lists, but this doesn't seem to've come up before. - -- gwern -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEAREKAAYFAkkqJTsACgkQvpDo5Pfl1oLYoQCgjRSqno+DtyZiLbAS5Pk4l8Ao sqkAn15sBq+alBy5ZVLHIMthjZ6eU6E6 =N8Qj -----END PGP SIGNATURE-----

3 6

extralibs and the haskell-platform
by Ian Lynagh 17 Jan '09

17 Jan '09

Hi all, As the plan is for the extralibs to become part of the Haskell Platform, I think we ought to have a BTS for them. I therefore propose that we create a project on community.haskell.org for each of those that are maintained by libraries(a)haskell.org. For now that will only provide a BTS (trac), but once the GHC VCS switch has happened it would make sense to move the authoritative darcs repos to community too. The packages in question are: haskell-src html HUnit mtl network parallel stm (I've left QuickCheck off as I think that it has real maintainers again) Also, I've created a haskell-platform mailing list: http://projects.haskell.org/cgi-bin/mailman/listinfo/haskell-platform Currently it gets mail for any changes to the haskell-platform trac, but I think that it makes sense to send mail for all the HP package tracs there too, so you don't need to subscribe to dozens of low-traffic lists to keep on top of what's going on. If the volume becomes too high then we can move the noisier packages to their own list. For now at least, discussion should stay on libraries(a)haskell.org. Any comments/objections? Thanks Ian

2 2

Proposal #2875: Correct SYB's representation of Char
by José Pedro Magalhães 16 Jan '09

16 Jan '09

Hello all, SYB uses DataRep to represent datatypes: -- | Public representation of datatypes data DataRep = AlgRep [Constr] | IntRep | FloatRep | StringRep | NoRep I believe that StringRep should be CharRep. Note that IntRep is used for the primitive Int and Integer datatypes, FloatRep for Float and Double, and StringRep (apparently) for Char. String, however, is represented as AlgRep [[],(:)]: *Main> dataTypeOf 'p' DataType {tycon = "Prelude.Char", datarep = StringRep} *Main> dataTypeOf "p" DataType {tycon = "Prelude.[]", datarep = AlgRep [[],(:)]} This makes sense, since String is not a primitive datatype. But it causes the apparently wrong behavior: *Main> fromConstr (mkStringConstr (dataTypeOf "a") "ab") :: String "*** Exception: mkStringConstr *Main> fromConstr (mkStringConstr (dataTypeOf 'a') "ab") :: String "*** Exception: constrIndex The correct way of using mkStringConstr is to construct a Char. This, however, only works for strings with a single character: *Main> fromConstr (mkStringConstr (dataTypeOf 'a') "b") :: Char 'b' *Main> fromConstr (mkStringConstr (dataTypeOf 'a') "ab") :: Char *** Exception: gunfold *Main> fromConstr (mkStringConstr (dataTypeOf 'a') "") :: Char *** Exception: gunfold I find this behavior counterintuitive. Therefore I propose to rename StringRep to CharRep and mkStringConstr to mkCharConstr. For backwards compatibility, this entails: * Deprecating mkStringConstr and StringConstr * Deprecating mkStringRep and StringRep * Introducing mkCharConstr and CharConstr * Introducing mkCharRep and CharRep Additionally, due to deprecation warnings, the following have to change as well: * libraries/template-haskell/Language/Haskell/TH/Quote.hs * compiler/utils/Serialized.hs A patch is attached in #2875 ( http://hackage.haskell.org/trac/ghc/ticket/2875) I propose a discussion period of 4 weeks, therefore until the 8th of January. Thanks, Pedro

2 4

Add Applicative instances for MTL types
by Bas van Dijk 16 Jan '09

16 Jan '09

Hello, In a project of mine I needed an Applicative instance for Identity and noticed it didn't exist. So I decided to add Applicative (and Alternative instances where possible) for all MTL types. When I was about to submit a library proposal I saw there already existed one. So I added my patch to that ticket. My patch I different in that my Applicative instances don't require a Monad constraint. This also means that most Functor instances now also depend on Applicative rather than on Monad. See the ticket for the details: http://hackage.haskell.org/trac/ghc/ticket/2316 Discussion period: 5 weeks (taking the holidays into account) This is the old thread about this proposal: http://thread.gmane.org/gmane.comp.lang.haskell.libraries/9140/focus=9154 regards, Bas

5 9

blame: mtl MonadReader instance for ContT
by Nicolas Frisby 01 Jan '09

01 Jan '09

This mtl instance is in conflict with a usage of shift/reset that I'm porting from ML (Sheard's type-directed partial evaluator). > instance (MonadReader r' m) => MonadReader r' (ContT r m) where > local f m = ContT $ \c -> do > r <- ask > local f (runContT m (local (const r) . c)) I need to remove the "local (const r)" bit. I think this is just a different point in the diverse design space of dynamic binding/delimited control. Since Haskell lacks visibility management for importing instances, I'm forced to duplicate the rest of the ContT code in order to replace this instance with my own. 1) I am hoping this list can recall the reason for putting this design decision into the mtl. Perhaps there's a reason to prefer this particular side-effect interaction. 2) I am suggesting that the mtl be restructured such that this instance be delegated to a separate module so that i) I can avoid importing it along with the other useful instances ii) and the mtl could provide alternative instances. An alternative to breaking out the modules would be to not provide the instance. Another, more radical alternative could involve phantom types. I don't know of any in the mtl, but this same consideration might be applicable to other libraries with "arbitrary" instances.

4 5

Different QSem(N) (was Re: IORef vs TVar performance: 6 seconds versus 4 minutes)
by ChrisK 29 Dec '08

29 Dec '08

I think I can improve on your code. Bertram Felgenhauer wrote: > But why does it manually manage the waiters at all? MVars are fair, in > ghc at least. So this should work: > > data Sem = Sem (MVar Int) (MVar Int) I changed the above to be a data > > newSem :: Int -> IO Sem > newSem initial = liftM2 Sem (newMVar initial) newEmptyMVar > > -- | Wait for a unit to become available > waitSem :: Sem -> IO () > waitSem (Sem sem wakeup) = do > avail' <- modifyMVar sem (\avail -> return (avail-1, avail-1)) Threads can get out of order at this point. This "order bug" may be undesirable. Also, killing the thread while it waits for "wakeup" below would be bad. You need an exception handler and some kind of cleanup. > when (avail' < 0) $ takeMVar wakeup >>= putMVar sem > -- | Signal that a unit of the 'Sem' is available > signalSem :: Sem -> IO () > signalSem (Sem sem wakeup) = do > avail <- takeMVar sem > if avail < 0 then putMVar wakeup (avail+1) > else putMVar sem (avail+1) You should change this from "= do" to "= block $ do". > > (I should turn this into a library proposal.) > > Bertram If you do not need to take N at a time then the untested code below has no "order bug" and is fair. > module Sem where > > import Control.Concurrent.MVar > import Control.Monad(when,liftM2) > > data Sem = Sem { avail :: MVar Int -- ^ provides fast path and fair queue > , lock :: MVar () } -- ^ Held while signalling the queue > > -- It makes no sense here to initialize with a negative number, so > -- this is treated the same as initializing with 0. > newSem :: Int -> IO Sem > newSem init | init < 1 = liftM2 Sem newEmptyMVar (newMVar ()) > | otherwise = liftM2 Sem (newMVar init) (newMVar ()) > > waitSem :: Sem -> IO () > waitSem (Sem sem _) = block $ do > avail <- takeMVar sem > when (avail > 1) (signalSemN (pred avail)) > > signalSem :: Sem -> IO () > signalSem = signalSemN 1 > > signalSemN :: Int -> Sem -> IO () > signalSemN i (Sem sem lock) | i <= 1 = return () > | otherwise = > withMVar lock $ \ _ -> block $ do > old <- tryTakeMVar sem > case old of > Nothing -> putMVar sem i > Just v -> putMVar sem $! succ i All waitSem block in arrival order with the takeMVar in waitSem. The signalSemN avoid conflicting by serializing on the "MVar ()" lock. The above is quite fast so long as the semaphore holds no more than the value 1. Once it hold more than 1 the waiter must take time to add back the remaining value. Note that once threads are woken up in order, they may still go out of order blocking for the () lock when adding back the remaining value (in the presence of other signalers). The above is also exception safe. The only place it can die is during the takeMVar and this merely remove a blocked waiter. I see no way to add a fair waitSemN without changing Sem. But if I change Sem then I can make a fair waitSemN. The untested code is below: > module Sem where > > import Control.Concurrent.MVar > import Control.Monad(when,liftM3) > import Control.Exception.Base > > data Sem = Sem { semWait :: MVar () -- for serializing waiting threads > , semAvail :: MVar Int -- positive quantity available > , semSignal :: MVar () -- for serializing signaling threads > } > > > newSem i | i<=0 = liftM3 Sem (newMVar ()) newEmptyMVar (newMVar ()) > | otherwise = liftM3 Sem (newMVar ()) (newMVar i) (newMVar ()) > > waitSem :: Sem -> IO () > waitSem = waitSemN 1 > > waitSemN :: Int -> Sem -> IO () > waitSemN i sem@(Sem w a s) | i<=0 = return () > | otherwise = withMVar w $ \ _ -> block $ do > let go n = do > avail <- onException (takeMVar a) (signalSemN (i-n) sem) > case compare avail n of > LT -> go $! n-avail > EQ -> return () > GT -> signalSemN (avail-n) sem > go i > > signalSem :: Sem -> IO () > signalSem = signalSemN 1 > > signalSemN :: Int -> Sem -> IO () > signalSemN i (Sem _ a s) | i<=0 = return () > | otherwise = withMVar s $ \ _ -> block $ do > ma <- tryTakeMVar a > case ma of Nothing -> putMVar a i > Just v -> putMVar a $! v+i Trying for exception safety makes the above slightly tricky. It works by allowing only a single thread to get the semWait lock. This keeps all the arriving threads in the fair blocking queue for the semWait lock. The holder of the semWait lock then nibbles at semAvail's positive value until it is satisfied. Excess value is added back safely with signalSemN. Cheers, Chris Kuklewicz

2 1

Cabal Bug / Feature and Improvement
by Dominic Steinitz 29 Dec '08

29 Dec '08

Currently if I do not want to use the default locations for cabal then the following: >dist\build\cabal\cabal.exe --config-file=urghh --remote-repo-cache=foo --verbose=3 update will create urghh but will still (silently) put the repository cache at repoLocalDir = "C:\\Documents and Settings\\steinitd\\Application Data\\cabal\\packages\\hackage.haskell.org" I've made a few changes which will put the cache at "foo". I don't have access to darcs at the moment so I've attached the changes. Let me know if a darcs patch is worth doing. Dominic. In Config.hs: loadConfig :: Verbosity -> GlobalFlags {-Flag FilePath-} -> Flag Bool -> IO SavedConfig loadConfig verbosity globalFlags {-configFileFlag-} userInstallFlag = addBaseConf $ do configFile <- maybe defaultConfigFile return (flagToMaybe (globalConfigFile globalFlags){-configFileFlag-}) minp <- readConfigFile mempty configFile case minp of Nothing -> do notice verbosity $ "Config file " ++ configFile ++ " not found." notice verbosity $ "Writing default configuration to " ++ configFile commentConf <- commentSavedConfig initialConf <- initialSavedConfig writeConfigFile configFile commentConf (initialConf `mappend` (mempty {savedGlobalFlags = globalFlags})) return initialConf The other cases of loadConfig are unchanged. In Main.hs global replace: loadConfig verbosity (globalConfigFile globalFlags) by loadConfig verbosity globalFlags

2 4