July 2008 - Libraries

There are too many error handling conventions used in library code!
by dons＠cse.unsw.edu.au 13 Aug '11

13 Aug '11

This article on the 8 different error handling strategies various common Haskell libs use: http://www.randomhacks.net/articles/2007/03/10/haskell-8-ways-to-report-err… got me thinking: we need to standardise/recommend a small set of methods for library error handling. The lack of consistent error reporting between libs results in verbose code, as we're not able to use a single error handling technique when gluing together code from different libs (i.e. we can't just use Maybe or Either/ErrorT). (It's always nice when you *can* use, say, a Maybe monad to glue code from a number of sources, but eventually you reach code with a different error system, and the Maybe monad breaks down). So, some questions I think we can tackle: * can we identify error handling strategies from the list that should not be used anymore? (throwDyn?) * can we move some of the outlying libraries to a more consistent error framework? * do we need more support for gluing together our different error systems? (it should be easier to glue code using , say, Maybe/Either). * what role does MonadError play here, as a generic error handler? * can we make precise recommendations about which error strategies to use? -- Don

11 16

Arrow without `>>>'
by Valery V. Vorotyntsev 03 Sep '08

03 Sep '08

Hi, friends, I've built GHC from darcs, and... Could anybody tell me, what's the purpose of Arrow[1] not having `>>>' method? 1. http://darcs.haskell.org/packages/base/Control/Arrow.hs $ ghci GHCi, version 6.9.20080104: http://www.haskell.org/ghc/ :? for help Loading package base ... linking ... done. Prelude> :m + Control.Arrow Prelude Control.Arrow> :i Arrow class (Control.Category.Category a) => Arrow a where arr :: (b -> c) -> a b c pure :: (b -> c) -> a b c first :: a b c -> a (b, d) (c, d) second :: a b c -> a (d, b) (d, c) (***) :: a b c -> a b' c' -> a (b, b') (c, c') (&&&) :: a b c -> a b c' -> a b (c, c') -- Defined in Control.Arrow instance Arrow (->) -- Defined in Control.Arrow instance (Monad m) => Arrow (Kleisli m) -- Defined in Control.Arrow Prelude Control.Arrow> Leaving GHCi. I can't build[2] arrows-0.3 package without `>>>' in Arrow. 2. vvv@fun:~/src/arrows$ darcs w { hunk ./Control/Arrow/Operations.hs 36 +import Control.Category ((>>>)) hunk ./Control/Arrow/Transformer/CoState.hs 23 +import Control.Category ((>>>)) } vvv@fun:~/src/arrows$ runhaskell Setup build Preprocessing library arrows-0.3... Building arrows-0.3... [ 3 of 12] Compiling Control.Arrow.Transformer.CoState ( Control/Arrow/Transformer/CoState.hs, dist/build/Control/Arrow/Transformer/CoState.o ) Control/Arrow/Transformer/CoState.hs:29:7: `>>>' is not a (visible) method of class `Arrow' The question arises "should I?", but this is one of lambdabot's[3] depencies. 3. http://code.haskell.org/lambdabot/ Thanks a lot! -- vvv

4 5

Syb Renovations? Issues with Data.Generics
by Claus Reinke 21 Aug '08

21 Aug '08

Calling all Syb/Data.Generics users!-) I keep running into problems with Data.Generics, mostly because I actually want to use it (no claims that it is the best or final solution, or that other approaches aren't equally in need of support, just that it is the best-supported working approach right now). Some tricky issues are (sometimes against published expectations) solvable, suggesting useful additions to the library, but some seemingly trivial things have me stumped, suggesting (to me, at least;-) a need for improvements either in the library or in its documentation. Part of the reason I'm interested in this now is that Data/Typeable instances seem likely (I hope:-) to be added to the GHC Api, where Thomas Schilling is working on improvements http://hackage.haskell.org/trac/ghc/wiki/GhcApiStatus http://hackage.haskell.org/trac/ghc/wiki/GhcApiAstTraversals also, the old question of porting HaRe to the GHC Api is currently being looked into again, by Chaddaï Fouché, and crucially depends on Syb's generic traversals. As it is still holiday season, it is a bit early for proposal deadlines, but I'd like to start a discussion of Syb/Data.Generics and collect the issues and solutions arising, in the hope of following up with concrete proposals for improvements. To start the discussion, a simple item: 1. inconvenient convenience instances of Data for non-"data" types Data.Generics.Instances defines instances of Data for many types, including some abstract types that don't really fit into the concrete value based model of Data, like 'IO a' and 'a->b'. Those instances give runtime errors for some class methods, and mainly offer faked (no-op) gmap traversals, serving as a convenience/enabler for 'deriving instance Data': http://www.haskell.org/pipermail/generics/2008-June/000346.html A list of the odd instances in Data.Generics.Instances, with examples of their oddities, can be found here: http://www.haskell.org/pipermail/generics/2008-June/000347.html My suggestion is to split this module into two, and stop the implicit import/export of the incomplete instances from Data.Generics. Reactions to this suggestion have been muted so far (Simon PJ was as surprised as I was about the existence of these instances, but has no strong opinion about the issue, Alexey Rodriguez supports the suggestion, Ian Lynagh points out the difficulty of transition), which is one reason why I'll try to move the discussion to libraries@. Pro: - the instances are still available, and only one explicit import away, so 'deriving instance Data' for types containing uninteresting functions is still convenient - the problematic instances are no longer implicitly imported, so applications that don't want these instances can now avoid them completely, or define their own instances - these convenience instances are not just inconvenient for some applications, due to the way intances are handled in Haskell; they actually violate some "natural" invariants like "everything queries every substructure of the specified type", "everywhere applies a transformation at every substructure of matching type" - the situation is similar to Text.Show.Functions, as the convenience instances don't provide the full expected functionality, just barely enough for deriving to get by Cons: - due to the implicit import and use of these instances, there is no obvious transition scheme; it seems that the least painful process would be to make the change without transition/deprecation period and to document the explicit import option [it would be useful to have a way of deprecating instance imports, so that any deriving scheme depending on imports from a deprecated location would trigger a warning, in this case suggesting the new import location] As I said, I'd like to wait until at least the Syb authors are back from holidays before setting any proposal deadlines, but I'd like to invite feedback from Syb users on this and other Syb issues. Here is a preview on other items I'd like to raise later on, please add your own: 2. Data.Generics.Utils Since Data/Typeable are compiler-derivable (in GHC) while other classes like Functor/Traversable/etc are not, it would be useful if generic instances for those other classes could be defined in terms of Data/Typeable. The Uniplate library already does this for its own classes via Data.Generics.PlateData, and it appears that at least Functor is defineable as well (code exists, proof is only informal at this stage, and those invariant violations and runtime errors in the implicitly imported dummy instances from (1) really get in the way): http://www.haskell.org/pipermail/generics/2008-June/000343.html http://www.haskell.org/pipermail/generics/2008-July/000349.html http://www.haskell.org/pipermail/generics/2008-July/000351.html What other classes can be defined in this way? Traversable (traverse) seems very nearly possible, what else? 3. Performance Naive use of Syb traversal schemes can lead to huge performance losses. Experienced users tend to write their own traversal schemes, using Syb's low-level Api directly, but we can take inspiration from some Uniplate/PlateData optimization techniques and generalise them for use with Syb's high-level traversal scheme Api, yielding similar performance gains for everywhere/everything: http://www.haskell.org/pipermail/generics/2008-July/000353.html Another direction that might be worth exploring is to use Maps instead of nested generic extensions to define adhoc-overloaded transformation and queries (I've actually started playing with that, but am currently stuck on GHC ticket #2463). 4. Useability There is probably nothing one can do to make the types of Syb's low-level Api less of a brain hazard, but not all of the stumbling blocks seem to be necessary consequences of the carefully crafted edifice of interactions between nearly polymorphic types, runtime type checks and type reflection. Examples: - there doesn't seem to be a way to get hold of a types' constructors, only of constructor representations, structure scaffolds, and structure generators - the actual domain on which a transformation/query acts is hidden behind the near-polymorphic default type of generic extensions - I can't seem to figure out how to use typeOf1, when the other Syb operations only give me 'forall a . Data a => a'; instead, I seem to be forced to use something like: [ mkTyConApp tyCon (init tyArgs) | not (null tyArgs) ] where (tyCon,tyArgs) = splitTyConApp typeRep - others? What are your personal gripes with Syb/Data/Typeable, and for which of them do you see a chance of addressing them by changing/adding code? Claus

5 12

ANN: protocol-buffers (very early edition) version 0.0.5
by Chris Kuklewicz 14 Aug '08

14 Aug '08

Hi all, The Haskell version of protocol-buffer is still pre-bootstrap, but further, so there is a 0.0.5 release on http://hackage.haskell.org/cgi-bin/hackage-scripts/package/protocol-buffers and a darcs repository on http://darcs.haskell.org/packages/protocol-buffers/ The leading 0.0 in the type should show how early this code is :) What are protocol buffers? The are google's data interchange format: http://code.google.com/apis/protocolbuffers/docs/overview.html > What are protocol buffers? > > Protocol buffers are a flexible, efficient, automated mechanism for > serializing structured data – think XML, but smaller, faster, and simpler. You > define how you want your data to be structured once, then you can use special > generated source code to easily write and read your structured data to and from > a variety of data streams and using a variety of languages. You can even update > your data structure without breaking deployed programs that are compiled against > the "old" format. In addition to a manual translation of the self describing 'decriptor.proto' it has a partial code generator in Gen.hs and a stub of a parser in ProtoParser.hs and an intermediate data type in WireMessage.hs that I expect to sit between the byte stream reader/writer and the message types. Eventually the parser with read 'decriptor.proto' and the generator will emit source code to replace the manual translation. FAQ 1: Why is the Text.DescriptorProtos.* hierarchy so huge and the imports so plentiful? ANSWER 1: The namespace management. The namespace paradigm is the nested OO style namespace of C++, so I need modules and imports to fake nested Haskell namespaces. Cheers, Chris Kuklewicz

1 1

A fancier Get monad or two (a la binary and binary-strict)
by Chris Kuklewicz 06 Aug '08

06 Aug '08

Summary: I have two new Get-like monads for binary data (byte-aligned) that also (*) Suspend parsing and request more import when reading past the end of data. It is possible to respond with Nothing to indicate the hard end of data. (*) Return failures instead of calling error (*) Offer lookAhead,lookAheadM,lookAheadE like Data.Binary.Get (*) Are BinaryParser instances from Data.Binary.Strict.Class (*) Are Monad Transformers (and thus MonadIO) (*) Are MonadError,Plus,Reader,Writer,State and Applicative,Alternative (*) They differ because one is also MonadCont while the other is simpler (*) Simplified Non-transformer versions (applied to Identity) are defined Note: Making the suspend/resume-with-more-input work efficiently with MonadError/Alternative is one improvement to binary-strict. Make callCC work with sane semantics at the same time was non-trivial. Long version and where to find the source: In the process of writing an implementation of google's protocol buffers, I used binary's Put monad and binary-strict's BinaryParser class (i.e. binary-strict's Incremental Get monad). The Get in the binary package works on Lazy ByteString (good for me) but tends to call "error" when unhappy, which was not ideal. The Incremental Get from the binary-strict package does not want to work on Lazy bytestrings (it can be adapted), but the internals hold onto data too long and obstruct garbage collection (this is how it appears to me when I read the code). So I have made a fancier Get monad that offers a large collection of features. I am announcing this in case other people might want to further adapt my solution for their own ends (or just use it outright). Removing feature from my code ought to be much easier than adding them was. The code itself is currently living in a few files inside the protocol-buffers project at http://darcs.haskell.org/packages/protocol-buffers/Text/ProtocolBuffers/ The files are "MyGet.hs" or "MyGetW.hs", there are two flavors. (The "testDepth" file is annotated test output from MyGetW's testDepth function). They depend on binary-strict, but this would be easy to comment out. They both contain code "lifted" from binary and "inspired" by binary-strict. MyGetW.hs is the version with a MonadCont instance. MyGet.hs has the (forall b.) to simplify the types and thus has no MonadCont instance. I have tested the code, but it is still quite new. I doubt I will have time to make a proper cabal release. I may see if the maintainers of the binary-strict or binary packages are interested in a fancier Get monad. Cheers, Chris K

4 10

Unordered map
by Andrew Wagner 01 Aug '08

01 Aug '08

All, Forgive me if this has been brought up before, but I was just thinking about the fact that there's no Map structure in the standard library that doesn't pretty much require an Ord instance for its keys. I say "pretty much" because, while you can declare a Map with keys of a type that's not an Ord instance, you can't do hardly anything with it. E.g., you can't do an insert. Makes it kind of pointless. Now, I understand the fact that this is for efficiency purposes, but it does seem a bit inconsistent. Why allow the construction of a Map you can't do anything with? Here are two possible more consistent ways of handling this: Suppose I have some data structure that is inherently unordered, and I want to use it as a key into a Map. How would I get around this limitation on the client side? Well, the most obvious way is to create some nonsensical Ord instance, like compare _ _ = Eq. Is that the recommended solution? If so, we could create unordered versions of all the ordered methods that assume this 'ordering'. If not, what is the recommended way, and why not create unordered versions that use that? The other way to make the library more consistent, perhaps, would be to simply move the Ord requirement up to that data structure: that is, make it Ord k => Map k a, and then have a new data structure like UnorderedMap (or, to use a more standard term, Dictionary). Thoughts? Am I completely missing something?

8 12

proposal #2461: add Traversable generalizations of mapAccumL and mapAccumR
by Ross Paterson 01 Aug '08

01 Aug '08

The proposal is to add the following functions to Data.Traversable, generalizing the list versions in Data.List: -- |The 'mapAccumL' function behaves like a combination of 'fmap' -- and 'foldl'; it applies a function to each element of a structure, -- passing an accumulating parameter from left to right, and returning -- a final value of this accumulator together with the new structure. mapAccumL :: Traversable t => (a -> b -> (a, c)) -> a -> t b -> (a, t c) -- |The 'mapAccumR' function behaves like a combination of 'fmap' -- and 'foldr'; it applies a function to each element of a structure, -- passing an accumulating parameter from right to left, and returning -- a final value of this accumulator together with the new structure. mapAccumR :: Traversable t => (a -> b -> (a, c)) -> a -> t b -> (a, t c) These functions are handy for things like labelling trees, zipping, etc. Deadline: 3 weeks

7 12

darcs patch: Fix Unicode bug in haskell-src
by Andrea Vezzosi 31 Jul '08

31 Jul '08

Language.Haskell.Parser currently can't parse modules with unicode symbols in operators, which are allowed by the haskell98 standard. The attached patch is a small change in the lexer code to fix this. Since parseModule expects a String of Unicode codepoints as usual, examples/hsparser.hs will still raise errors on modules that use unicode, because it doesn't decode the source. The correct behaviour, assuming an utf-8 source, can be tested like this: > import System.IO.UTF8 as U -- from utf8-string > import Language.Haskell.Parser > import Language.Haskell.Pretty > main = do file <- U.readFile "module.hs" > case parseModule of > ParseOk m -> U.putStrLn (prettyPrint m) > e@(ParseError _ _) = print e

3 2

Owning SYB
by Simon Peyton-Jones 29 Jul '08

29 Jul '08

Dear Generic folk [I'm spamming libraries(a)haskell.org too, in case anyone interested in generics is not on generics(a)haskell.org.] As you know, Claus has offered a somewhat-detailed proposal for changes to the SYB library (below). But I don't think that we have an active maintainer for any of the generic-programming libraries (esp SYB) apart from Uniplate. Then there's the related question of what generic-programming technology to promote for clients of the GHC API. The obvious candidates are Claus himself, or Alexey Rodriguez, or Thomas Schilling; but perhaps there are others too? Maybe no one has stepped forward because you all think that I'm on the job! But I'm not... I'm busy with GHC itself, and would love a maintainer for SYB and associated gubbins. I fear that otherwise we may lose the benefits of Claus's homework. Simon | -----Original Message----- | From: generics-bounces(a)haskell.org [mailto:generics-bounces@haskell.org] On Behalf Of Claus Reinke | Sent: 21 July 2008 14:16 | To: generics(a)haskell.org | Subject: [Hs-Generics] Data.Generics with GPS (using Maps to avoid getting lost in Data) | | | summary: speed up Syb with Uniplate-inspired techniques | | == Performance issues in Syb == | | As it stands, classic Syb, while well-supported in GHC isn't exactly the | fastest generic programming option. In fact, it routinely seems to come | out last in performance comparisons, sometimes by a substantial factor. | | That isn't always a problem in practice, because the gains in | expressiveness/conciseness/maintainability outweigh the loss in | performance, because traversal performance is not the bottleneck, or | because practical use often involves hand-tuned traversal schemes | instead of the example schemes typically used in benchmarks. | | Be that as it may, performance is a consideration, negative results have | been published, and it seems that performance of experimental code has | led to Syb being abandoned in at least one project. While it would be | unrealistic to expect Syb traversals to compete with hand-written ones | with current compilers, there are several obvious areas where Syb | traversal performance could be subjected to improvements: | | (a) traversing irrelevant parts of the structure (because everything is | treated generically) | (b) combining results in simple, but inefficient ways ((++) nesting with | the structure) | (c) repeated runtime type checks to determine which function in a | generically extended transformation/query to apply | | While runtime type checks are inherent in SYB (and alternatives have | been proposed that claim to avoid them), all of a/b/c can be addressed | to some extent in defining tuned traversals using the basic library. | There tends to be a trade-off for a vs c, as avoiding senseless generic | traversals of specific substructures implies additional runtime type | checks to identify those substructures [...rest snipped...]

3 3

Re: compositional gMap in SYB1
by Claus Reinke 29 Jul '08

29 Jul '08

> Hello! Hello again!-) > Claus wrote: >> Perhaps we can combine our versions to get the best of both? > > I have done so. I have committed a new version of > /comparison/SYB1_2/GMap.lhs [for readers of libraries@: that is in the generics code repo at http://darcs.haskell.org/generics/comparison/SYB1_2/GMap.lhs , look for gmap2/gmapt and comments; the topic is defining Functor fmap generically (hence gmap) in terms of Data/Typeable] Ah, of course! Silly me, if I had followed my own choice of metaphor to the end, I would have known that X marks the spot only on a map, not on the real thing (or else everyone could find the treasure..). Your shallow, incrementally built not-quite copy is just another representation of a map, and since both map and X are hidden from users of the function, everyone is happy. That something out of nothing trick is also used in the PlateData optimization and in my adaptation of it. The difference, apart from choice of representation, is mainly in whether to extract substructure placeholders/types on a type basis once per traversal or on a value basis, incrementally. Here is a re-run of the technique, for defining traverse in terms of Data/Typeable. Apart from being useful for Data.Traversable library users, the type of 'traverse id' is similar to the type of transpose, which should interest the generics readers. {-# LANGUAGE DeriveDataTypeable #-} {-# LANGUAGE ExistentialQuantification #-} {-# LANGUAGE ScopedTypeVariables #-} {-# LANGUAGE RankNTypes #-} import Control.Applicative import Data.Generics data X = X deriving (Data,Typeable) traverseData :: forall f a b t . (Applicative f,Typeable1 f,Typeable a,Typeable b, Data (t a),Data (t b),Data (t X)) => (a -> f b) -> t a -> f (t b) traverseData f = traverseWithMap f (Dyn (undefined::t X)) where traverseWithMap :: forall f a b x y . (Applicative f,Typeable1 f,Typeable a,Typeable b,Data x,Data y) => (a -> f b) -> Dyn -> x -> f (y) traverseWithMap f (Dyn t) x | typeOf t==typeOf X = maybe (error "traverseData: non-applicable type marked") ($x) (cast f) | otherwise = rebuildWith (traverseWithMap f) t1 x where t1 = Dyn (fromConstr (toConstr x) `asTypeOf` t) rebuildWith :: forall f x y . (Applicative f,Data x,Data y) => (forall x y . (Data x,Data y) => Dyn -> x -> f y) -> Dyn -> x -> f y rebuildWith f (Dyn t) x = case gunfold (k f) (\g -> UnfldStateF2 (pure g) tkids kids) con of UnfldStateF2 a [] [] -> a where (tcon,tkids) = dynamize t (con,kids) = dynamize x k :: forall a b . (Data a) => (forall x y . (Data x,Data y) => Dyn -> x -> f y) -> UnfldStateF2 f (a->b) -> UnfldStateF2 f b k f (UnfldStateF2 ca (tkid:tkids) ((Dyn kid):kids)) = UnfldStateF2 (ca <*> (f tkid kid)) tkids kids data UnfldStateF2 f a = UnfldStateF2 (f a) [Dyn] [Dyn] data Dyn = forall a. Data a => Dyn a data Kids a = Kids{growUp:: [Dyn]} dynamize :: Data a => a -> (Constr,[Dyn]) dynamize x = (toConstr x, growUp $ gfoldl f (const (Kids [])) x) where f (Kids l) a = Kids (l ++ [Dyn a]) test = do print $ traverseData (Just . not) tuple print $ traverseData (Just . not) list traverseData print tuple >>= print traverseData print list >>= print where tuple = (True,True) list = [True,True] Btw, while the type of 'traverseData id' matches that of transpose, its behavious doesn't quite, depending on example: *Main> :t traverseData id traverseData id :: (Data (t X), Data (t b), Data (t (f b)), Typeable b, Typeable1 f, Applicative f) => t (f b) -> f (t b) *Main> traverseData id [Just x|x<-[1..3]] Just [1,2,3] *Main> traverseData id [[1..3],[4..6]] [[1,4],[1,5],[1,6],[2,4],[2,5],[2,6],[3,4],[3,5],[3,6]] > optimization: if it is determined that a structured value has no > components to map, no traversal is performed and the value is returned > as it is. That should speed things up a little. The trick is producing > something out of nothing. That optimization is another indication of the relation to the substructure types map building, although pleasingly different in form (always nice to learn new tricks;-). > Too bad that the deadline to amend a Haskell workshop paper > has passed, and so has the deadline for the generics workshop. Perhaps > we should write a separate paper? Sounds like a good idea. When you're back. > I'm afraid I would be out of town again for two weeks and so > would not be able to follow the discussion closely. Ah, you're just trying to hide the two-week round trips for lightspeed messages resulting from your space travels!-) Thanks for yet another useful contribution, Claus

1 0