- Haskell-Cafe - Haskell.org

GHC optimizer consuming memory
by mike frai 30 Apr '11

30 Apr '11

Hi, While using Michael Snoyman's persistent package, I discovered that a certain bit of code generated from it made the GHC optimizer consume a ridiculous amount of memory (ie. only when using the "-O" flag). I know very little about the GHC compiler and Michael Snoyman recommended I post my findings here to see what others thought. I've done what I can to extract the code generated from the persistent package and simplify/strip it down to the bare minimum of what's needed to cause this issue. You can find it in the attachment FromTest.hs. From my testing, removing five fields from Rec and the equivalent from the fromValue function - the memory consumption drops dramatically and I am able to compile. For your convenience, I've also attached FromTest.out, which is the output I get when compiling FromTest.hs using "ghc -O -v3". This is just the output up to the point it starts consuming a lot of memory. I've never seen anything after this point because my system slows down to a crawl (due to swapping) and I end up killing the process. Please let me know what you guys think or if you need any more information about this. Thanks, - Mike

2 1

How to keep cabal and ghci package versions in sync?
by Gracjan Polak 30 Apr '11

30 Apr '11

Hi all, I have a project with a .cabal file listing package dependencies using the usual version constraints ==X.Y.* <Z.W or >=K.J syntax. Standard route cabal configure; cabal build works correctly as it is able to select working set of package versions. I have also a .ghci file. When I run GHCi it uses all latest installed packages in the system. This prevents the project from loading. I tried to use 'cabal-dev ghci', but this still selects latest global packages. So, how to I load up ghci with the same package versions as selected by cabal? Thanks! -- Gracjan

5 7

Template Haskell support for GADTs
by Boris Lykah 30 Apr '11

30 Apr '11

Hi all! I am writing a library which allows to refer to the separate fields of a datatype. The fields are described as GADT with one constructor for each field. The constructors return GADT with the field type. The auxiliary data structures for this should be generated automatically via Template Haskell. I found that the GADTs produced by TH are not equivalent to the usual ones. They require additional extension -XTypeFamilies(for the equality constraints) along with -XGADTs, and, which is more important, are less type-safe. To describe the data structure I used ForallC as it was suggested in the closed ticket http://hackage.haskell.org/trac/ghc/ticket/3497 (Template Haskell support for GADTs) This is simplified code for data Sample = Sample {foo::String, bar::Int} written manually: data SampleField a where FooField :: SampleField String BarField :: SampleField Int and a GADT with similar structure generated via TH. $(do let gadt = mkName "THSampleField" let tv = mkName "a" let con1 = forallC [plainTV tv] (cxt $ [equalP (varT tv) (conT ''String)]) $ normalC (mkName "THFooField") [] let con2 = forallC [plainTV tv] (cxt $ [equalP (varT tv) (conT ''Int)]) $ normalC (mkName "THBarField") [] result <- dataD (cxt []) gadt [plainTV tv] [con1, con2] [] return [result] ) which produces data THSampleField a where THFooField :: a ~ String => THSampleField a THBarField :: a ~ Int => THSampleField a The expression asTypeOf FooField BarField fails to compile as expected because String cannot match Int, but asTypeOf THFooField THBarField is a valid expression of type (THSampleField a), which is very confusing and breaks the existing code. Am I missing something? If this is the only way to create the GADTs then I think we should consider reopening the ticket. -- Regards, Boris

1 0

More ideas for controlled mutation
by Edward Z. Yang 30 Apr '11

30 Apr '11

Laziness can be viewed as a form of controlled mutation, where we overwrite a thunk with its actual value, thus only running the code once and reaping great time benefits. I've been toying around with some ideas where we do alternative forms of controlled mutation. One such idea has to do with memoization. One way we memoize functions is by building up a data-structure that covers the entirety of input domain of the function, with the values in each slot being thunks for the corresponding function call. Now, this is all very nice and well, but for some types of inputs (like machine integers) we end up making a very big structure for a function for which, in practice, we'll only store and retrieve a few values of the domain. Hash tables take advantage of this fact by simply chaining together values in a linked list if they land in the same bucket. Could we have similarly bucketized memoization? What we want here is for a *thunk to possibly evaluate to different values, but calls to the API be observationally equivalent.* That is, if the only way I can inspect a dictionary list is do a lookup, I don't care if my representation is [(1,4),(2,2)] or [(2,2),(1,4)]. An obvious way to do this is to use unsafePerformIO to read out an IORef stating the value currently being looked up, and have the thunk evaluate to the pair of that key and the result. There are some synchronization concerns, of course: ideally we would only take out a lock on the thunk once we realize that the value doesn't already exist in the memotable, but I don't think there's a way in GHC Haskell to observe if a value is a thunk or not (maybe such a mechanism would be useful?) This seems related to lazy IO, where thunks are co-opted into performing input-output effects. After all, mutation is just a controlled form of IO. Is lazy IO evil or not? One consensus is that for operations that involve limited resources (file descriptors, etc.) lazy IO is too opaque for its own good. It seems we also haven't cracked it's performance problems either (for the relatively un-objectionable generation of an infinite stream of random numbers, Don Stewart notes: "There are real overheads here. Consider eagerly filling chunks and extracting elements piecewise."). But code that looks pure and can be reasoned about like pure code, but has the efficiency benefits of mutation is a very attractive thing indeed. I think it's worth some thought, though the programming world at large has a hard enough time reasoning about laziness even when everything is completely pure! Cheers, Edward P.S. An obvious question is whether or not we could use this technique to implement splay heaps or hash tables with pure interfaces. My opinion is no, because these structures demand you be able to *observe* the mutation.

4 5

Parsing binary 'hierachical' objects for lazy developers
by John Obbele 30 Apr '11

30 Apr '11

Hi Haskellers, I'm currently serializing / unserializing a bunch of bytestrings which are somehow related to each others and I'm wondering if there was a way in Haskell to ease my pain. The first thing I'm looking for, is to be able to automatically derive "Serializable" objects, for example: --------------------------------------------------------------- import Data.Serialize -- using cereal as an example data MyFlag = One | Two | Three instance Serialize [MyFlag] where put = putWord16le . marshalFlags get = unmarshal `fmap` getWord16le data ObjectA = ObjectA { attribute0 :: Word8 , attribute1 :: Word16le , attribute2 :: [MyFlag] } deriving (Serialize) -- magic goes here! --------------------------------------------------------------- Unfortunately ghci complains that 'Serialize' is not a derivable class. Yet, deriving the Serialize instance for ObjectA should be simple, since all the three attributes are already serializable themselves... Second issue, I would like to find a way to dispatch parsers. I'm not very good at expressing my problem in english, so I will use another code example: --------------------------------------------------------------- -- let's say we have two objects with almost the same structure: data ObjectA = ObjectA { objLength :: Int , objType :: TypeId , attribute2a :: [MyFlag] } data ObjectB = ObjectB { objLength :: Int , objType :: TypeId , attribute2b :: Word32le } --------------------------------------------------------------- When we begin to deserialize theses objects, we don't know their final type, we just know how to read their length and their typeId. Only then can we determine if what we are parsing is an ObjectA or an ObjectB. Once we now the object type, we can resume the parsing and return either an ObjectA or ObjectB. Oki, so I may have read too much of Peter Seibel's chapter on binary-data parsing in Common Lisp or spent too much time working on object-oriented code, but currently, I have no idea on how to write this 'simply' in Haskell :( any help would be welcome /john

4 6

Re: [Haskell-cafe] Iteratee: manyToOne
by John Lato 29 Apr '11

29 Apr '11

> > From: Felipe Almeida Lessa <felipe.lessa(a)gmail.com> > > On Thu, Apr 28, 2011 at 1:10 PM, Felipe Almeida Lessa > <felipe.lessa(a)gmail.com> wrote: > > On Thu, Apr 28, 2011 at 12:09 PM, Felipe Almeida Lessa > > <felipe.lessa(a)gmail.com> wrote: > >> I foresee one problem: what is the leftover of 'manyToOne xs' if each > >> x in xs needs different lengths of input? > >> > >> One possible untested-but-compiling solution: > > [snip] > > > > Like I said, that manyToOne implementation isn't very predictable > > about leftovers. ?But I guess that if all your iteratees consume the > > same input OR if you don't care about leftovers, then it should be > > okay. > > Sorry for replying to myself again. =) > > I think you can actually give predictable semantics to manyToOne: > namely, the leftovers from the last iteratee are returned. This new > implementation should be better: > If you do this, the user needs to take care to order the iteratees so that the last iteratee has small leftovers. Consider: manyToOne [consumeALot, return ()] In this case, the entire stream consumed by the first iteratee will need to be retained and passed on by manyToOne. In many cases, the user may not know how much each iteratee will consume, which can make these semantics problematic. Iteratee has 'enumPair', (renamed 'zip' in HEAD) which returns the leftovers from whichever iteratee consumes more. This avoids the problem of retaining extra data, and seems simpler to reason about. Although if you really need to consume a predictable amount of data, the safest is probably to run the whole thing in a 'take'. John Lato

3 3

Why not Darcs?
by Andrew Coppin 29 Apr '11

29 Apr '11

I'm sure this must be a VFAQ, but... There seems to be universal agreement that Darcs is a nice idea, but is unsuitable for "real" projects. Even GHC keeps talking about getting rid of Darcs. Can anybody tell me what the "problems" with Darcs actually are?

29 52

Inputs to classic FRP: unsafeInterleaveIO/unsafePerformIO
by Edward Amsden 29 Apr '11

29 Apr '11

As far as I can tell, with classic FRP implementations (those which use behaviors as a first-class abstraction), the only way to create a behavior or event based on some external input (for instance keypresses or microphone input) is to do something with unsafePerformIO or unsafeInterleaveIO. A behavior is a value, which when evaluated at a specific time would have to either block its evaluation until input could be read, or check the input at that particular time. Is there any other way of implementing external behaviors besides that? -- Edward Amsden Student Computer Science Rochester Institute of Technology www.edwardamsden.com

3 7

Python is lazier than Haskell
by Henning Thielemann 29 Apr '11

29 Apr '11

I like to apply for the quote of the week. :-) "If Haskell is great because of its laziness, then Python must be even greater, since it is lazy at the type level." Dynamically typed languages only check types if they have to, that is if expressions are actually computed. Does this prove that laziness at type level as in Python is a great thing or does this prove that laziness at the value level as in Haskell is a bad thing or does this prove nothing? Hope for controversial discussion ...

22 26

Iteratee: manyToOne
by Dmitry Olshansky 29 Apr '11

29 Apr '11

Hello, does somewhere exist function with type like this - manyToOne :: [Iteratee a m b] -> Iteratee a m [b] ? I.e. I need to process one input through many Iteratees indepentently in constant space and collect results. It is similar by type with sequenceM but as far as I understand sequenceM won't use the same input for all Iteratees. Dima

3 5