October 2011 - Haskell-Cafe

Cannot understand liftM2
by Nicola Paolucci 13 Feb '12

13 Feb '12

Hi All, I'm loving learning Haskell quite a bit. It is stretching my brain but in a delightfull way. I've googled, I've hoogled but I haven't found a clear explanation for what exactly liftM2 does in the context below. Using the cool lambdabot "pointless" utility I found out that: > \x -> snd(x) - fst(x) is the same as: > liftM2 (-) snd fst I like the elegance of this but I cannot reconcile it with its type. I can't understand it. I check the signature of liftM2 and I get: Prelude> :t liftM2 Prelude> liftM2 :: (Monad m) => (a1 -> a2 -> r) -> m a1 -> m a2 -> m r Can someone help me understand what's happening here ? What does a Monad have to do with a simple subtraction ? What is actually the "m" of my example ? I am sure if I get this I'll be another step closer to illumination ... Thanks, Nick

15 18

Parsing workflow
by Nils Schweinsberg 21 Jan '12

21 Jan '12

Hi! I'm having a really hard time to write a correct parser for a small language I've developed. I have been trying to write a parser using parsec, but always get a lot of error messages like "unexpected "\n", expected ..., new-line or..." when trying to run the parser. Then I read about the happy parser and really liked the separation of lexing the text into tokens and parsing the actual logic behind those tokens. Since I couldn't get familiar with the lexer "alex" I gave up on the alex-happy-approach again and went back to parsec. But with that lexer->parser idea on my mind, my parser currently looks a lot like a lexer. So I came up with the idea of using a combination of parsec and happy, where I generate a list of tokens for my text via parsec and analyse it with happy. My questions would be: - Is this a valid approach? - What is your workflow on parsing complex data structures? - What about performance? Since my project is going to be an interpreted language parsing performance might be interesting aswell. I've read that happy is in general faster than parsec, but what if I combine both of them as I said above? I guess that parsing a simple list of tokens without any nested parser structures would be pretty fast? - Do you have any other ideas on how to improve my parser? - What are your general thoughts on happy vs. parsec? Thanks for any replies, Nils

10 17

SMP parallelism increasing GC time dramatically
by Tom Thorne 10 Jan '12

10 Jan '12

I am having some strange performance issues when using SMP parallelism, that I think may be something to do with GC. Apologies for the large readouts below but I'm not familiar enough to know what is and isn't relevant! I have a pure function that is mapped over a list of around 10 values, and this happens several times for each iteration of my program. It does some fairly intensive calculations using hmatrix, generating intermediate matrices along the way. The computation is significantly more complex for some values, so the work done by each call is not spread equally. I did some profiling and it seems like the program is spending about 50% of its time in that function. First of all, without any attempts at parallelism, I see this from ./Main +RTS -s 67,142,126,336 bytes allocated in the heap 147,759,264 bytes copied during GC 109,384 bytes maximum residency (58 sample(s)) 354,408 bytes maximum slop 3 MB total memory in use (0 MB lost due to fragmentation) Generation 0: 104551 collections, 0 parallel, 1.13s, 1.11s elapsed Generation 1: 58 collections, 0 parallel, 0.01s, 0.01s elapsed Parallel GC work balance: -nan (0 / 0, ideal 1) MUT time (elapsed) GC time (elapsed) Task 0 (worker) : 0.00s ( 67.06s) 0.00s ( 0.00s) Task 1 (worker) : 0.00s ( 67.09s) 0.00s ( 0.00s) Task 2 (bound) : 66.95s ( 67.09s) 1.14s ( 1.12s) SPARKS: 0 (0 converted, 0 pruned) INIT time 0.00s ( 0.00s elapsed) MUT time 66.95s ( 67.09s elapsed) GC time 1.14s ( 1.12s elapsed) EXIT time 0.00s ( 0.00s elapsed) Total time 68.09s ( 68.21s elapsed) %GC time 1.7% (1.6% elapsed) Alloc rate 1,002,835,517 bytes per MUT second Productivity 98.3% of total user, 98.2% of total elapsed gc_alloc_block_sync: 0 whitehole_spin: 0 gen[0].sync_large_objects: 0 gen[1].sync_large_objects: 0 This looks ok to me... Then if I try to use Control.Parallel to parallelise my code, simpy replacing a map with parMap (rdeepseq), on a 12 core machine using +RTS -N12 -s I get this: 66,065,148,144 bytes allocated in the heap 197,202,056 bytes copied during GC 181,312 bytes maximum residency (251 sample(s)) 387,240 bytes maximum slop 12 MB total memory in use (3 MB lost due to fragmentation) Generation 0: 37592 collections, 37591 parallel, 245.32s, 26.67s elapsed Generation 1: 251 collections, 251 parallel, 3.12s, 0.33s elapsed Parallel GC work balance: 2.41 (24219609 / 10058220, ideal 12) MUT time (elapsed) GC time (elapsed) Task 0 (worker) : 0.00s ( 0.00s) 0.00s ( 0.00s) Task 1 (worker) : 0.00s ( 0.00s) 0.00s ( 0.00s) Task 2 (worker) : 0.00s ( 17.97s) 0.00s ( 0.00s) Task 3 (worker) : 0.00s ( 19.35s) 0.00s ( 0.00s) Task 4 (worker) : 0.00s ( 40.28s) 0.00s ( 0.00s) Task 5 (worker) : 0.00s ( 45.08s) 0.00s ( 0.00s) Task 6 (worker) : 0.00s ( 47.06s) 0.00s ( 0.00s) Task 7 (worker) : 18.30s ( 49.73s) 16.24s ( 1.71s) Task 8 (worker) : 0.00s ( 51.22s) 0.00s ( 0.00s) Task 9 (worker) : 0.00s ( 53.75s) 0.00s ( 0.00s) Task 10 (worker) : 0.00s ( 54.17s) 0.00s ( 0.00s) Task 11 (worker) : 5.65s ( 54.30s) 0.70s ( 0.08s) Task 12 (worker) : 0.00s ( 54.41s) 0.41s ( 0.04s) Task 13 (worker) : 4.34s ( 54.58s) 4.50s ( 0.48s) Task 14 (worker) : 5.82s ( 54.76s) 5.91s ( 0.64s) Task 15 (worker) : 6.50s ( 55.01s) 3.37s ( 0.38s) Task 16 (worker) : 7.60s ( 55.21s) 8.56s ( 0.94s) Task 17 (worker) : 11.05s ( 55.21s) 9.04s ( 0.96s) Task 18 (worker) : 11.75s ( 55.21s) 12.94s ( 1.43s) Task 19 (worker) : 16.02s ( 55.21s) 13.32s ( 1.43s) Task 20 (worker) : 26.98s ( 55.23s) 7.43s ( 0.77s) Task 21 (worker) : 7.36s ( 55.23s) 7.47s ( 0.83s) Task 22 (worker) : 16.08s ( 55.23s) 10.25s ( 1.12s) Task 23 (worker) : 7.04s ( 55.23s) 4.99s ( 0.57s) Task 24 (worker) : 28.47s ( 55.23s) 8.78s ( 0.94s) Task 25 (worker) : 7.43s ( 55.23s) 1.62s ( 0.18s) Task 26 (worker) : 6.33s ( 55.23s) 11.42s ( 1.23s) Task 27 (worker) : 9.80s ( 55.23s) 8.72s ( 0.95s) Task 28 (worker) : 4.88s ( 55.26s) 8.92s ( 0.99s) Task 29 (worker) : 0.00s ( 55.26s) 0.00s ( 0.00s) Task 30 (bound) : 5.59s ( 55.26s) 0.59s ( 0.06s) Task 31 (worker) : 41.16s ( 55.26s) 3.48s ( 0.38s) Task 32 (worker) : 17.03s ( 55.26s) 3.90s ( 0.42s) Task 33 (worker) : 14.89s ( 55.26s) 5.29s ( 0.58s) Task 34 (worker) : 6.30s ( 55.26s) 1.99s ( 0.21s) Task 35 (worker) : 16.13s ( 55.26s) 13.95s ( 1.50s) Task 36 (worker) : 16.70s ( 55.26s) 13.02s ( 1.41s) Task 37 (worker) : 11.68s ( 55.26s) 15.45s ( 1.68s) Task 38 (worker) : 7.65s ( 55.26s) 13.61s ( 1.50s) Task 39 (worker) : 14.18s ( 55.26s) 12.48s ( 1.35s) Task 40 (worker) : 5.51s ( 55.26s) 11.92s ( 1.31s) Task 41 (worker) : 3.10s ( 55.26s) 8.17s ( 0.93s) SPARKS: 42525 (37707 converted, 519 pruned) INIT time 0.00s ( 0.00s elapsed) MUT time 365.55s ( 55.26s elapsed) GC time 248.45s ( 27.00s elapsed) EXIT time 0.00s ( 0.00s elapsed) Total time 614.01s ( 82.26s elapsed) %GC time 40.5% (32.8% elapsed) Alloc rate 180,724,403 bytes per MUT second Productivity 59.5% of total user, 444.4% of total elapsed gc_alloc_block_sync: 1339708 whitehole_spin: 36 gen[0].sync_large_objects: 197621 gen[1].sync_large_objects: 6396 It seems like suddenly GC is taking a lot longer. I also tried using Control.Monad.Par, with runPar $ do parMap, and using +RTS -N12 -s I get this: 71,480,561,920 bytes allocated in the heap 275,752,912 bytes copied during GC 198,704 bytes maximum residency (343 sample(s)) 394,232 bytes maximum slop 12 MB total memory in use (3 MB lost due to fragmentation) Generation 0: 37575 collections, 37574 parallel, 169.08s, 16.67s elapsed Generation 1: 343 collections, 343 parallel, 6.24s, 0.61s elapsed Parallel GC work balance: 2.56 (33928871 / 13275450, ideal 12) MUT time (elapsed) GC time (elapsed) Task 0 (worker) : 0.70s ( 46.46s) 0.00s ( 0.00s) Task 1 (worker) : 2.94s ( 46.46s) 0.00s ( 0.00s) Task 2 (worker) : 0.10s ( 46.46s) 0.00s ( 0.00s) Task 3 (worker) : 0.06s ( 46.47s) 0.00s ( 0.00s) Task 4 (worker) : 1.70s ( 46.47s) 0.00s ( 0.00s) Task 5 (worker) : 0.07s ( 46.47s) 0.00s ( 0.00s) Task 6 (worker) : 2.75s ( 46.47s) 0.00s ( 0.00s) Task 7 (worker) : 1.21s ( 46.47s) 0.00s ( 0.00s) Task 8 (worker) : 0.43s ( 46.47s) 0.00s ( 0.00s) Task 9 (worker) : 10.44s ( 46.47s) 0.00s ( 0.00s) Task 10 (worker) : 0.06s ( 46.47s) 0.00s ( 0.00s) Task 11 (worker) : 1.36s ( 46.47s) 0.00s ( 0.00s) Task 12 (worker) : 0.00s ( 46.50s) 46.69s ( 4.63s) Task 13 (worker) : 0.00s ( 46.50s) 0.00s ( 0.00s) Task 14 (bound) : 0.05s ( 46.50s) 0.56s ( 0.06s) Task 15 (worker) : 24.84s ( 46.50s) 2.24s ( 0.21s) Task 16 (worker) : 28.00s ( 46.50s) 1.95s ( 0.19s) Task 17 (worker) : 35.30s ( 46.50s) 2.79s ( 0.27s) Task 18 (worker) : 29.03s ( 46.50s) 4.06s ( 0.39s) Task 19 (worker) : 32.71s ( 46.50s) 7.04s ( 0.69s) Task 20 (worker) : 32.74s ( 46.50s) 11.47s ( 1.09s) Task 21 (worker) : 23.35s ( 46.50s) 15.16s ( 1.49s) Task 22 (worker) : 36.00s ( 46.50s) 7.94s ( 0.76s) Task 23 (worker) : 0.00s ( 46.50s) 35.84s ( 3.63s) Task 24 (worker) : 24.70s ( 46.50s) 21.26s ( 2.06s) Task 25 (worker) : 23.07s ( 46.50s) 18.33s ( 1.82s) SPARKS: 0 (0 converted, 0 pruned) INIT time 0.00s ( 0.00s elapsed) MUT time 307.74s ( 46.50s elapsed) GC time 175.32s ( 17.29s elapsed) EXIT time 0.00s ( 0.00s elapsed) Total time 483.06s ( 63.79s elapsed) %GC time 36.3% (27.1% elapsed) Alloc rate 232,274,501 bytes per MUT second Productivity 63.7% of total user, 482.4% of total elapsed gc_alloc_block_sync: 1619540 whitehole_spin: 258 gen[0].sync_large_objects: 232661 gen[1].sync_large_objects: 6378 This seems slightly better, but the GC time is still much bigger than I would expect it to be. I don't think it is solely because of parallel GC, since running the parallel code with +RTS -N1 gives results similar to the non-parallel version. Am I simply misunderstanding the way the GC time is represented in this output, or is there something amiss? (I also noticed that there seem to be a lot more than 12 threads running if I look at top while the program is running, but I assume this is something to do with the runtime spawning more threads for GC etc)

14 23

Proposal to incorporate Haskell.org
by Don Stewart 16 Dec '11

16 Dec '11

Hello everyone. The haskell.org committee[1], in the interest of the long-term stability of the open source Haskell community infrastructure, has decided to incorporate haskell.org as a legal entity. This email outlines our recommendation, and seeks input from the community on this decision. The committee's proposal is that haskell.org incorporate as an entity under the Software Freedom Conservancy umbrella organization (the same group that Darcs joined recently): http://sfconservancy.org/ If we proceed with this move, haskell.org will be a legal entity, and registered as a non-profit, allowing us to more directly accept (US tax-deductible) donations, and to invest in assets that benefit the Haskell open source community. We welcome your feedback on the proposal attached below. -- Don Stewart (on behalf of the Haskell.org committee) ------------------------------------------------------------------------ = A proposal for the incorporation of Haskell.org = In recent years, haskell.org has started to receive assets, e.g. money from Google Summer Of Code, donations for Hackathons, and a Sparc machine for use in GHC development. We have also started spending this money: in particular, on hosting haskell.org itself. There is also interest in running fundraising drives for specific things such as Hackathon sponsorship and hosting fees. However, haskell.org doesn't currently exist as a legal entity, meaning that these assets have had to be held on our behalf by other entities, such as Galois and various universities. This leads to tricky situations, with no-one being sure who should decide how the haskell.org assets can be used. To solve these problems, we propose that haskell.org applies to become a member project of the Software Freedom Conservancy (SFC) <http://conservancy.softwarefreedom.org/>. The SFC is a non-profit organization that provides free financial and administrative services to open source projects. Additionally, it has 501(c)(3) status, meaning donations from the US are tax-deductible. The SFC would hold haskell.org's money and other assets, and would be able to accept donations on behalf of haskell.org. The haskell.org committee, as described here [2], will make decisions on spending assets and other decisions related to governing the non-profit. Before proceeding, we are inviting input from the community in the form of specific objections or queries regarding the plan. We've tried to answer some of the most likely questions: Q: Does this mean that my Haskell project must now be covered by a copyleft licence such as GPL? A: No, but Haskell projects using haskell.org resource should use an Open Source licence <http://www.opensource.org/licenses/alphabetical>. Q: Will it still be possible to use community.h.o to host non-open-source material, such as academic papers? A: An overall minority of such content, as is the current situation, is not a problem. Q: Will it still be possible to have job ads on the haskell.org mailing lists and website? A: Yes. Q: Will this affect our ability to host the Haskell Symposium <http://www.haskell.org/haskell-symposium/> and Industrial Haskell Group<http://industry.haskell.org/> webpages within haskell.org? A: No. Q: What will be the relationship between haskell.org and other organizations such as the Haskell Symposium and Industrial Haskell Group? A: Those organisations will continue to exist as separate entities. Q: If an umbrella non-profit organisation "The Haskell Foundation" was created, would haskell.org be able to join it? A: Yes. It's likely that in such a scenario, the Haskell Foundation would become the owner of the haskell.org domain name, with the cost divided between the members. The entity that is part of the SFC would be renamed "community.haskell.org" in order to avoid confusion. [1]: http://www.haskell.org/haskellwiki/Haskell.org_committee [2]: http://www.haskell.org/haskellwiki/Haskell.org_committee#Operation

10 16

Hexpat: Lazy I/O problem with huge input files
by Aleksandar Dimitrov 10 Dec '11

10 Dec '11

Hello Haskell Cafe, I really hope this is the right list for this sort of question. I've bugged the folks in #haskell, they say go here, so I'm turning to you. I want to use Hexpat to read in some humongous XML files (linguistic corpora,) since it's the only Haskell XML library (I could find) that takes ByteStrings as input. I stumbled on a problem when using one of the examples from the docs of Text.XML.Expat.Tree. The "cookbook recipe" there suggests *first* processing the data, and only then looking into the parser error to see if there has been an error. I understand this should prevent the parse tree from being fully evaluated before use. Unfortunately, that is not what happens on my system (ghc 6.12.1, if that's of importance.) This is the code from the docs, that I modified to read files: > import Text.XML.Expat.Tree > import System.Environment (getArgs) > import Control.Monad (liftM) > import qualified Data.ByteString.Lazy as C >· > -- This is the recommended way to handle errors in lazy parses > main = do > f <- liftM head getArgs >>= C.readFile > let (tree, mError) = parse defaultParseOptions f > print (tree :: UNode String) >· > -- Note: We check the error _after_ we have finished our processing > -- on the tree. > case mError of > Just err -> putStrLn $ "It failed : "++show err > Nothing -> putStrLn "Success!" Given a 42M test file, an invocation like this: % ghc --make -O2 Hexpat.hs % ./Hexpat input.xml > dump.xml will gobble up some 2Gigs of RAM (at least. I usually kill it before it starts thrashing the swap space, since that almost crashes my entire machine.) If I remove the last 3 lines: > import Text.XML.Expat.Tree > import System.Environment (getArgs) > import Control.Monad (liftM) > import qualified Data.ByteString.Lazy as C > > main = do > f <- liftM head getArgs >>= C.readFile > let (tree, mError) = parse defaultParseOptions f > print (tree :: UNode String) the same invocation and input file barely uses a megabyte or two of RAM and finishes really quickly. Why is that? Is this a mistake in the Hexpat docs, or am I doing something wrong? Lazy IO has always been a little bit of a mystery to me, and just when I thought I had it... Thanks for any help on the matter! Aleks

3 3

doctest: Interpreter exited with an error: ExitFailure 127
by informationen 16 Nov '11

16 Nov '11

Hi, i have upgraded to doctest version 0.4.1. Now when i try to run the example from the webpage, i get: doctest: Interpreter exited with an error: ExitFailure 127 What's wrong here and how can i fix it? Kind regards Chris This is the content of Fib.hs: module Fib where -- | Compute Fibonacci numbers -- -- Examples: -- -- >>> fib 10 -- 55 -- -- >>> fib 5 -- 5 fib 0 = 0 fib 1 = 1 fib n = fib (n - 1) + fib (n - 2) main = do print $ fib 4

3 4

Re: [Haskell-cafe] Amazon AWS storage best to use with Haskell?
by dokondr 16 Nov '11

16 Nov '11

On Mon, Oct 31, 2011 at 6:50 PM, John Lenz <lenz(a)math.uic.edu> wrote: > CouchDB works great, although I decided to go with SimpleDB since then it > is amazon's problem to scale and allocate disk and so forth, which I like > better. For couchdb, you can use my package couchdb-enumerator on hackage. > > > Regarding CouchDB. So far I have my records keyed by Id and stored in Data.Map which I serialize to text file. Using Data.Map functions I do many operations with these records including mapping functions over keys and values, accumulation, lookup, intersection, union etc. When I move this data to CouchDB and start using couchdb-enumerator to work with it, how natural will it be to implement all these functions that I use from Data.Map? Or maybe it makes more sense to store my serialized Data.Map as a blob in CouchDB? And do not use views or similar CouchDB / SimpleDB interfaces at all? Just retrieve necessary blob and deserialize it to Data.Map, update and then store modified blob to CouchDB again? It would be great if somebody had time to implement Data.List, Data.Map, etc on top of generic NoSQL DB interface with specific instances for CouchDB, SimpleDB, etc.

4 9

Message
by Goutam Tmv 07 Nov '11

07 Nov '11

Would you ever see yourself write a web application like Twitter or Facebook in Haskell?

16 20

Is it possible to get the information of instances of a type?
by Magicloud Magiclouds 07 Nov '11

07 Nov '11

Hi, If this was in ruby or other languages that support reflection, it won't be a question. But in Haskell, could I write a code to list the classes that a type instanced? TemplateHaskell as well. -- 竹密岂妨流水过山高哪阻野云飞

7 9

Working with the code For "Typing Haskell In Haskell"
by Patrick LeBoutillier 02 Nov '11

02 Nov '11

Hi all, I'm working with the code that accompanies this paper (http://web.cecs.pdx.edu/~mpj/thih/) and I'm trying to use it but I can't figure out how to get started. I have the following code but it is not giving me the expected result: import TypingHaskellInHaskell mapt = "map" :>: Forall [Star, Star] ([] :=> ((TGen 0 `fn` TGen 1) `fn` TAp tList (TGen 0) `fn` TAp tList (TGen 1))) idt = "id" :>: Forall [Star] ([] :=> (TGen 0 `fn` TGen 0)) exprt = Ap (Const mapt) (Const idt) test = runTI $ tiExpr initialEnv [] exprt When I execute the test function above in ghci I get: ([],TVar (Tyvar "v3" Star)). I was expecting someting like below for the type part: TAp tList (TGen 0) `fn` TAp tList (TGen 0) What I want is the library to compute for me the type of "map id". What is the proper way to achieve this? Has anybody on the list worked with this code before? Thanks as lot, Patrick -- ===================== Patrick LeBoutillier Rosemère, Québec, Canada

5 5