- Glasgow-haskell-users

Building GHC 6.12 with GHC 6.12 + alex
by Antoine Latter 05 Jan '10

05 Jan '10

Folks, I've run into trouble building GHC 6.12 with alex built from GHC 6.12: "/Users/alatter/usr/bin/alex" -g compiler/parser/Lexer.x -o compiler/stage1/build/Lexer.hs alex: compiler/parser/Lexer.x: hGetContents: invalid argument (Illegal byte sequence) however things like: ghci> readFile "compiler/parser/Lexer.x" apear to work just fine. Has anyone else seen this? Has anyone else succeeded in this? This is on Mac OS X 10.6, running on Intel Core 2 Duo. I'm trying to compile a recent pull of the 6.12 branch. Thanks, Antoine

2 5

Where did the GHC API go?
by Jason Dusek 04 Jan '10

04 Jan '10

Maybe I missed an email about this... http://www.haskell.org/ghc/docs/latest/html/libraries/ghc/index.html The link is down. -- Jason Dusek

9 19

GHC.Prim.ByteArray# - confusing documentation
by Antoine Latter 01 Jan '10

01 Jan '10

Folks, I found some of the documentation in GHC.Prim confusing - so I thought I'd share. The documentation for the ByteArray# type[1] explains that's it's a raw region in memory that also remembers it's size. Consequently I expected sizeOfByteArray# to return the same number that I passed in to newByteArray#. But it doesn't - It returned however much it decided to allocate, which on my platform is always a multiple of four bytes. This is something which could be clarified in the documentation. So it turns out I need to carry my own length around in something like (Int, ByteArray#). (I tested all of this with the primitive[2] package, which is a thin wrapper around GHC.Prim) Antoine 1: http://hackage.haskell.org/packages/archive/ghc-prim/0.1.0.0/doc/html/GHC-P… 2: http://hackage.haskell.org/package/primitive

4 6

Re: Re: ANN: Happstack 0.4.1
by Antoine Latter 01 Jan '10

01 Jan '10

On Mon, Dec 21, 2009 at 6:31 AM, <jeremy(a)n-heptane.com> wrote: > Hello, > > That sort of missing symbol error at link time is often (but not always) a > sign that some libraries got recompiled but not others. So there are > references to the old symbol names hanging around. > > I would try to ghc-pkg unregister syb-with-class and everything that depends > on it, and then try cabal install happstack again. > I'm pretty well stumped at this point. I've cleared off everything and gone up to GHC 6.12 HEAD, and a 'cabal install happstack-data' gives me the same symbol not defined error in Happstack.Data.Xml.Base. But here's the spooky part, if I run it by hand like so: ghc --make src/Happstack/Data/Xml/Base.hs src/Happstack/Data/Default.hs src/Happstack/Data/ DeriveAll.hs src/Happstack/Data/Normalize.hs src/Happstack/Data/Migrate.hs after resolving issues due to CPP not being run, everything runs to completion, no errors. Also, the list of things we're pulling in during the template-haskell execution is much smaller (see bellow). Has anyone seen this, where template-haskell behaves different when run from cabal-install (or Setup.hs) than from ghc --make (or ghci)? >>>>> 2 of 5] Compiling Happstack.Data.Migrate ( src/Happstack/Data/Migrate.hs, src/Happstack/Data/Migrate.o ) [3 of 5] Compiling Happstack.Data.Default ( src/Happstack/Data/Default.hs, src/Happstack/Data/Default.o ) [4 of 5] Compiling Happstack.Data.DeriveAll ( src/Happstack/Data/DeriveAll.hs, src/Happstack/Data/DeriveAll.o ) [5 of 5] Compiling Happstack.Data.Xml.Base ( src/Happstack/Data/Xml/Base.hs, src/Happstack/Data/Xml/Base.o ) Loading package ghc-prim ... linking ... done. Loading package integer-gmp ... linking ... done. Loading package base ... linking ... done. Loading package ffi-1.0 ... linking ... done. Loading package array-0.3.0.0 ... linking ... done. Loading package bytestring-0.9.1.5 ... linking ... done. Loading package containers-0.3.0.0 ... linking ... done. Loading package pretty-1.0.1.1 ... linking ... done. Loading package syb-0.1.0.2 ... linking ... done. Loading package template-haskell ... linking ... done. Loading package syb-with-class-0.6.1 ... linking ... done. mkUsageInfo: internal name? Element{tc a4av} <<<<< That was the successful build, using ghc --make. The error for te failing build is quoted bellow. Antoine > > > > On Sun 12/20/2009 2:06 AM , Antoine Latter aslatter(a)gmail.com sent: > > 2009/12/19 Jeremy Shaw <jeremy(a)n-heptane.com>: >> Happstack 0.4.1 STABLE is now available. > > At some point I was able to get happstack to build on GHC 6.12, but now I > can't. > > I'm getting errors in happstack-data, specifically it looks like > errors when I'm running TemplateHaskell during compile-time: > >>>>>> > [ 7 of 16] Compiling Happstack.Data.Xml.Base ( > src/Happstack/Data/Xml/Base.hs, dist/build/Happstack/Data/Xml/Base.o ) > Loading package ghc-prim ... linking ... done. > Loading package integer-gmp ... linking ... done. > Loading package base ... linking ... done. > Loading package array-0.3.0.0 ... linking ... done. > Loading package bytestring-0.9.1.5 ... linking ... done. > Loading package containers-0.3.0.0 ... linking ... done. > Loading package pretty-1.0.1.1 ... linking ... done. > Loading package template-haskell ... linking ... done. > Loading package syb-with-class-0.6.1 ... linking ... done. > Loading package HUnit-1.2.2.1 ... linking ... done. > Loading package syb-0.1.0.2 ... linking ... done. > Loading package base-3.0.3.2 ... linking ... done. > Loading package old-locale-1.0.0.2 ... linking ... done. > Loading package time-1.1.4 ... linking ... done. > Loading package random-1.0.0.2 ... linking ... done. > Loading package QuickCheck-1.2.0.0 ... linking ... done. > Loading package extensible-exceptions-0.1.1.1 ... linking ... done. > Loading package mtl-1.1.0.2 ... linking ... done. > Loading package old-time-1.0.0.3 ... linking ... done. > Loading package parsec-2.1.0.1 ... linking ... done. > Loading package hsemail-1.3 ... linking ... done. > Loading package network-2.2.1.5 ... linking ... done. > Loading package SMTPClient-1.0.1 ... linking ... done. > Loading package filepath-1.1.0.3 ... linking ... done. > Loading package unix-2.4.0.0 ... linking ... done. > Loading package directory-1.0.1.0 ... linking ... done. > Loading package process-1.0.1.2 ... linking ... done. > Loading package hslogger-1.0.7 ... linking ... done. > Loading package deepseq-1.1.0.0 ... linking ... done. > Loading package parallel-2.2.0.1 ... linking ... done. > Loading package strict-concurrency-0.2.2 ... linking ... done. > Loading package unix-compat-0.1.2.1 ... linking ... done. > Loading package happstack-util-0.4.1 ... linking ... done. > Loading package binary-0.5.0.2 ... linking ... done. > Loading package haskell98 ... linking ... done. > Loading package HaXml-1.13.3 ... linking ... done. > Loading package ffi-1.0 ... linking ... done. > ghc: > unknown symbol > `_sybzmwithzmclasszm0zi6zi1_DataziGenericsziSYBziWithClassziInstances_dataTypeZMad6eZN_closure' > <<<<< > > If you check the load-list, we are loading syb-with-class. > > This is on Mac OS X 10.6 on a 64-bit intel chip, if that makes any > difference. > > Has anyone seen anything like this? Is it likely I've screwed up my > GHC install somehow? > > Antoine > > -- > > You received this message because you are subscribed to the Google Groups > "HAppS" group. > To post to this group, send email to happs(a)googlegroups.com. > To unsubscribe from this group, send email to > happs%2Bunsubscribe(a)googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/happs?hl=en. > > > >

3 2

GHC-6.12.1: broken configure
by Kirill A. Shutemov 30 Dec '09

30 Dec '09

I want to build ghc for i586-alt-linux-gnu. > ./configure --host=i586-alt-linux-gnu --build=i586-alt-linux-gnu checking for gfind... no checking for find... /bin/find checking for sort... /bin/sort checking for ghc... /usr/bin/ghc checking version of ghc... 6.10.1 Target platform inferred as: i386-unknown-linux Unknown arch i586 Why i586 is unknown? It's a valid architecture. Why does it check vendor? Many of distribution define own vendor. Why do not use config.guess to guess correct host/target/build instead of reinvent wheel? Please fix it. It will simplify packaging GHC for distributions.

5 5

Re: [Haskell-beginners] Performance of parallel mergesort
by Yitzchak Gale 30 Dec '09

30 Dec '09

This discussion definitely does not belong on the Haskell-Beginners list. Besides not being a topic for beginners, being there is keeping it under the radar of many or most of the people who work on these things in Haskell. I am moving the discussion to the GHC users list, as suggested by Antoine. For those joining us in progress, the first part of the thread starts here: http://www.haskell.org/pipermail/beginners/2009-December/003045.html On Mon, Dec 28, 2009 at 5:19 AM, Jon Harrop wrote: > On Sunday 27 December 2009 20:56:51 Stephen Blackheath wrote: >> Jon Harrop wrote: >> > This is something that concerns me. Lots of discussions of parallelism, >> > including the Haskell literature I have read, neglect this critical >> > problem >> > of making sure that more time is spent doing useful work than spawning >> > tasks >> > (sparks). How is this solved in Haskell? Do you just put magic numbers in >> > that work on the machine you're currently using? >> >> It is simply not true that Haskell literature neglects the question of >> spark granularity - this is very basic and is always covered. Read >> "Real World Haskell" (available free online). There's no 'magic >> number'. You must explicitly write your code to give the right granule >> size. > > There is no "right granule" size. That's the whole point: the optimum is a > function of the machine. If you hardcode the granularity then your code isn't > future proof and isn't portable. > > From chapter 24 of Real World Haskell on sorting: > > "At this fine granularity, the cost of using par outweighs any possible > usefulness. To reduce this effect, we switch to our non-parallel sort after > passing some threshold." > > From the Sorting.hs file, parSort2 accepts a threshold "d": > > parSort2 :: (Ord a) => Int -> [a] -> [a] > parSort2 d list@(x:xs) > | d <= 0 = sort list > | otherwise = force greater `par` (force lesser `pseq` > (lesser ++ x:greater)) > where lesser = parSort2 d' [y | y <- xs, y < x] > greater = parSort2 d' [y | y <- xs, y >= x] > d' = d - 1 > parSort2 _ _ = [] > > From the SortMain.hs file, it is always invoked with the magic number "2": > > testFunction = parSort2 2 > > Moreover, their approach of subdividing a fixed number of times is suboptimal > because it inhibits load balancing. > > Later, about parallelized IO, they give the code: > > chunkedReadWith :: (NFData a) => > ([LB.ByteString] -> a) -> FilePath -> IO a > chunkedReadWith func path = > withChunks (lineChunks (numCapabilities * 4)) func path > > where "4" is one magic number that gets multiplied by the magic number the > user supplied via the +RTS -N<n> command-line option. > > They make no attempt to adapt the granularity to the machine at all and rely > entirely upon magic numbers. Consequently, their parallel sort that got a 25% > speedup on two cores achieves a 30% slowdown on my 8 core. > >> I don't know the exact cost of sparking, but in my experience it >> is quite small - so - as long as your granule is doing *some* real work, >> it should speed up. > > Can you quantify it, e.g. How many FLOPS? > >> Jon Harrop wrote: >> > The parallelism is obviously not obtaining any real speedup so something >> > is wrong. But can anyone fix it? >> >> I've spent a bit of time measuring parallel speedup on real commercial >> projects, and this is what I've found: >> >> 1. ghc-6.12 performs significantly better than ghc-6.10, and has now >> been released, therefore don't use ghc-6.10. > > Ok. > >> 2. The defaults generally work better than giving huge heap sizes. Your >> -K100000000 - maximum heap size per thread - will either do nothing or >> cause an artificial slowdown (I have observed this with the minimum heap >> size option). Don't use it, unless experimentation proves it makes >> things better. > > On the contrary, the Haskell program dies without it: > > $ time ./mergesort +RTS -N8 > Stack space overflow: current size 8388608 bytes. > Use `+RTS -Ksize' to increase it. > [1]+ Done kwrite mergesort.hs > > real 0m33.320s > user 3m29.397s > sys 0m0.592s > > I had to add that -K command line option just to get the program to run to > completion. > >> 3. +RTS -s is well worth using. It breaks the time down into MUT >> (mutator) and GC (garbage collector). > > Its says 78.5% GC time (with GHC 6.10). > >> 4. MUT parallelization is excellent, but the parallel GC is not so good. >> If your code is GC-heavy it can spend around half of its time garbage >> collecting, which doesn't parallelize so well, and this eats into the >> speedup. > > Ok. > >> 5. There seems to be a scalability problem with the parallel gc for >> larger numbers of cores (namely 8). I am guessing somewhat, but my >> experiments tend to confirm the issue raised in Simon Marlow's (the >> implementor of GHC parallelization) recent paper that it's to do with >> "stopping the world" for gc. > > Do you mean this bug: > > http://hackage.haskell.org/trac/ghc/ticket/3553 > >> If GHC's lack of perfection at this point in time makes Haskell "look >> bad" I don't mind. I am not selling anything, so the reader at least >> knows they're getting the truth. I see this as one of the great >> advantages of open source. > > I'm sure we'd all rather see speedups. :-) > >> Progress on GHC has been very rapid in the last couple of years, and so >> I know we'll continue to see the speed of GHC's parallelism improving in >> leaps and bounds. It's actually still quite a new area, considering the >> difficulty of some of the technical issues and how recent it is that >> multicores are widely available on consumer hardware. > > My original intent was to test a claim someone made: that mergesort in Haskell > is competitively performant and trivially parallelized. > >> I know you OCaml/F# guys are making great progress too. > > F# is production ready but OCaml is dead in the water when it comes to > multicore. I'm working on an OCaml replacement called HLVM but it is early > days yet. > > -- > Dr Jon Harrop, Flying Frog Consultancy Ltd. > http://www.ffconsultancy.com/?e > _______________________________________________ > Beginners mailing list > Beginners(a)haskell.org > http://www.haskell.org/mailman/listinfo/beginners >

4 4

hsc2hs and base
by Bryan O'Sullivan 29 Dec '09

29 Dec '09

Hi, Simon — Is it licit to have source files in base depend on preprocessing by hsc2hs? My suspicion is "no", but I want to be sure. Thanks, Bryan.

3 3

Qualified names in import lists
by Niklas Broberg 28 Dec '09

28 Dec '09

Hi all, I have a bug report [1] for haskell-src-exts pertaining to the use of qualified names in import specifications, e.g. module Main where import Foo (Bar.bar) GHC apparently accepts this code, but I can find no mention of such a feature in the GHC docs. Personally I don't see why this should be allowed at all, as it breaks the abstraction layer w.r.t. re-exporting names from other, possibly internal, modules. If there's some reasonable use for it, it should at the very least be tied to a documented and registered extension. I've submitted a ticket for GHC [2] to either remove this feature, or properly document it. I'd be curious to hear what the reasoning behind it is, if any. Cheers, /Niklas ps. Why is there no "GHC accepts invalid program" option for "Type of failure"? Too few cases? I set this ticket to Other. [1] http://trac.haskell.org/haskell-src-exts/ticket/57 [2] http://hackage.haskell.org/trac/ghc/ticket/3792

2 2

Poor array behaviour
by Herk, Robert van 23 Dec '09

23 Dec '09

Hi all, I stumbled upon something odd with respect to arrays. I know about GHC not doing card marking and traversing whole arrays one each GC for each array with alterations, but still I don't understand the behaviour. The situation is like this. I am building a compiler. This compiler manipulates a large graph (for optimizations, etc.). This graph is in memory. As the graph is vast, we did a lot of effort to optimize access to it. The nodes in the graph are IORefs to some data structure, and this data structure contains its edges. Each node stores its edges in buckets. This is because edges have different types (some are control flow links, others are other types of dependencies). Most of the graph manipulations only traverse over one type of edge, so we figured it would be faster to store the edges in buckets to support such queries. Inside the buckets, there are Data.Maps containing the actual edges for that bucket. The keys in this map are the target nodes of that edges, which are IORefsOrds, which are pairs of a unique integers and a IORef, such that they can be ordered and used as keys in a Map. The values are lists of edges to that target. The weird thing is in the buckets. Per node, all buckets are stored in an array. We gave each edge type an integer key. And we use that key as array index to determine the bucket. I've tried implementing this array in two ways: 1) Each node contains a immutable array with IORefs. The IORefs contain the actual Data.Maps for the buckets. So, for instance, initializing a node looks something like import Data.Array uBucket = 8 //There are 8 buckets because there are 8 types of edges. emptyEdges = do buckets <- sequence ( take uBucket $ repeat (newIORef Map.empty) ) let myArray = listArray (0, 7) buckets return myArray So in this solution we have an extra layer of indirection, namely the IORefs, but the array is immutable. Because when the edges change for a particular node, we can write directly into the IORef and leave the array untouched. 2) Each node contains a mutable array that contains the Data.Maps directly. So, for instance, initializing a node looks something like: import Data.Array.IO emptyEdges = do myArray <- newArray (0, 7) Map.empty return $! myArray Of course, one expect 2 to be quickest. However, it turns out that for 2, the application is spending much more time in GC, and __even without GC__ the application is still slower! I find both things rather weird: I know that for huge arrays I am expected to suffer from the missing card-marking "bug", but my array sizes are only 8. Yet the difference I get in GC time are huge: Solution 1 ------------- 13,476,008,560 bytes allocated in the heap 1,714,767,712 bytes copied during GC 151,518,528 bytes maximum residency (23 sample(s)) 1,743,176 bytes maximum slop 325 MB total memory in use (2 MB lost due to fragmentation) Generation 0: 25689 collections, 0 parallel, 2.43s, 2.53s elapsed Generation 1: 23 collections, 0 parallel, 1.89s, 2.05s elapsed INIT time 0.00s ( 0.00s elapsed) MUT time 20.58s ( 20.80s elapsed) GC time 4.32s ( 4.58s elapsed) RP time 0.00s ( 0.00s elapsed) PROF time 0.00s ( 0.00s elapsed) EXIT time 0.00s ( 0.00s elapsed) Total time 24.90s ( 25.38s elapsed) %GC time 17.4% (18.0% elapsed) Alloc rate 654,751,131 bytes per MUT second Productivity 82.6% of total user, 81.1% of total elapsed Solution 2 ------------ 15,901,133,296 bytes allocated in the heap 9,083,063,848 bytes copied during GC 117,501,208 bytes maximum residency (23 sample(s)) 1,902,568 bytes maximum slop 265 MB total memory in use (2 MB lost due to fragmentation) Generation 0: 30315 collections, 0 parallel, 44.73s, 44.89s elapsed Generation 1: 23 collections, 0 parallel, 1.78s, 1.91s elapsed INIT time 0.00s ( 0.00s elapsed) MUT time 25.93s ( 26.31s elapsed) GC time 46.51s ( 46.80s elapsed) RP time 0.00s ( 0.00s elapsed) PROF time 0.00s ( 0.00s elapsed) EXIT time 0.00s ( 0.00s elapsed) Total time 72.43s ( 73.11s elapsed) %GC time 64.2% (64.0% elapsed) Alloc rate 613,325,569 bytes per MUT second Productivity 35.8% of total user, 35.5% of total elapsed Is the behaviour I am seeing to be expected? And if so, wouldn't it make more sense to implement Data.Array.IO internally such that it contains an immutable array of IORefs? I also saw that in the next GHC, card marking will be done per 128 array items. Yet this behaviour seems to point out that at least for my application problems can already occur with array size 8. And is it expected that solution 1), that has an extra layer of indirection, still outperforms solution 2) even with GC times substracted? Regards, Robert The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.

2 1

Weird warnings when using ViewPatterns
by Bas van Dijk 23 Dec '09

23 Dec '09

Hello, In my usb-safe[1] library I make extensive use of the ViewPatterns[2] language extension. However I get strange warnings when using them. See for example the following function: resetDevice ∷ (pr `ParentOf` cr, MonadIO cr) ⇒ RegionalDeviceHandle pr → cr () resetDevice (internalHandle → DeviceHandle { internalDevHndl , configAlreadySetMVar }) = ... where: type RegionalDeviceHandle r = RegionalHandle USB.Device r internalHandle ∷ RegionalHandle resource r → Handle resource and: 'Handle' is an associated data type with the following definition: data Handle USB.Device = DeviceHandle { internalDevHndl ∷ USB.DeviceHandle , configAlreadySetMVar ∷ MVar Bool } Note that I also use the NamedFieldPuns[3] language extension. When compiling the module I get lots of warnings like this: System/USB/Safe.hs:372:0: Warning: Pattern match(es) are overlapped In the definition of `resetDevice': resetDevice ((internalHandle -> (DeviceHandle {internalDevHndl, configAlreadySetMVar}))) = ... System/USB/Safe.hs:372:0: Warning: Pattern match(es) are non-exhaustive In the definition of `resetDevice': Patterns not matched: I don't understand which patterns are overlapped or which patterns are not matched. Is this a GHC bug? Note that I'm using ghc-6.10.4 so it's possible that it's fixed in 6.12.1. regards, Bas [1] usb-safe: http://hackage.haskell.org/package/usb-safe-0.4 or darcs get http://code.haskell.org/~basvandijk/code/usb-safe [2] http://www.haskell.org/ghc/docs/latest/html/users_guide/syntax-extns.html#v… [3] http://www.haskell.org/ghc/docs/latest/html/users_guide/syntax-extns.html#r…

2 3