September 2008 - Glasgow-haskell-users

Some initial results with DPH
by Austin Seipp 23 Sep '08

23 Sep '08

(I'm posting this here in the hope people like Manual and Roman will see it and take some interest.) I've built the GHC 6.10 beta (that is, The Glorious Glasgow Haskell Compilation System, version 6.10.0.20080921) mainly so I could play around with DPH because it's too exciting to ignore any longer. The program I'm using here is pretty trivial but it's a simple enough test case, I think. It's really cribbed from dph's examples almost entirely, I just made some superficial data sets to test on and here are the results. The code is attached in a .tar file; you'll need to tweak the Makefile to point to your GHC instead of mine if you use it. Things to note: * The vectorise pass boosts compilation times *a lot*. I don't think this is exactly unwarrented since it seems like a pretty complicated transformation, but while making the primitive version using just the unlifted interface the compilation takes about 1.5 seconds, for the vectorised version it's on the order of 15 seconds. For something as trivial as this dot-product thing, that's a bit of a compilation time, though. * It's pretty much impossible to use ghc-core to examine the output core of the vectorised version - I let it run and before anything started showing up in `less` it was already using on the order of 100mb of memory. If I just add -ddump-simpl to the command line, the reason is obvious: the core generated is absolutely huge. * For the benchmark included, the vectorised ver. spends about 98% of its time from what I can see in the GC before it dies from stack overflow. I haven't tried something like +RTS -A1G -RTS yet, though. * The vectoriser is really, really touchy. For example, the below code sample works (from DotPVect.hs): > import Data.Array.Parallel.Prelude.Int as I > > dotp :: [:Int:] -> [:Int:] -> Int > dotp v w = I.sumP [: (I.*) x y | x <- v, y <- w :] This however, does not work: > dotp :: [:Int:] -> [:Int:] -> Int > dotp v w = I.sumP [: (Prelude.*) x y | x <- v, y <- w :] That is, just using the version from Prelude causes the vectoriser to err like so: > ghc -o vect --make test.hs -fcpr-off -threaded -Odph -funbox-strict-fields -fdph-par > [1 of 2] Compiling DotPVect ( DotPVect.hs, DotPVect.o ) > *** Vectorisation error *** > Tycon not vectorised: GHC.Num.:TNum > make: *** [vect] Error 1 > This is a particularly strange occurance; reason being if we dig into the source code of the dph-par package we see that Data.Array.Parallel.Prelude.Int simply re-exports Data.Array.Parallel.Prelude.Base.Int (which is a hidden module, mind you) which is where (*) is defined, and it is defined like so: > (+), (-), (*) :: Int -> Int -> Int > (+) = (P.+) > (-) = (P.-) > (*) = (P.*) Where 'P' is the qualified Prelude import. Am I misunderstanding something here about how the dph packages are layed out? I think this is pretty correct and if so it's a really really strange happening to be honest. I also ran into a few other errs relating to the vectoriser dying - if I can find some I'll reply to this with some results. So far this all seems pretty negative, but on the flip-side... * The unlifted interface exported by the dph-prim-{par,seq} packages is wonderful and already works really well. See http://hpaste.org/10621 for an example - super low-GC time, both of my cores get used. * As I have increased the data sets for the dot-product example, my cores continue to get used and the GC time stays really, really low which is a great thing. * I've yet to hit any strange compilation err or problem when using the primitive packages. * Strictly speaking the dph-{par,seq} packages seem to expose more to code and combinators to those pieces of code using the vectorisation pass, but the combinators here are simple, they work and you get good results. * GHC hits a lot of optimizations, with ghc-core you can see thousands upon thousands of rules firing all over the place to aggressively inline/transform things! The DPH work done so far seems fantastic and I realize this is all in the works so everything I say now might not be worth anything tomorrow, and GHC 6.10 is only shipping with a very limited version of the system, but I figured some people would like to see initial results on some small test cases. I plan on exploiting this package a lot more in the future and testing it with larger computations and data sets, and when I do I'll be sure to give the feedback to you guys (once HEAD is steaming along again and the 6.10 branch has calmed.) Austin

2 2

skip -O2 via-C
by Serge D. Mechveliani 20 Sep '08

20 Sep '08

I take back my two recent reports about the effect of -O2, -fvia-C. Because 1) -O is sufficient, 2) there are too many issues that are more important than the behavior under -O2 and via-C. So, it is a good idea to save effort. Regards, ----------------- Serge Mechveliani mechvel(a)botik.ru

1 0

testing ghc-6.10-candidate
by Serge D. Mechveliani 20 Sep '08

20 Sep '08

Simon Peyton-Jones <simonpj(a)microsoft.com> wrote on 15 Sep 2008 about Instances and DoCon To: "Serge D. Mechveliani" <mechvel(a)botik.ru> CC: "cvs-ghc(a)haskell.org" <cvs-ghc(a)haskell.org> > Serge > > I take it all back. I've been persuaded that it's best to accept > DoCon-sty le instances as they appear, and I've put quite a bit of > work into making it so. > > So you should not need to change any code. Do try the HEAD > (any recent snapshot should do) if you can. All right, I have tested ghc-6.9.20080910 on docon-2.12-pre. Find the source on http://botik.ru/pub/local/Mechveliani/ghcBugs/docon-2.12-pre.zip It is put there only for testing GHC, and it has the compilation flag denotation in docon.cabal improved for newer Cabal and ghc-6.10. The tet shows the followng. 1. It accepts the DoCon-style overlapping instances (as in docon-2.11) -- in the example of ... => LinSolvRing (Fraction a) that you pointed out below. 2. But in several places DoCon has parasitic additions (similar to the below MulSemigroup (Fraction a)) to the context for overlapping instances. These places are marked in the docon-2.12-pre source with 'overlaps in ghc'. I would like to cancel them, similar as we now cancel the above MulSemigroup (Fraction a) in the context. But ghc-6.9.20080910 does not allow this. So, I wonder: what is the difference? 3. `Making' under -O2 reports for the module Det_.hs several warnings of the kind ... [25 of 83] Compiling Det_ ( Det_.hs, dist/build/Det_.o ) SpecConstr: too many specialisations for one function (see -fspec-constr-count): Function: dt{v X1E1R} [lid] Specialisations: [([sc_s1E2A{v} [lid], sc_s1E2B{v} [lid], ... What might this mean? 4. It has improved: it `makes' docon-2.12 under -O fast enough, without code explosion, the code is as fast as expected. It also `makes' the test Main under -O fast enough (unlike ghc-6.8.3). 5. The same is for -O2. But this does not gain more performance. -O2 -fvia-C also does not gain more performance, but leads to 7 times longer compilation. Anyway, I am satisfied with -O. Regards, ----------------- Serge Mechveliani mechvel(a)botik.ru ------------------------------------------------------------------- | Good news: I have cured the massive code-explosion problem you | encountere with GHC 6.8. I'm | about to commit a patch for that, so it'll be in GHC 6.10. | | However, I'm now getting a complaint about overlapping instances | (DoCon 2.11). GHC 6.10 is | being a bit more consistent about coherence. Here is one example: | | Fraction.hs:124:48: | Overlapping instances for MulSemigroup (Fraction a) | arising from a use of `inv' at Fraction.hs:124:48-57 | Matching instances: | instance [overlap ok] (GCDRing a) =3D> MulSemigroup (Fraction a) | -- Defined at Fract_.hs:438:9-46 | instance [overlap ok] MulSemigroup (Fraction Integer) | -- Defined at Fraction.hs:260:9-39 | (The choice depends on the instantiation of `a' | To pick the first instance above, use -XIncoherentInstances | when compiling the other instance declarations) | | It's absolutely right! If you wanted the instance for | (MulSemigroup (Fraction Integer)), then | the call to inv could in principle use | MulSemigroup (Fraction Integer) instance, and give | different behaviour. (See the user manual stuff about incoherent- | instances.) | | Possible solutions: | a) add -XAllowIncoherentInstances | b) make the instance declaration look like | instance (MulSemigroup (Fraction a), GCDRing a) => | LinSolvRing (Fraction a) | thereby deferring the choice [..]

2 2

Overlapping instance declarations
by Simon Peyton-Jones 19 Sep '08

19 Sep '08

| 2. But in several places DoCon has parasitic additions | (similar to the below MulSemigroup (Fraction a)) | to the context for overlapping instances. | These places are marked in the docon-2.12-pre source with | 'overlaps in ghc'. | I would like to cancel them, similar as we now cancel the above | MulSemigroup (Fraction a) in the context. | But ghc-6.9.20080910 does not allow this. | So, I wonder: what is the difference? No you can't cancel them! Consider class Foo a where fop :: a->a instance Foo [a] where ... instance Foo [Int] where .. class Bar a where bop :: a -> a instance Bar [a] where bop = fop In the Bar [a] instance, we get a constraint (Foo [a]). It's wrong to commit to the Foo [a] instance, because if you are ultimately building a Bar [Int] instance you want the Foo [Int] instance of fop! The Right Thing is to add (Foo [a]) to the context of the Bar instance thus instance Foo [a] => Bar [a] where bop = fop I'll expand the user manual (in the section about overlapping instances) to cover this point. The other situation that DoCon contains is more like this: class C a where { op1,op2 :: a -> a } instance C [Int] where... instance C a => C [a] where op1 x = op2 x ++ op2 x op2 x = ... In the C [a] instance you'll see that op1 calls op2, thereby giving rise to a C [a] constraint. I *originally thought* that the same reasoning applied as above, so we should reject the program. But applying the solution I describe above would give instance C [a], C a => C [a] where .. which is silly. Here the Right Thing is to satisfy the C [a] constraint "locally", without worrying about the overlap. This is justified because we'll only be *in* the code for op1 in the C [a] instance if we know that 'a' does not match Int. Thanks for bringing this up. Simon

2 1

Type binders in rules
by Simon Peyton-Jones 19 Sep '08

19 Sep '08

Friends This is a message for people who use RULES, to ask your opinion. Have a look at http://hackage.haskell.org/trac/ghc/ticket/2600 and add your comments if you want. The intro to the ticket appears below, so you can get an idea of whether you are interested. Simon Roman writes: I found an example I came across while working on the recycling paper and which I subsequently forgot about. Suppose we have: {-# LANGUAGE Rank2Types #-} module T where class T t where to :: [a] -> t a from :: t a -> [a] tmap :: (a -> a) -> t a -> t a {-# RULES "myrule" forall f x. from (tmap f (to x)) = map f (from (to x)) #-} Alas, this fails with: Ambiguous type variable `t' in the constraint: `T t' arising from a use of `to' at T.hs:12:40-43 Probable fix: add a type signature that fixes these type variable(s) Of course, I'd like the t on the rhs to be the same as on the lhs but I don't see how to tell this to GHC. Is it actually possible? The only solution I've found was to add a dummy argument to 'to': to' :: t a -> [a] -> t a to = to' undefined {-# RULES "myrule" forall f t x. from (tmap f (to' t x)) = map f (from (to' t x)) #-} That's ugly, of course. Am I missing something or is this just impossible to do with the current system?

1 0

-optl-s (strip) by default
by Bulat Ziganshin 18 Sep '08

18 Sep '08

Hello glasgow-haskell-users, when GHC builds executable, it adds debug info by default. since this info is useless for Haskell and since it significantly increases executable size, will it be better to delete it by default by passing -optl-s option to the linker? another question: by adding " -optl-Xlinker -optl--large-address-aware" to cmdline we will allow mingw32-compiled programs to use up to 4gb of memory (instead of 2gb with current settings). now i have 4gb of memory+vista64 so can test this setting if required -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

4 6

Who defines __GLASGOW_HASKELL__, __HADDOCK__, etc?
by Claus Reinke 18 Sep '08

18 Sep '08

So far, I had assumed that every tool defined its own macro, but it seems that __GLASGOW_HASKELL__ is defined by ghc and by cabal, while __HADDOCK__ is defined only by the latter. Is that right? Context is http://trac.haskell.org/haddock/ticket/48 , ie, how to adapt sources that had to bypass or help along haddock1 for haddock2 (which should need fewer workarounds). Claus

2 1

simple questions about +RTS -s output
by Peter Hercek 18 Sep '08

18 Sep '08

Hi, There is an example of +RTS -s output at the end. I have few simple questions: What does "9 Mb total memory in use" mean? Is it in mega bytes (MB) or in mega bits (Mb)? I would expect memory usage to be in bytes (B) but a unit for bits (b) seems to be used. Looks like heap size in mega bytes... What does MUT abbreviation stand for? Peter. 1,175,063,464 bytes allocated in the heap 25,951,288 bytes copied during GC (scavenged) 929,036 bytes copied during GC (not scavenged) 3,932,160 bytes maximum residency (4 sample(s)) 2237 collections in generation 0 ( 0.15s) 4 collections in generation 1 ( 0.04s) 9 Mb total memory in use INIT time 0.00s ( 0.00s elapsed) MUT time 2.95s ( 20.33s elapsed) GC time 0.19s ( 0.20s elapsed) EXIT time 0.00s ( 0.00s elapsed) Total time 3.15s ( 20.53s elapsed) %GC time 6.0% (1.0% elapsed) Alloc rate 397,455,026 bytes per MUT second Productivity 93.9% of total user, 14.4% of total elapsed

3 2

RE: Adding (some) libraries to a GHC tree
by Jost Berthold 17 Sep '08

17 Sep '08

Simon Peyton-Jones wrote: > great. Can you just do the merge? You are hereby appointed an "accredited > wiki maintainer"! Thank you :) I edited GettingTheSources as we said, and removed the links from the /Building page. Neil, if you think the extra links should be kept, sorry and please revert. Jost

3 2

RE: Adding (some) libraries to a GHC tree
by Jost Berthold 17 Sep '08

17 Sep '08

Mitchell, Neil wrote: > Hi > > The paragraph in the new page is better than the old one, but it makes > sense to have the information in only one place. When looking for the > information "getting the sources" didn't occur to me as a description of > how to get any extra libraries (although on reflection, I'm surprised I > didn't try it). > > Thanks > > Neil Hi everybody, all of you are right! :) The information I have added should definitely be close to where --extra is explained. However, the overlap is not that big: "Getting more packages" talks about getting a 'package of packages', all extras in one go. What I added is information about not getting everything, but exactly the few lib.s which you miss. (another question is, how many people would want that...) As a result: I suggest my paragraph should be put right below the explanation of darcs --extra, perhaps shortened accordingly. The only reason I made it a (tiny) new page is that I saw the extra section "contributed doc.", so I thought I should put my contribution here to be moved into more "official" pages by other wiki maintainers (i.e. Neil in this case) later. Jost >> -----Original Message----- >> From: Simon Peyton-Jones [mailto:simonpj@microsoft.com] >> Sent: 17 September 2008 1:40 pm >> To: Mitchell, Neil; Jost Berthold >> Cc: glasgow-haskell-users(a)haskell.org >> Subject: RE: Adding (some) libraries to a GHC tree >> >> good stuff -- but with a big overlap with the "Getting more >> packages" section in >> http://hackage.haskell.org/trac/ghc/wiki/Building/GettingTheSo >> urces. Do you agree? Would it be possible to merge your new >> paragraphs into the existing page? >> >> If the existing page is too big and hard to navigate we could >> break it up. >> >> >> S >> | -----Original Message----- >> | From: glasgow-haskell-users-bounces(a)haskell.org >> | [mailto:glasgow-haskell-users- bounces(a)haskell.org] On Behalf Of >> | Mitchell, Neil >> | Sent: 17 September 2008 10:50 >> | To: Jost Berthold >> | Cc: glasgow-haskell-users(a)haskell.org >> | Subject: RE: Adding (some) libraries to a GHC tree >> | >> | Hi Jost, >> | >> | Many thanks, that's perfect. >> | >> | I moved the page to >> | http://hackage.haskell.org/trac/ghc/wiki/Building/AddingLibs , and >> | referenced it from http://hackage.haskell.org/trac/ghc/wiki/Building >> | >> | Thanks >> | >> | Neil >> | >> | > -----Original Message----- >> | > From: Jost Berthold [mailto:berthold@Mathematik.Uni-Marburg.de] >> | > Sent: 17 September 2008 10:36 am >> | > To: Mitchell, Neil >> | > Cc: glasgow-haskell-users(a)haskell.org >> | > Subject: Adding (some) libraries to a GHC tree >> | > >> | > Hi Neil, >> | > >> | > You can manually add any package which is listed in >> | > http://hackage.haskell.org/trac/ghc/wiki/DarcsRepositories >> | > by getting it (with darcs), directly into the /libraries >> subdirectory. >> | > When you later use darcs-all, the added library will be included >> | > when pulling new changes etc. >> | > >> | > I have added a wiki page to explain this: >> | > http://hackage.haskell.org/trac/ghc/wiki/AddingLibsToBuild >> | > (feel free to remove it and add the material to a more convenient >> | > place) >> | > >> | > HTH >> | > Jost >> | > >> | > >> | > >> | >> | >> ====================================================================== >> | ======== Please access the attached hyperlink for an important >> | electronic communications disclaimer: >> | >> | http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html >> | >> ====================================================================== >> | ======== >> | >> | _______________________________________________ >> | Glasgow-haskell-users mailing list >> | Glasgow-haskell-users(a)haskell.org >> | http://www.haskell.org/mailman/listinfo/glasgow-haskell-users >> >> >> > > ============================================================================== > Please access the attached hyperlink for an important electronic > communications disclaimer: > > http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html > ============================================================================== > >

2 1