November 2005 - Glasgow-haskell-users

HSQL installation woes
by Ed Osinski 06 Nov '05

06 Nov '05

Hi, I'm trying to install GHC and related software under Solaris 9 and am stuck at HSQL. I initially used the binary package of ghc 6.4.1. It installed w/o any problems, and I was able to build alex, happy, haddock and cpphs. However, when trying to install HSQL-1.6, it complained about "ld" not recognizing "-x". I guess this is due to the binary package for Solaris being built on a machine that used the GNU linker, which recognizes "-x"? Anyway, this led me to build ghc 6.4.1 from source, with gcc 4.0.2, which was successful. :-) However, when I try to install HSQL-1.6, it dies during the build step. :-( Here is the output: /sware/build/HSQL-1.6/HSQL |> runghc Setup.lhs configure Warning: No license-file field. Configuring hsql-1.6... configure: searching for ghc in path. configure: found ghc at /usr/local/bin/ghc configure: looking for package tool: ghc-pkg near compiler in /usr/local/bin/ghc configure: found package tool in /usr/local/bin/ghc-pkg configure: Using install prefix: /usr/local configure: Using compiler: /usr/local/bin/ghc configure: Compiler flavor: GHC configure: Compiler version: 6.4.1 configure: Using package tool: /usr/local/bin/ghc-pkg configure: Using haddock: /usr/local/bin/haddock configure: Using happy: /usr/local/bin/happy configure: Using alex: /usr/local/bin/alex configure: Using hsc2hs: /usr/local/bin/hsc2hs configure: Using cpphs: /usr/local/bin/cpphs configure: Reading installed packages... configure: Dependency base-any: using base-1.0 /sware/build/HSQL-1.6/HSQL |> runghc Setup.lhs build Preprocessing library hsql-1.6... Building hsql-1.6... Chasing modules from: Database.HSQL,Database.HSQL.Types Compiling Database.HSQL.Types ( ./Database/HSQL/Types.hs, dist/build/./Database/HSQL/Types.o ) Compiling Database.HSQL ( ./Database/HSQL.hs, dist/build/./Database/HSQL.o ) *** Exception: waitForProcess: interrupted (Interrupted system call) This error happens consistently, but sometimes one line earlier. Googling reveals that at least one person encountered this exception on Solaris in the recent past: http://www.mail-archive.com/glasgow-haskell-users@haskell.org/msg07794.html Is this a showstopper, or is there some fix or workaround? - Ed

1 0

[ ghc-Feature Requests-1349036 ] Compiling multiple executables with -make
by SourceForge.net 05 Nov '05

05 Nov '05

Feature Requests item #1349036, was opened at 2005-11-05 15:20 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=358032&aid=1349036&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Priority: 5 Submitted By: Mads Lindstrøm (supermule) Assigned to: Nobody/Anonymous (nobody) Summary: Compiling multiple executables with -make Initial Comment: Hi I am very happy with the -make option. However, I think it could be improved even further. I would like ghc to be able to compile multiple executables with one invocation of ghc, using the -make option. Compiling multiple executables in one invokation could safe a lot of compile-time, when the executables shares most of their source files. I believe the time would be saved, as it would remove the need to check all source files for possible recompilation for each executable. Furthermore, we would also only need to build the dependency graph once. /Mads Lindstrøm ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=358032&aid=1349036&group_…

1 0

RE: Silly IO problem
by Simon Marlow 04 Nov '05

04 Nov '05

On 03 November 2005 12:57, skaller wrote: > On Thu, 2005-11-03 at 09:49 +0100, Mirko Rahn wrote: >>> It prints n, rather than tak(3n,2n,n). >> >> ??? No: >> >> *Main> :set args 0 >> *Main> main >> 0.0 >> *Main> :set args 1 >> *Main> main >> 2.0 >> *Main> :set args 2 >> *Main> main >> 3.0 >> *Main> :set args 3 >> *Main> main >> 6.0 >> >> The same with some versions of ghc. > > Ah .. it seems to be a bug in gcc 4.0 > > The Glorious Glasgow Haskell Compilation System, version 6.4 > [running on Ubuntu which uses Debian package] > > gcc (GCC) 4.0.2 20050808 (prerelease) (Ubuntu 4.0.1-4ubuntu9) > x86_64 > > Summary: gcc does not work with GHC 6.4 on x86_64 with > optimisation higher than -O1 > > Perhaps the Evil Mangler is not working, or, possibly > the version isn't even registered. Thanks for the report, I'll look into gcc 4.x before we release 6.4.2. Cheers, Simon

1 0

RE: Rules for use of unsafeThaw...
by Simon Marlow 04 Nov '05

04 Nov '05

On 03 November 2005 17:08, Jan-Willem Maessen wrote: > I've recently been experimenting with unsafeFreeze/unsafeThaw in > GHC. Judicious use of these functions vastly reduces GC overhead in > Data.HashTable. However, a slightly mis-timed GC will cause the > whole mess to crash. I am attempting to understand the invariants > required to safely use unsafeFreeze/unsafeThaw. > > I believe the following usage ought to be 100% safe: > > 1) Take the last and only reference to a mutable array, and call > unsafeFreeze to obtain an immutable array. > > 2) Take the last and only reference to an immutable array, and call > unsafeThaw to obtain a mutable array. I would like it to be the case that no usage of unsafe{Freeze,Thaw} can crash the GC. That perhaps isn't true right now, but I'd like to understand why not. Multiple unsafeThaws on the same array might cause problems in 6.4.1 (but not the HEAD), due to the way the old generation mutable list works. That's fixable, if it is the problem. Multiple unsafeFreezes should be harmless. Can you give me a test program? Cheers, Simon

1 0

Rules for use of unsafeThaw...
by Jan-Willem Maessen 04 Nov '05

04 Nov '05

I've recently been experimenting with unsafeFreeze/unsafeThaw in GHC. Judicious use of these functions vastly reduces GC overhead in Data.HashTable. However, a slightly mis-timed GC will cause the whole mess to crash. I am attempting to understand the invariants required to safely use unsafeFreeze/unsafeThaw. I believe the following usage ought to be 100% safe: 1) Take the last and only reference to a mutable array, and call unsafeFreeze to obtain an immutable array. 2) Take the last and only reference to an immutable array, and call unsafeThaw to obtain a mutable array. Are there any other usage modes which are GC-safe? In particular, is it possible for a reference to the frozen array to exist somewhere on the heap when a call to unsafeThaw occurs? Is there a particular set of circumstances under which we might get away with this (eg, no allocation during the interval where the ambiguous references occur)? -Jan-Willem Maessen

2 2

Re: Silly IO problem
by skaller 03 Nov '05

03 Nov '05

On Thu, 2005-11-03 at 09:49 +0100, Mirko Rahn wrote: > > It prints n, rather than tak(3n,2n,n). > > ??? No: > > *Main> :set args 0 > *Main> main > 0.0 > *Main> :set args 1 > *Main> main > 2.0 > *Main> :set args 2 > *Main> main > 3.0 > *Main> :set args 3 > *Main> main > 6.0 > > The same with some versions of ghc. Ah .. it seems to be a bug in gcc 4.0 The Glorious Glasgow Haskell Compilation System, version 6.4 [running on Ubuntu which uses Debian package] gcc (GCC) 4.0.2 20050808 (prerelease) (Ubuntu 4.0.1-4ubuntu9) x86_64 Summary: gcc does not work with GHC 6.4 on x86_64 with optimisation higher than -O1 Perhaps the Evil Mangler is not working, or, possibly the version isn't even registered. ---------------------------------------------- skaller@rosella:/work/felix/flx$ ghc -O3 -fvia-C -optc -O3 -o speed/exes/ghc/takfp speed/src/haskell/takfp.hs skaller@rosella:/work/felix/flx/speed/exes/ghc$ ./takfp 8 8.0 skaller@rosella:/work/felix/flx$ ghc -fvia-C -o speed/exes/ghc/takfp speed/src/haskell/takfp.hs skaller@rosella:/work/felix/flx$ speed/exes/ghc/takfp 8 9.0 skaller@rosella:/work/felix/flx$ speed/exes/ghc/takfp 3 6.0 skaller@rosella:/work/felix/flx$ ghc -O3 -fvia-C -optc -O2 -o speed/exes/ghc/takfp speed/src/haskell/takfp.hs skaller@rosella:/work/felix/flx$ speed/exes/ghc/takfp 3 3.0 skaller@rosella:/work/felix/flx$ ghc -O3 -fvia-C -optc -O1 -o speed/exes/ghc/takfp speed/src/haskell/takfp.hs skaller@rosella:/work/felix/flx$ speed/exes/ghc/takfp 3 6.0 -- John Skaller <skaller at users dot sf dot net> Felix, successor to C++: http://felix.sf.net

1 0

RE: jhc vs ghc and the surprising result involving ghc generatedassembly.
by Simon Peyton-Jones 03 Nov '05

03 Nov '05

All this interesting talk of back ends is somewhat topical at GHC HQ....It may well be that we can share some technology here. GHC uses C-- as its intermediate representation; Core ==> C-- either==> assembly code or==> C Currently the Core->C-- step is done in one big go, by the code generator. The C-- code uses NO native C-- calls; instead, it uses tail calls only, and builds a totally explicit stack as a data structure in the heap. But Chris Thompson-Walsh, as his undergrad project, is writing a CPS converter for C--, so we can do this Core ==> C-- ==> C-- with tail calls only The first step is code generation, converting lambda code to C--, but now this is C-- with native C-- calls. The stack is implicit, entirely under C--'s control. The second step is the CPS conversion. After the CPS conversion the stack is totally explicit. One reason for doing all this is that it factors the rather-complicated code generator into two pieces. The second reason is that if we had a full-blown C-- compiler available, supporting the full runtime interface, we could print out the C-- before the CPS conversion, and use the C-- compiler for the rest. Anyway, apart from the first code-gen step, not much of this process is particularly GHC specific. At the moment it's entirely implemented in Haskell. One of the things Chris is going to look at (if he has time) is packaging the Cmm data type, its various useful functions (e.g. data flow analysis), and his CPS converter, as a Cabal package that GHC just uses. Then other people could use it too. Furthermore the Cmm data type would be the perfect place to spot self-tail calls, eliminate memory operations etc -- all the things that John has been mentioning in his message. That is, a C-- ==> C-- optimiser would be a nice independent project. My reason for mentioning all this now, when it's still partly vapourware, is to see if anyone else is interested. In particular, perhaps a C-- optimiser for jhc would be usable in GHC and vice versa. Simon | -----Original Message----- | From: glasgow-haskell-users-bounces(a)haskell.org [mailto:glasgow-haskell-users- | bounces(a)haskell.org] On Behalf Of John Meacham | Sent: 27 October 2005 00:37 | To: glasgow-haskell-users(a)haskell.org | Subject: Re: jhc vs ghc and the surprising result involving ghc generatedassembly. | | On Wed, Oct 26, 2005 at 12:24:14PM -0400, Jan-Willem Maessen wrote: | > Nice analysis. I indeed found with phc that shadow stack references | > absolutely killed performance, and I aggressively cached stack | > locations in locals, spilling to stack only when GC information | > needed to be accurate. [There was a giant infrastructure to save | > only live data to stack, but we won't go into that now as it was the | > source of almost all the codegen bugs...] | | phc? there is another haskell compiler out there? | | > This makes a big difference. The phc compiler even put comments in | > the code so that I could figure out what came from where. | | yeah, that is something I would like to add. Unfortunatly I wasn't | forward thinking enough to put annotation points everywhere I might | eventually need them so I will have to go through and do that at some | point :) | | > I'm impressed that gcc found this. It's definitely living a bit | > dangerously, and your suggestions below for self tail call handling | > are the ones I found most effective. (They also allowed me to bypass | > some prologue garbage, since phc used a one-C-function-per-Haskell- | > function model with internal resumption points.) Non-self tail calls | > I was careful to compile to: | > return f(...); | > I expect from the above that gcc does better at spotting tail calls now. | [..rest cut..]

4 3

Silly IO problem
by skaller 03 Nov '05

03 Nov '05

This code doesn't work: ------------------------ import System(getArgs) main = do n <- getArgs >>= readIO.head putStrLn (show (tak (3*n) (2*n) n)) tak :: Float -> Float -> Float -> Float tak x y z | y>=x = z | otherwise = tak (tak (x-1) y z) (tak (y-1) z x) (tak (z-1) x y) -------------- It prints n, rather than tak(3n,2n,n). Can someone give me the right encoding please? -- John Skaller <skaller at users dot sf dot net> Felix, successor to C++: http://felix.sf.net

2 1

jhc vs ghc and the surprising result involving ghc generated assembly.
by John Meacham 03 Nov '05

03 Nov '05

(I apologize in advance if this message seems self congradulatory, but after a long time of being disheartened by jhc only having marginal gains over ghc, I am finally seeing some substantial benefits, many of which are the result of optimizations that can actually be ported back to ghc) So, what started out as a simple pat on my own back for getting strictness and CPR finally working turned into an adventure in assembly language. An incidental comparason between jhc and ghc's output completly surprised me, for tight mathematical inner loops jhc is producing code that runs 3-7x faster than ghc. This surprised me because I expected them to be identical, they determine the same strictness and produce the same worker/wrapper split, but due to a quirk of how ghc generates C code, it ends up producing an ultimatly slower result. fortunatly, I think the problem is easy to mitigate. (and jhc loses its lead once again :) ) (all the following snippets are unedited/unreformatted output of the programs specified except for the removal of some profiling tags) the motivating example is the plain old basic accumulating factorial function: fac :: Int -> Int -> Int fac 1 r = r fac n r = fac (n - 1) (n*r) the strictness info is correctly infered that fac is strict in all its arguments and returns a CAF so it is translated into a nice and speedy version that takes unboxed arguments and returns unboxed results. here is the C code that jhc generates. (As an aside, I am very proud of how readable and how much structure the jhc generated C code preserves of the original haskell. it's a small thing, perhaps only implementors appreciate it, but I am glad I spent the time needed to do so.) /* fW@.fMain.fac */ static int fWXAXDfMainXDfac(int v94, int v95) { int v96; int v97; int v98; int v99; switch (v94) { case 1: return v95; break; default: v96 = v94; v97 = (int)(v94 - 1); v98 = (int)(v94 * v95); v99 = fWXAXDfMainXDfac(v97, v98); return v99; break; } } notice that besides being a bit verbose and using a tailcall, this is exactly what A C programmer would write. In fact, the generated assembly is quite nice and I think perhaps provably optimal even compared to hand-coded assembly :) jhc assembly output: fWXAXDfMainXDfac: .L108: cmpl $1, %edi je .L110 imull %edi, %esi decl %edi jmp .L108 .L110: movl %esi, %eax ret notice, a tiny inner loop, 4 instructions and one conditional jump. no memory acceses whatsoever. now, lets look at what ghc does with the same function: the generated C code: EI_(Main_zdwfac_info); FN_(Main_zdwfac_entry) { W_ _s27g; W_ _s27i; W_ _s27l; FB_ _s27g = *Sp; if (_s27g != 0x1UL) goto _c282; R1.p = (P_)(Sp[1]); Sp=Sp+2; JMP_(*Sp); _c282: _s27l = _s27g * (Sp[1]); _s27i = _s27g - 0x1UL; Sp[1] = _s27l; *Sp = _s27i; JMP_((W_)&Main_zdwfac_info); FE_ } it looks complicated, but what it effectivly does is pop the arguments off the stack, run the code but with an explicit jump rather than recursion and push the result back onto the stack. other than the stack stuff at the beginnig and end, we would expect this to get compiled to roughly the same assembly as the jhc version with a nice tight inner loop and just some stack futzing boilerplate. and now the generated assembly. Main_zdwfac_info: .text .align 8 .text movq (%rbp), %rdx cmpq $1, %rdx jne .L2 movq 8(%rbp), %r13 leaq 16(%rbp), %rbp movq (%rbp), %rax .L4: jmp *%rax .L2: movq %rdx, %rax imulq 8(%rbp), %rax movq %rax, 8(%rbp) leaq -1(%rdx), %rax movq %rax, (%rbp) movl $Main_zdwfac_info, %eax jmp .L4 ack! lets count what happens on each iteration of the loop: we have 5 (!) memory acceses and two jumps, one of them being indirect! in fact, each time through the loop, it loads the same values into the same registers. this is really bad compared to the jhc assembly. the basic issue is an interaction between ghc's use of global registers, a stack, and indirect calls. an indirect jump is very expensive. modern processors are pipelined, having many instructions in the queue at once, looking ahead and beginning to evaluate what is coming up. when an indirect jump is encountered, the CPU has no choice but to flush the whole pipeline because it has no idea where it goes, even with conditional branches cpus can predict with good accuracy which branch will be taken and at worst, it discards the pipeline, but with an indirect jump, there is no chance of it having a full pipeline. However the major issue is the following. %rbp is the global stack pointer pointing to the STG stack, since it is global it can be modified from anywhere, since gcc can't know if the function it jumps to modified said register it has no choice but to load all values relative to it every time through the loop because it may have changed. furthermore gotos and labels are very problematic for gcc to optimize around. for various tiresome reasons gcc cannot perform (most) code motion optimizations across explicit labels and gotos, especially when they deal with the global register variables and memory stores and loads. since these are arguably some of the most important optimizations, this is quite bad for the generated assembly. loop: if () goto loop; is not equivalent to a do-while loop, loop invarients cannot be hoisted out of the above for instance (except in some cases... it is all quite tricky and we want gcc to have as much freedom as possible). all in all, this makes the code generated by gcc compiling something generated by ghc not very good at all. there are a couple of things we can do to mitigate these problems: get rid of indirect jumps whenever possible. use C control constructs rather than gotos. A couple simple rules seem to help greatly. * turn anything of the form JMP_((W_)&self) where self is oneself into a goto that gotos a label at the beginning of the function. * do simple pattern matthing on the basic blocks to recognize where C control constructs can be placed. for instance turn if (x) { goto y; } blah.. baz.. JMP_(foo) into if (x) { goto y; } else { blah.. baz.. JMP_(foo) } extending the else to after the next jump or goto. * getting stack dereferences out of your loops. manually performing the first two optimizations mentioned above we get: EI_(Main_zdwfac_info); FN_(Main_zdwfac_entry) { W_ _s27g; W_ _s27i; W_ _s27l; FB_ fac_entry: _s27g = *Sp; if (_s27g != 0x1UL) { goto _c282; } else { R1.p = (P_)(Sp[1]); Sp=Sp+2; JMP_(*Sp); } _c282: _s27l = _s27g * (Sp[1]); _s27i = _s27g - 0x1UL; Sp[1] = _s27l; *Sp = _s27i; goto fac_entry; FE_ } and this produces the assembly: Main_zdwfac_info: .text .align 8 .text movq %rbp, %rdx movq (%rbp), %rcx cmpq $1, %rcx je .L3 .L6: movq %rcx, %rax imulq 8(%rdx), %rax movq %rax, 8(%rdx) leaq -1(%rcx), %rax movq %rax, (%rbp) .L4: movq %rbp, %rdx movq %rax, %rcx cmpq $1, %rax jne .L6 .L3: movq 8(%rdx), %r13 leaq 16(%rdx), %rbp jmp *(%rbp) we still have some unnecesarry memory accesses, but the indirect jump and the spurious jump are gone and we have less instructions in the main loop making this code noticably faster. in order to get rid of the unessesary memory accesess, we need to either 1. fully convert it to use C control constructs, so gcc will do it for us. (code motion and loop invarient being inhibited again by gotos) or 2. do it ourselves by analyzing when the consumer of what we are putting on the stack is ourselves. the first is greatly preferable but not always possible. These should be straightforward to implement in the C code generator. it also suggests we might want to try to use the native C calling convention on leaf nodes that deal with unboxed values (so we get register passing and return values for free) or taking groups of mutually recursive functions and turning them all into one function with explicit jumps between them. to show they are actually optimizing the same function, here are both of their core representaions, other than syntax, they are identical: jhc core: (uses unicode, utf8 formatted) W@.fMain.fac∷int → int → int = λx9216∷int.λx9222∷int.(case x9216 of 1 → x9222 ; x9238∷case <(int)-(int,int) x9216 1∷int> of x9252∷int → case <(int)*(int,int) x9216 x9222∷int> of x34∷int → case W@.fMain.fac x9252 x34 of x9208∷int → x9208;;;;;) ghc core: %rec {zdwfac :: GHCziPrim.Intzh -> GHCziPrim.Intzh -> GHCziPrim.Intzh = \ (ww::GHCziPrim.Intzh) (ww1::GHCziPrim.Intzh) -> %case (GHCziPrim.Intzh) ww %of (ds::GHCziPrim.Intzh) {%_ -> Main.zdwfac (GHCziPrim.zmzh ds (1::GHCziPrim.Intzh)) (GHCziPrim.ztzh ds ww1); (1::GHCziPrim.Intzh) -> ww1}}; some random notes: the 3x-7x factor was tested on an i386, on x86_64 the margin is much much greater for reasons that are still unclear. while testing this I noticed that jhc and ghc have dramatically different results on x86_64 pretty much across the board, if their programs take comparable amounts of time on i386, the jhc one will run twice as fast as the ghc one on x86_64. I think ghc must be tickling the x86_64 in a particularly bad way for this dramatic of an effect to be observed. I will poke around the generated assembly of ghc some more and see if I can find the culprit. (this is also online at: http://repetae.net/jhc-vs-ghc-assembly.txt ) John -- John Meacham - ⑆repetae.net⑆john⑈

5 10

RE: jhc vs ghc and the surprising result involving ghc generatedassembly.
by Simon Marlow 03 Nov '05

03 Nov '05

On 01 November 2005 16:32, Florian Weimer wrote: > * Simon Marlow: > >> gcc started generating this rubbish around version 3.4, if I recall >> correctly. I've tried disabling various optimisations, but can't >> seem to convince gcc not to generate the extra jump. You don't get >> this from the native code generator, BTW. > > But the comparison is present in the C code. What do you want GCC to > do? I didn't mean to sound overly critical of gcc. But here's what I was complaining about - the code generated by gcc (3.4.x) is as follows: Main_zdwfac_info: .text .align 8 .text movq (%rbp), %rdx cmpq $1, %rdx jne .L2 movq 8(%rbp), %r13 leaq 16(%rbp), %rbp movq (%rbp), %rax .L4: jmp *%rax .L2: movq %rdx, %rax imulq 8(%rbp), %rax movq %rax, 8(%rbp) leaq -1(%rdx), %rax movq %rax, (%rbp) movl $Main_zdwfac_info, %eax jmp .L4 there's an obvious simplification - the last two instructions should be replaced by just jmp Main_zdwfac_info eliminating one branch and a mov. This occurs all over the place in our code. Whenever a function has more than one computed goto, gcc insists on commoning up the jmp instructions even when it's a really bad idea, like above. Actually if I add -O2, then I get better code, so perhaps this isn't a real problem. Although gcc still generates this: Fac_zdwfac_info: .text .align 8 movq (%rbp), %rdx testq %rdx, %rdx jne .L3 movq 8(%rbp), %r13 addq $16, %rbp movq (%rbp), %rax jmp *%rax .p2align 4,,7 .L3: movq 8(%rbp), %rax imulq %rdx, %rax decq %rdx movq %rdx, (%rbp) movq %rax, 8(%rbp) movl $Fac_zdwfac_info, %eax jmp *%rax and fails to combine the movs with the jmp instruction (we do this simplification ourselves when post-processing the assembly code). I'll compile up gcc 4 and see what happens with that. Cheers, Simon

3 4