- Glasgow-haskell-users

Re: More on FreeBSD/amd64
by Gregory Wright 03 Apr '07

03 Apr '07

Hi Ian, On Mar 29, 2007, at 8:36 PM, Ian Lynagh wrote: > > Hmm, oh well. > > OK, so we know that the wrong value is being passed to > newPinnedByteArray#, right? There aren't many calls to that: > > libraries/base/Foreign/Marshal/Alloc.hs > libraries/base/GHC/ForeignPtr.hs (4 calls) > libraries/base/GHC/Handle.hs > > so the easiest way forward is probably to print something unique, and > the size passed, in each one and try to work backwards towards the > source. e.g. > > mallocPlainForeignPtrBytes (I# size) = IO $ \s -> > case newPinnedByteArray# size s of { (# s, mbarr# #) -> > > => > > mallocPlainForeignPtrBytes i@(I# size) = do > print ('A', i) > IO $ \s -> > case newPinnedByteArray# size s of { (# s, mbarr# #) -> > > (it might be worth printing something after the existing IO action as > well, just to avoid confusion as to what is happening. > Is there a version of "print" I can use for debugging these libraries? Just adding "print" causes a cycle in module dependencies. I can now start gdb instruction stepping close to the crash and it looks as if the trouble comes from mallocForeignPtr. I'm trying to find out what calls that now. Best Wishes, Greg

3 6

ghc development version (6.7) from MacPorts
by Gregory Wright 02 Apr '07

02 Apr '07

Hi, For those who live on the bleeding edge, I have made a port of the latest development version of ghc for OS X through MacPorts. (If you are not familiar with MacPorts, see http://macports.org.) The build is from the darcs repository, and installs its binaries as <bin_name>-6.7.<date_code> . This naming allows the development version to be installed alongside the released ghc-6.6 with no conflicts. The port should work with OS X 10.3 and later on Intel or PPC. The name of the port is "ghc-devel". Note that because this is really a _development_ version, it may be buggy or not build at all. For day to day use you want the released ghc-6.6. Have fun, Greg

1 0

Re: [Haskell-cafe] ghc-6.7.20070330 on Mac OS X
by Pepe Iborra 02 Apr '07

02 Apr '07

(redirecting to glasgow-haskell-users) It is well known that the readline lib that comes with OS X is no good, and you need to use a replacement. A nice post from the blogosphere explaining all this: http://mult.ifario.us/articles/2006/10/17/ghc-6-6-and-mac-os-x- readline-quick-fix If that doesn't help, please post the errors that you get when building the readline package. Cheers pepe On 01/04/2007, at 16:33, Ruben Zilibowitz wrote: > I am trying to build this version of GHC on Mac OS X. I'm currently > using GHC 6.6. The build is failing with the following error: > > ghci/InteractiveUI.hs:69:7: > Could not find module `System.Console.Readline': > Use -v to see a list of the files searched for. > <<ghc: 310591992 bytes, 51 GCs, 4238510/9457140 avg/max bytes > residency (4 samples), 26M in use, 0.02 INIT (0.00 elapsed), 2.11 > MUT (15.73 elapsed), 0.31 GC (0.36 elapsed) :ghc>> > make[2]: *** [depend] Error 1 > make[1]: *** [stage2] Error 2 > make: *** [bootstrap2] Error 2 > > So it seems I need to install the Readline library. But building > the Readline library is also giving me errors on my system. > > Just wondering if someone else has managed to build ghc-6.7.x on > Mac OS X? If so, what did you need to do? > > Cheers, > > Ruben > > _______________________________________________ > Haskell-Cafe mailing list > Haskell-Cafe(a)haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe

3 2

Arrows and GHC rewrite rules
by Eric Cheng 31 Mar '07

31 Mar '07

In GHC's Control.Arrow implementation, there are several rewrite rules which exploit some properties of arrows. For example, > "compose/arr" forall f g . arr f >>> arr g = arr (f >>> g) > "first/arr" forall f . first (arr f) = arr (first f) > "second/arr" forall f . second (arr f) = arr (second f) ... and so forth. Now say I have a simple arrow instance: > newtype SF a b = SF ([a] -> [b]) > > instance Arrow SF where > arr f = SF (map f) > > first (SF f) = SF g > where g l = let (x, y) = unzip l > in zip (f x) y > > (SF f) >>> (SF g) = SF (g . f) and some arrow composition: > foo :: SF (Int,Int) (Int,Int) > foo = first (arr (+1)) >>> first (arr (+2) >>> arr (+3)) I was expecting that ghc would at least make use of compose/arr or even first/arr, but it didn't appear to be the case. I was using ghc-6.6, and -ddump-simpl-stats showed a bunch of list-specific rules were fired (map, mapFB, fold/build, etc), but I did not see any arrow-related rules being applied. Were the rewrite rules subsumed by inlining or beta-reduction? Am I missing something here? Thanks, Eric

2 2

RULES and type classes
by Mike Hamburg 31 Mar '07

31 Mar '07

Is there any way to use RULES substitutions with type classes? I'm writing a reactive programming arrow (same idea as Yampa, different design goals), and it would help performance (and not just in the speed sense) to be able to tell when a value derived with arr hasn't changed. So I'd like to be able to specialize arr to functions whose result is an instance of Eq. I tried {-# RULES "reactiveArr/Eq" reactiveArr = reactiveArrEq #-} but got the message Control/Arrow/Reactive/Reactive.hs:89:41: No instance for (Eq b) arising from instantiating a type signature at Control/Arrow/Reactive/Reactive.hs:89:41-89 Possible fix: add (Eq b) to the tcRule When checking the transformation rule "reactiveArr/Eq" I tried adding various sorts of type signatures, but I couldn't find any way around this... is it a restriction in the RULES rewrite engine? Is there a workaround, or some mechanism other than RULES that I should be using? I could write a special "arrEq" function, but I'd like to minimize the number of extraneous operations outside the Arrow class. Thanks, Mike Hamburg

7 7

rules
by skaller 31 Mar '07

31 Mar '07

I'm curious when and how GHC applies rewrite rules, and how expensive it is. Felix also has rewrite rules. It uses a woefully expensive algorithm to apply them: 1) elide rules that refer to functions that have themselves been elided since they can't be applied. 2) For each rule in turn For each subexpression going down and up the subexpression tree, try to match the rule If there is a match perform a rewrite and set a 'modified' flag If the expression was modified, clear the flag and goto step 2. This process is 'more or less' applied before and after every other term rewriting (eg inlining) because reductions can create inlining opportunities, reduce the amount of inlining, but inlining also exposes new instances of rule patterns. AFAICS my algorithm is exponential or worse, however I assume it is still efficient simply because not many rule patterns turn up, and the ones that do get reduced away quickly. Felix allows these reductions in typeclasses, but it doesn't selectively apply them to functions using those classes -- the reductions are all performed globally (all rules on all code). -- John Skaller <skaller at users dot sf dot net> Felix, successor to C++: http://felix.sf.net

2 2

Re: More on FreeBSD/amd64
by Gregory Wright 30 Mar '07

30 Mar '07

Hi Ian, On Mar 30, 2007, at 8:22 AM, Ian Lynagh wrote: > On Fri, Mar 30, 2007 at 09:31:07AM +0100, Simon Marlow wrote: >> Ian Lynagh wrote: >>> >>> OK, so we know that the wrong value is being passed to >>> newPinnedByteArray#, right? There aren't many calls to that: >>> >>> libraries/base/Foreign/Marshal/Alloc.hs >>> libraries/base/GHC/ForeignPtr.hs (4 calls) >>> libraries/base/GHC/Handle.hs > > Oh, I've just realised, it's the 6.4.2 libraries you need to look at, > not the 6.6 ones. Let me make sure I understand this: the problem show up when running the 6.6 compiler in compiler/stage1/ghc-6.6-20070314. This compiler is linked with the ghc-6.4.2 RTS _and_ the ghc-6.4.2 libraries, correct? So when I look for symbols mentioned in the RTS stack, (e.g., "s311_info") I have to refer to the -ddump-stg output from the ghc-6.4.2 build. Is this true? > >>> so the easiest way forward is probably to print something unique, >>> and >>> the size passed, in each one and try to work backwards towards the >>> source. e.g. >> >> Or just single-step (by *instruction*, not line) from an earlier >> point >> before the crash. I think Greg was stepping by line before, which >> is why >> he didn't see anything happen before the erroneous call. > > If you're going to try this then I find it easier with > > GhcLibHcOpts += -O0 -g > > when building the (6.4.2) libraries. > Yes, I did this. Thank you for your help! Best Wishes, Greg

2 1

More on FreeBSD/amd64
by Gregory Wright 30 Mar '07

30 Mar '07

Hi Ian, I have made some more progress on understanding the build failure on FreeBSD/amd64. I could use a check on my understanding of the problem, though. The setup: I have an unregisterized ghc-6.4.2 successfully built on FreeBSD/amd64. It was bootstrapped from .hc files compiled on FreeBSD/i386. I am attempting to use this compiler (the ghc-inplace from the unregisterized build, not a fully installed compiler) to build a recent ghc-6.6 branch from darcs (20070314). The build of ghc-6.6-20070314 fails when compiling rts/Linker.c. The failure is mostly reproducible (more about that below). It's also worth remembering that when I tried to build an unregisterized ghc-6.6 on FreeBSD/amd64 using .hc files from ghc-6.6 built on FreeBSD/i386, I had a crash at the same place, while trying to build rts/Linker.c The failure comes from trying to allocate a huge amount of memory. newPinnedByteArrayzh_fast is called with a giant argument, 0x4000000010. So it looks like we're after 16 bytes, but the upper 32 bits has some junk in it. The above was the state of things just over a week ago. Since then, I've worked to track down whether the bug is in the ghc-6.4.2 runtime or in the ghc-6.6 code. The compiler that fails is the 6.6 stage1 compiler, which is a 6.6 compiler linked with the runtime from the unregisterized 6.4.2 (let me know if I'm wrong about that). I have rebuilt the unregisterized 6.4.2 with optimization turned off. I haven't got any more information this way; it seems the problem is really on the 6.6 side. Here is my reasoning: I run the 6.6 compiler under the debugger: greenhouse-george> gdb /tmp/ghc/compiler/stage1/ghc-6.6.20070314 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... (gdb) dir /tmp/ghc-6.4.2/ghc/rts Source directories searched: /tmp/ghc-6.4.2/ghc/rts:$cdir:$cwd (gdb) b newPinnedByteArrayzh_fast Breakpoint 1 at 0x163cbb4 (gdb) run -B/tmp/ghc -v -optc-O -optc-Wall -optc-W -optc-Wstrict- prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations - optc-Winline -optc-Waggregate-return -optc-Wbad-function-cast -optc- I../includes -optc-I. -optc-Iparallel -optc-DCOMPILING_RTS -optc- fomit-frame-pointer -optc-I/usr/local/include -optc-fno-strict- aliasing -H16m -O -optc-O2 -static -I/usr/local/include -I. -#include HCIncludes.h -fvia-C -dcmm-lint -c Linker.c -o Linker.o Starting program: /tmp/ghc/compiler/stage1/ghc-6.6.20070314 -B/tmp/ ghc -v -optc-O -optc-Wall -optc-W -optc-Wstrict-prototypes -optc- Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc- Waggregate-return -optc-Wbad-function-cast -optc-I../includes -optc- I. -optc-Iparallel -optc-DCOMPILING_RTS -optc-fomit-frame-pointer - optc-I/usr/local/include -optc-fno-strict-aliasing -H16m -O -optc-O2 - static -I/usr/local/include -I. -#include HCIncludes.h -fvia-C -dcmm- lint -c Linker.c -o Linker.o Breakpoint 1, 0x000000000163cbb4 in newPinnedByteArrayzh_fast () By playing around with it, I have isolated --- more or less --- when the failure occurs. In this run, it was after 973 calls to newPinnedByteArrayzh_fast. If I quit gdb and re-run it, the number of calls is consistently the same. However, from one system boot to the next, it varies a bit. Yesterday morning, I had to skip 976 calls. (gdb) c 973 Will ignore next 972 crossings of breakpoint 1. Continuing. Glasgow Haskell Compiler, Version 6.6.20070314, for Haskell 98, compiled by GHC version 6.4.2 Using package config file: /tmp/ghc/driver/package.conf.inplace wired-in package base not found. wired-in package rts mapped to rts-1.0 wired-in package haskell98 not found. wired-in package template-haskell not found. Hsc static flags: -static -static Created temporary directory: /tmp/ghc1073_0 *** C Compiler: gcc -x c Linker.c -o /tmp/ghc1073_0/ghc1073_0.s -v -S -Wimplicit -O - D__GLASGOW_HASKELL__=606 -O -Wall -W -Wstrict-prototypes -Wmissing- prototypes -Wmissing-declarations -Winline -Waggregate-return -Wbad- function-cast -I../includes -I. -Iparallel -DCOMPILING_RTS -fomit- frame-pointer -I/usr/local/include -fno-strict-aliasing -O2 -I /usr/ local/include -I . -I /tmp/ghc/includes -fwrapv Breakpoint 1, 0x000000000163cbb4 in newPinnedByteArrayzh_fast () OK, I should be at the call which is going to blow up: (gdb) bt #0 0x000000000163cbb4 in newPinnedByteArrayzh_fast () #1 0x00000000016377ea in StgRun (f=0x163cbb0 <newPinnedByteArrayzh_fast>, basereg=0x260fcd0) at StgCRun.c:93 #2 0x0000000001631f48 in schedule (mainThread=0x2611080, initialCapability=0x0) at Schedule.c:932 #3 0x0000000001633190 in waitThread_ (m=0x2611080, initialCapability=0x0) at Schedule.c:2156 #4 0x0000000001633085 in scheduleWaitThread (tso=0x8021c0000, ret=0x0, initialCapability=0x0) at Schedule.c:2050 #5 0x000000000162d60f in rts_evalLazyIO (p=0x1d409d0, ret=0x0) at RtsAPI.c:459 #6 0x000000000162ca24 in main (argc=33, argv=0x7fffffffe960) at Main.c:104 (gdb) f 1 #1 0x00000000016377ea in StgRun (f=0x163cbb0 <newPinnedByteArrayzh_fast>, basereg=0x260fcd0) at StgCRun.c:93 93 f = (StgFunPtr) (f)(); Since the 6.4.2 runtime is unregisterized, the parameter are passed through the virtual, not actual registers (is this correct?). The code for newPinnedByteArrayzh_fast says that the requested number of bytes is passed in through R1, so take a look: (gdb) p basereg->rR1 $1 = {w = 274877906960, a = 0x4000000010, c = 16, i8 = 16 '\020', f = 2.24207754e-44, i = 274877906960, p = 0x4000000010, cl = 0x4000000010, offset = 274877906960, b = 0x4000000010 <Error reading address 0x4000000010: Bad address>, t = 0x4000000010} This request should certainly wedge the system by gobbling up all of the memory. The question now is what put the bad value into R1? It seems as if newPinnedByteArrayzh_fast is just getting too big a request. I looked at the assembly code for newPinnedByteArrayzh_fast, and the arguments passed down to the functions it calls are calculated correctly, but are just too big. I also tried setting the breakpoint one call earlier (972 instead of 973), and then stepping through until I reached the next breakpoint. newPinnedByteArrayzh_fast calls allocatePinned successfully and control come back to StgCRun.c at line 87. This is the miniinterpreter loop. If I keep stepping control just stays in the miniinterpreter loop until newPinnedByteArrayzh_fast is called again. (I used the gdb "until" command to run the loop until it either finished or hit a breakpoint. It hit the breakpoint.) This leads me to believe that the problem is in the haskell code, not the C code of the runtime system. Looking at the RTS stack, (gdb) pmem basereg->rSp 32 0x802dfe808: 0x0 0x802dfe800: 0x0 0x802dfe7f8: 0x2216f80 <stg_stop_thread_info> 0x802dfe7f0: 0x2217000 <stg_noforceIO_info> 0x802dfe7e8: 0x2149b90 <GHCziConc_childHandler_closure> 0x802dfe7e0: 0x0 0x802dfe7d8: 0x2214b60 <stg_catch_frame_info> 0x802dfe7d0: 0x2214b20 <stg_unblockAsyncExceptionszh_ret_info> 0x802dfe7c8: 0x802a121b0 0x802dfe7c0: 0x1d61f60 <s99O_info> 0x802dfe7b8: 0x802a15848 0x802dfe7b0: 0x1 0x802dfe7a8: 0x2214b60 <stg_catch_frame_info> 0x802dfe7a0: 0x2214b40 <stg_blockAsyncExceptionszh_ret_info> 0x802dfe798: 0x22179e0 <stg_ap_v_info> 0x802dfe790: 0x802a17af0 0x802dfe788: 0x802a17b08 0x802dfe780: 0x802a121b0 0x802dfe778: 0x802a15870 0x802dfe770: 0x1d61a40 <s99v_info> 0x802dfe768: 0x217aef0 <s6Z8_info> 0x802dfe760: 0x802a17b88 0x802dfe758: 0x802d04e98 0x802dfe750: 0x2125f40 <s3eo_info> 0x802dfe748: 0x802a18970 0x802dfe740: 0x802a17b08 0x802dfe738: 0x802a17b78 0x802dfe730: 0x1d61620 <r37t_closure> 0x802dfe728: 0x400000000 0x802dfe720: 0x802d04ed0 0x802dfe718: 0x400000000 0x802dfe710: 0x220db80 <s31b_info> (gdb) I'm guessing that the culpable module defines the symbol "s31b". I added -ddump-stg to the options for the 6.6 build. (In fact, deleted everything, added the -ddump-stg option, rebuilt and did the run under the debugger shown above, to make certain that everything was consistent.) The symbol "s31b" is from the module GHC.Compat.Unicode. This may be getting somewhere. Compat.Unicode does not exist in 6.4.2, which may explain why the unregisterized 6.4.2 is quite reliable --- I used it to build alex and happy with no trouble. Also, the Unicode module makes foreign calls to C library functions. FreeBSD has its own implementation of libc, not the same as linux, which may explain why FreeBSD/amd64 chokes and Linux/amd64 works. So. I am asking if this argument makes sense, especially my interpretation of the RTS stack dump. Any suggestions are appreciated. Best Wishes, Greg

3 3

GADT + Newtype deriving = Erroneous errors
by Stefan O'Rear 29 Mar '07

29 Mar '07

This code causes GHC to incorrectly fail - the case *is* reachable. (I invented this technique in an attempt to directly access the internal System FC newtype coercion; it promised until a few minutes ago to solve all the EnumMap performance concerns.) stefan@stefans:/tmp$ cat A.lhs > {-# OPTIONS_GHC -fglasgow-exts #-} > > data IsIntT x where IsIntT :: IsIntT Int > > class IsIntC a where isInt :: IsIntT a > instance IsIntC Int where isInt = IsIntT > > newtype Foo = Foo Int deriving(IsIntC) > > x :: IsIntT Foo -> Int > x IsIntT = (Foo 2) + 2 > > y = x (isInt :: IsIntT Foo) stefan@stefans:/tmp$ ghci -v0 A.lhs A.lhs:11:4: Inaccessible case alternative: Can't match types `Foo' and `Int' In the pattern: IsIntT In the definition of `x': x IsIntT = (Foo 2) + 2 Stefan

7 11

inter module optimizations
by Fawzi Mohamed 28 Mar '07

28 Mar '07

I had posted some data on inter-module optimizations that I had calculated when splitting my program from one computational module to many different ones. Tim Chevalier suggested that my calculation could be interesting to the people here. So I made the effort of preparing the various versions of my code and re doing the analysis better. Unfortunately I had already began renaming things without doing a darcs record, so in the split version some function names are different. I have a tar.bz archive of 21KB, but I did not know if it is considered rude to send attachments, but if someone is interested I can send him the file. Basically it mainly boils down to non-inlining of some important functions on a newtype ( type LatLocI = Word32 newtype LatLoc = LatLoc LatLocI deriving (Eq,Ord) ), because specialization should not be an issue as I had already given specific signatures to my functions. Also worth noting is that using the profiling with -O2 compilation makes one thing that inlining (or using a single module) makes the program slower, whereas the opposite is true. I think that the profiling overhead are incorrectly evaluated. I know that with -O2 one cannot expect profiling to be good, but it would be nice if it wouldn't be so misleading Here some data (obtained with a script that is also in the tar.bz archive) ******** allInOne: original program, monolithic main computational module * timings of -O2 executable 7.67user 0.00system 0:07.69elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+894minor)pagefaults 0swaps * timings of the executable with profiling total time = 15.25 secs (305 ticks @ 50 ms) total alloc = 5,888,786,120 bytes (excludes profiling overheads) ******** splitModule NoReexport NoInline directives: split computational module, no export list for split modules * timings of -O2 executable 10.14user 0.01system 0:10.17elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+901minor)pagefaults 0swaps * timings of the executable with profiling total time = 11.85 secs (237 ticks @ 50 ms) total alloc = 5,888,780,912 bytes (excludes profiling overheads) ******** splitModule Reexport NoInline directives: computational module, no export list for split modules, old module reexport using export list * timings of -O2 executable 8.88user 0.00system 0:08.90elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+901minor)pagefaults 0swaps * timings of the executable with profiling total time = 12.20 secs (244 ticks @ 50 ms) total alloc = 5,888,780,912 bytes (excludes profiling overheads) ******** splitModule NoReexport Inline directives: split computational module, no export list for split modules, explicit inline directives * timings of -O2 executable 6.44user 0.01system 0:06.46elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+895minor)pagefaults 0swaps * timings of the executable with profiling total time = 18.80 secs (376 ticks @ 50 ms) total alloc = 5,374,883,312 bytes (excludes profiling overheads) ************* Fawzi

2 2