March 2007 - Glasgow-haskell-users

partially applied type synonyms
by Arie Peterson 05 Mar '12

05 Mar '12

Dear list, Is there a good reason that partially applied type synonyms cannot be made instances of classes? - A simple example would be making the identity type 'Id' ("type Id x = x") instance of 'Monad'. This would eliminate the need for some silly occurrences of 'runIdentity', and perhaps narrow the gap between monads and monad transformers. - The reason I complain about this now is a more involved example, where introducing a newtype to work around the restriction is not an option. Well, not as far as I can see now, anyway. Partially applied type synonyms are, in a sense, the anonymous functions at the type level, and one might argue that they deserve the same 'first-class member status' as the lambda at the value level. Kind regards, Arie Peterson

3 2

ghci and ghc -threaded broken with pipes & forking
by John Goerzen 29 Dec '08

29 Dec '08

Hi, I've been hitting my head against a wall for the past couple of days trying to figure out why my shell-like pipeline code kept hanging. I found fd leakage (file descriptors not being closed), which disrupts EOF detection and can lead to deadlocks. I just couldn't find the problem. I finally tried compiling my test with ghc instead of running it in ghci. And poof, it worked fine the first time. I tried asking on #haskell, and got the suggestion that ghci uses -threaded. I tried compiling my test program with ghc -threaded, and again, same deadlock. My program never calls forkIO or forkOS or any other threading code. You can see my test case with: darcs get '--tag=glasgow ml' http://darcs.complete.org/hsh ghc -fglasgow-exts --make -o test2 test2.hs That'll run fine. If you add -threaded, it will hang. Ideas? Thanks, -- John

13 44

More speed please!
by Duncan Coutts 11 May '07

11 May '07

All, So here's the Put monad for the binary serialisation stuff: newtype Put a = Put { runPut :: (a -> Buffer -> [B.ByteString]) -> Buffer -> [B.ByteString] } data Buffer = Buffer {-# UNPACK #-} !(ForeignPtr Word8) {-# UNPACK #-} !Int -- offset {-# UNPACK #-} !Int -- used bytes {-# UNPACK #-} !Int -- length left This is all good, and pretty quick. Code like the following gets turned into great low level code: foo :: Word8 -> Put () foo !n = do word8 n word8 (n+1) word8 (n+17) It does a straight line sequence of writes into memory (there are rules that combine the bounds checking of adjacent writes). After that it calls the continuation with a new buffer. While almost everything is unboxed, the parameters to the continuation are not unboxed. So we have to allocate a new Buffer structure and pass that to the continuation: let { sat_s1F8 = NO_CCS PutMonad.Buffer! [ww1_s1Es ww2_s1Et ww3_s1Ew sat_s1F4 sat_s1F6]; } in w_s1DY GHC.Base.() sat_s1F8; w_s1DY being the continuation here. However we know that really the parameters to this continuation could really be unboxed since we construct all these continuations explicitly in this module. We could re-write the monad type by manually explicitly unboxing the Buffer argument: newtype Put a = Put { runPut :: (a -> Addr# -> ForeignPtrContents -> Int# -> Int# -> Int# -> [B.ByteString]) -> Addr# -> ForeignPtrContents -> Int# -> Int# -> Int# -> [B.ByteString] } Then we'd get no allocations and no heap checks in the fast path. Note that we could still get a continuation in from the 'outside' with the original type and convert it to one with the above type, though obviously that involves unpacking the arguments. This unpacking is basically what the wrapper of a worker/wrapper pair does of course. Obviously this is ugly. We'd much rather write the original and with just a couple annotations get the above representation. This is what I would like to write: newtype Put a = Put { runPut :: (a -> {-# UNPACK #-} !Buffer -> [B.ByteString]) -> {-# UNPACK #-} !Buffer -> [B.ByteString] } So I'm declaring a type and a data constructor that contains a function that is strict in one of it's arguments. I do not wish to distinguish the strictness in the type however, that is it should be perfectly ok to do this: foo :: Foo -> Buffer -> [B.ByteString] Put foo newBuffer :: Put Foo Suppose that in this case we do not know locally that foo is strict and can take it's args unboxed then upon applying the Put constructor we just have to apply a wrapper function that boxes up the args and supplies them to the wrapped function. Obviously that'd be a pessimisation, having to re-box args, however we can expect programmers to only use that UNPACK pragma when they know that it is going to be a win overall. So the ! on the arg is a semantic change and the pragma is a representation change but not a semantic change. The ! means the obvious thing, that the function is strict in that arg. So either the caller or calle must ensure the arg is evaluated to WHNF. Hmm, is the UNPACK actually needed then? Normally when ghc determines that a func is strict in an arg it make the worker unbox that arg if possible and that always seems to be a win. Mine you, in that case we know what the function is going to do with each arg, where as here we do not so perhaps a separate UNPACK makes sense. This issues crops up quite a bit with Monads I think. For example GHC defines it's IO monad to return an unboxed tuple: newtype IO a = IO (State# RealWorld -> (# State# RealWorld, a #)) This is fine for low level internal code, but for everyday monads it'd be nice to be able to write nice code and still get great performance. This applies to monads particularly since they very often fully encapsulate the function used as the internal representation, that is all the sites where the representation function is constructed and taken apart are in a single module. So we can 'see' that all uses are strict in some arg. Or if they're not we might want them to be. So instead of doing an analysis to figure out if we're always using it strictly (which is something that jhc/grin might be able to do?) we can just declare the data representation to be strict like we do with ordinary algebraic data types, that way it propagates strictness to the usage sites which is much more convenient than going around strictifying all the usage sites in an attempt to make an analysis notice that it's safe to do a representation change. Ok, I'm done. :-) Duncan

5 10

noinline in where clauses again
by Duncan Coutts 26 Apr '07

26 Apr '07

I winged before about NOINLINE pragma on things defined locally in a where clause not doing what I expected: http://www.mail-archive.com/glasgow-haskell-users@haskell.org/msg10338.html Turns out that one was fixed in ghc HEAD. I've got another similar one which is not working as I expect in 6.6 or in head from 1st March. This example is from an experimental re-implementation of the Put monad in Data.Binary: write :: Int -> (Ptr Word8 -> IO ()) -> Put () write !n body = Put $ \c buf@(Buffer fp o u l) -> if n <= l then write' c fp o u l else write' (flushOld c n fp o u) (newBuffer c n) 0 0 0 where {-# NOINLINE write' #-} write' c !fp !o !u !l = -- warning: this is a tad hardcore B.inlinePerformIO (withForeignPtr fp (\p -> body $! (p `plusPtr` (o+u)))) `seq` c () (Buffer fp o (u+n) (l-n)) {-# INLINE [1] write #-} Then we use it with things like word8 :: Word8 -> Put () word8 !w = write 1 (pokeWord8 w) pokeWord8 :: Word8 -> Ptr Word8 -> IO () pokeWord8 w p = poke p w Then there's a rule so that things like: foo :: Word8 -> Put () foo !n = do word8 n word8 (n+1) word8 (n+17) get turned into a single call to write. Anyway, the point is that when we look at the stg/core we see that in write above, the write' has been inlined at the two call sites where as I want both branches of the if test to make calls to write'. The code is here: http://haskell.org/~duncan/binary/PutMonad.hs http://haskell.org/~duncan/binary/PutTest.hs the stg from ghc-6.6 is: http://haskell.org/~duncan/binary/PutTest.stg with ghc head from 1st March the result is different but still inlines write' in both branches. Duncan

2 2

Arrows and GHC rewrite rules
by Eric Cheng 31 Mar '07

31 Mar '07

In GHC's Control.Arrow implementation, there are several rewrite rules which exploit some properties of arrows. For example, > "compose/arr" forall f g . arr f >>> arr g = arr (f >>> g) > "first/arr" forall f . first (arr f) = arr (first f) > "second/arr" forall f . second (arr f) = arr (second f) ... and so forth. Now say I have a simple arrow instance: > newtype SF a b = SF ([a] -> [b]) > > instance Arrow SF where > arr f = SF (map f) > > first (SF f) = SF g > where g l = let (x, y) = unzip l > in zip (f x) y > > (SF f) >>> (SF g) = SF (g . f) and some arrow composition: > foo :: SF (Int,Int) (Int,Int) > foo = first (arr (+1)) >>> first (arr (+2) >>> arr (+3)) I was expecting that ghc would at least make use of compose/arr or even first/arr, but it didn't appear to be the case. I was using ghc-6.6, and -ddump-simpl-stats showed a bunch of list-specific rules were fired (map, mapFB, fold/build, etc), but I did not see any arrow-related rules being applied. Were the rewrite rules subsumed by inlining or beta-reduction? Am I missing something here? Thanks, Eric

2 2

RULES and type classes
by Mike Hamburg 31 Mar '07

31 Mar '07

Is there any way to use RULES substitutions with type classes? I'm writing a reactive programming arrow (same idea as Yampa, different design goals), and it would help performance (and not just in the speed sense) to be able to tell when a value derived with arr hasn't changed. So I'd like to be able to specialize arr to functions whose result is an instance of Eq. I tried {-# RULES "reactiveArr/Eq" reactiveArr = reactiveArrEq #-} but got the message Control/Arrow/Reactive/Reactive.hs:89:41: No instance for (Eq b) arising from instantiating a type signature at Control/Arrow/Reactive/Reactive.hs:89:41-89 Possible fix: add (Eq b) to the tcRule When checking the transformation rule "reactiveArr/Eq" I tried adding various sorts of type signatures, but I couldn't find any way around this... is it a restriction in the RULES rewrite engine? Is there a workaround, or some mechanism other than RULES that I should be using? I could write a special "arrEq" function, but I'd like to minimize the number of extraneous operations outside the Arrow class. Thanks, Mike Hamburg

7 7

rules
by skaller 31 Mar '07

31 Mar '07

I'm curious when and how GHC applies rewrite rules, and how expensive it is. Felix also has rewrite rules. It uses a woefully expensive algorithm to apply them: 1) elide rules that refer to functions that have themselves been elided since they can't be applied. 2) For each rule in turn For each subexpression going down and up the subexpression tree, try to match the rule If there is a match perform a rewrite and set a 'modified' flag If the expression was modified, clear the flag and goto step 2. This process is 'more or less' applied before and after every other term rewriting (eg inlining) because reductions can create inlining opportunities, reduce the amount of inlining, but inlining also exposes new instances of rule patterns. AFAICS my algorithm is exponential or worse, however I assume it is still efficient simply because not many rule patterns turn up, and the ones that do get reduced away quickly. Felix allows these reductions in typeclasses, but it doesn't selectively apply them to functions using those classes -- the reductions are all performed globally (all rules on all code). -- John Skaller <skaller at users dot sf dot net> Felix, successor to C++: http://felix.sf.net

2 2

Re: More on FreeBSD/amd64
by Gregory Wright 30 Mar '07

30 Mar '07

Hi Ian, On Mar 30, 2007, at 8:22 AM, Ian Lynagh wrote: > On Fri, Mar 30, 2007 at 09:31:07AM +0100, Simon Marlow wrote: >> Ian Lynagh wrote: >>> >>> OK, so we know that the wrong value is being passed to >>> newPinnedByteArray#, right? There aren't many calls to that: >>> >>> libraries/base/Foreign/Marshal/Alloc.hs >>> libraries/base/GHC/ForeignPtr.hs (4 calls) >>> libraries/base/GHC/Handle.hs > > Oh, I've just realised, it's the 6.4.2 libraries you need to look at, > not the 6.6 ones. Let me make sure I understand this: the problem show up when running the 6.6 compiler in compiler/stage1/ghc-6.6-20070314. This compiler is linked with the ghc-6.4.2 RTS _and_ the ghc-6.4.2 libraries, correct? So when I look for symbols mentioned in the RTS stack, (e.g., "s311_info") I have to refer to the -ddump-stg output from the ghc-6.4.2 build. Is this true? > >>> so the easiest way forward is probably to print something unique, >>> and >>> the size passed, in each one and try to work backwards towards the >>> source. e.g. >> >> Or just single-step (by *instruction*, not line) from an earlier >> point >> before the crash. I think Greg was stepping by line before, which >> is why >> he didn't see anything happen before the erroneous call. > > If you're going to try this then I find it easier with > > GhcLibHcOpts += -O0 -g > > when building the (6.4.2) libraries. > Yes, I did this. Thank you for your help! Best Wishes, Greg

2 1

More on FreeBSD/amd64
by Gregory Wright 30 Mar '07

30 Mar '07

Hi Ian, I have made some more progress on understanding the build failure on FreeBSD/amd64. I could use a check on my understanding of the problem, though. The setup: I have an unregisterized ghc-6.4.2 successfully built on FreeBSD/amd64. It was bootstrapped from .hc files compiled on FreeBSD/i386. I am attempting to use this compiler (the ghc-inplace from the unregisterized build, not a fully installed compiler) to build a recent ghc-6.6 branch from darcs (20070314). The build of ghc-6.6-20070314 fails when compiling rts/Linker.c. The failure is mostly reproducible (more about that below). It's also worth remembering that when I tried to build an unregisterized ghc-6.6 on FreeBSD/amd64 using .hc files from ghc-6.6 built on FreeBSD/i386, I had a crash at the same place, while trying to build rts/Linker.c The failure comes from trying to allocate a huge amount of memory. newPinnedByteArrayzh_fast is called with a giant argument, 0x4000000010. So it looks like we're after 16 bytes, but the upper 32 bits has some junk in it. The above was the state of things just over a week ago. Since then, I've worked to track down whether the bug is in the ghc-6.4.2 runtime or in the ghc-6.6 code. The compiler that fails is the 6.6 stage1 compiler, which is a 6.6 compiler linked with the runtime from the unregisterized 6.4.2 (let me know if I'm wrong about that). I have rebuilt the unregisterized 6.4.2 with optimization turned off. I haven't got any more information this way; it seems the problem is really on the 6.6 side. Here is my reasoning: I run the 6.6 compiler under the debugger: greenhouse-george> gdb /tmp/ghc/compiler/stage1/ghc-6.6.20070314 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... (gdb) dir /tmp/ghc-6.4.2/ghc/rts Source directories searched: /tmp/ghc-6.4.2/ghc/rts:$cdir:$cwd (gdb) b newPinnedByteArrayzh_fast Breakpoint 1 at 0x163cbb4 (gdb) run -B/tmp/ghc -v -optc-O -optc-Wall -optc-W -optc-Wstrict- prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations - optc-Winline -optc-Waggregate-return -optc-Wbad-function-cast -optc- I../includes -optc-I. -optc-Iparallel -optc-DCOMPILING_RTS -optc- fomit-frame-pointer -optc-I/usr/local/include -optc-fno-strict- aliasing -H16m -O -optc-O2 -static -I/usr/local/include -I. -#include HCIncludes.h -fvia-C -dcmm-lint -c Linker.c -o Linker.o Starting program: /tmp/ghc/compiler/stage1/ghc-6.6.20070314 -B/tmp/ ghc -v -optc-O -optc-Wall -optc-W -optc-Wstrict-prototypes -optc- Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc- Waggregate-return -optc-Wbad-function-cast -optc-I../includes -optc- I. -optc-Iparallel -optc-DCOMPILING_RTS -optc-fomit-frame-pointer - optc-I/usr/local/include -optc-fno-strict-aliasing -H16m -O -optc-O2 - static -I/usr/local/include -I. -#include HCIncludes.h -fvia-C -dcmm- lint -c Linker.c -o Linker.o Breakpoint 1, 0x000000000163cbb4 in newPinnedByteArrayzh_fast () By playing around with it, I have isolated --- more or less --- when the failure occurs. In this run, it was after 973 calls to newPinnedByteArrayzh_fast. If I quit gdb and re-run it, the number of calls is consistently the same. However, from one system boot to the next, it varies a bit. Yesterday morning, I had to skip 976 calls. (gdb) c 973 Will ignore next 972 crossings of breakpoint 1. Continuing. Glasgow Haskell Compiler, Version 6.6.20070314, for Haskell 98, compiled by GHC version 6.4.2 Using package config file: /tmp/ghc/driver/package.conf.inplace wired-in package base not found. wired-in package rts mapped to rts-1.0 wired-in package haskell98 not found. wired-in package template-haskell not found. Hsc static flags: -static -static Created temporary directory: /tmp/ghc1073_0 *** C Compiler: gcc -x c Linker.c -o /tmp/ghc1073_0/ghc1073_0.s -v -S -Wimplicit -O - D__GLASGOW_HASKELL__=606 -O -Wall -W -Wstrict-prototypes -Wmissing- prototypes -Wmissing-declarations -Winline -Waggregate-return -Wbad- function-cast -I../includes -I. -Iparallel -DCOMPILING_RTS -fomit- frame-pointer -I/usr/local/include -fno-strict-aliasing -O2 -I /usr/ local/include -I . -I /tmp/ghc/includes -fwrapv Breakpoint 1, 0x000000000163cbb4 in newPinnedByteArrayzh_fast () OK, I should be at the call which is going to blow up: (gdb) bt #0 0x000000000163cbb4 in newPinnedByteArrayzh_fast () #1 0x00000000016377ea in StgRun (f=0x163cbb0 <newPinnedByteArrayzh_fast>, basereg=0x260fcd0) at StgCRun.c:93 #2 0x0000000001631f48 in schedule (mainThread=0x2611080, initialCapability=0x0) at Schedule.c:932 #3 0x0000000001633190 in waitThread_ (m=0x2611080, initialCapability=0x0) at Schedule.c:2156 #4 0x0000000001633085 in scheduleWaitThread (tso=0x8021c0000, ret=0x0, initialCapability=0x0) at Schedule.c:2050 #5 0x000000000162d60f in rts_evalLazyIO (p=0x1d409d0, ret=0x0) at RtsAPI.c:459 #6 0x000000000162ca24 in main (argc=33, argv=0x7fffffffe960) at Main.c:104 (gdb) f 1 #1 0x00000000016377ea in StgRun (f=0x163cbb0 <newPinnedByteArrayzh_fast>, basereg=0x260fcd0) at StgCRun.c:93 93 f = (StgFunPtr) (f)(); Since the 6.4.2 runtime is unregisterized, the parameter are passed through the virtual, not actual registers (is this correct?). The code for newPinnedByteArrayzh_fast says that the requested number of bytes is passed in through R1, so take a look: (gdb) p basereg->rR1 $1 = {w = 274877906960, a = 0x4000000010, c = 16, i8 = 16 '\020', f = 2.24207754e-44, i = 274877906960, p = 0x4000000010, cl = 0x4000000010, offset = 274877906960, b = 0x4000000010 <Error reading address 0x4000000010: Bad address>, t = 0x4000000010} This request should certainly wedge the system by gobbling up all of the memory. The question now is what put the bad value into R1? It seems as if newPinnedByteArrayzh_fast is just getting too big a request. I looked at the assembly code for newPinnedByteArrayzh_fast, and the arguments passed down to the functions it calls are calculated correctly, but are just too big. I also tried setting the breakpoint one call earlier (972 instead of 973), and then stepping through until I reached the next breakpoint. newPinnedByteArrayzh_fast calls allocatePinned successfully and control come back to StgCRun.c at line 87. This is the miniinterpreter loop. If I keep stepping control just stays in the miniinterpreter loop until newPinnedByteArrayzh_fast is called again. (I used the gdb "until" command to run the loop until it either finished or hit a breakpoint. It hit the breakpoint.) This leads me to believe that the problem is in the haskell code, not the C code of the runtime system. Looking at the RTS stack, (gdb) pmem basereg->rSp 32 0x802dfe808: 0x0 0x802dfe800: 0x0 0x802dfe7f8: 0x2216f80 <stg_stop_thread_info> 0x802dfe7f0: 0x2217000 <stg_noforceIO_info> 0x802dfe7e8: 0x2149b90 <GHCziConc_childHandler_closure> 0x802dfe7e0: 0x0 0x802dfe7d8: 0x2214b60 <stg_catch_frame_info> 0x802dfe7d0: 0x2214b20 <stg_unblockAsyncExceptionszh_ret_info> 0x802dfe7c8: 0x802a121b0 0x802dfe7c0: 0x1d61f60 <s99O_info> 0x802dfe7b8: 0x802a15848 0x802dfe7b0: 0x1 0x802dfe7a8: 0x2214b60 <stg_catch_frame_info> 0x802dfe7a0: 0x2214b40 <stg_blockAsyncExceptionszh_ret_info> 0x802dfe798: 0x22179e0 <stg_ap_v_info> 0x802dfe790: 0x802a17af0 0x802dfe788: 0x802a17b08 0x802dfe780: 0x802a121b0 0x802dfe778: 0x802a15870 0x802dfe770: 0x1d61a40 <s99v_info> 0x802dfe768: 0x217aef0 <s6Z8_info> 0x802dfe760: 0x802a17b88 0x802dfe758: 0x802d04e98 0x802dfe750: 0x2125f40 <s3eo_info> 0x802dfe748: 0x802a18970 0x802dfe740: 0x802a17b08 0x802dfe738: 0x802a17b78 0x802dfe730: 0x1d61620 <r37t_closure> 0x802dfe728: 0x400000000 0x802dfe720: 0x802d04ed0 0x802dfe718: 0x400000000 0x802dfe710: 0x220db80 <s31b_info> (gdb) I'm guessing that the culpable module defines the symbol "s31b". I added -ddump-stg to the options for the 6.6 build. (In fact, deleted everything, added the -ddump-stg option, rebuilt and did the run under the debugger shown above, to make certain that everything was consistent.) The symbol "s31b" is from the module GHC.Compat.Unicode. This may be getting somewhere. Compat.Unicode does not exist in 6.4.2, which may explain why the unregisterized 6.4.2 is quite reliable --- I used it to build alex and happy with no trouble. Also, the Unicode module makes foreign calls to C library functions. FreeBSD has its own implementation of libc, not the same as linux, which may explain why FreeBSD/amd64 chokes and Linux/amd64 works. So. I am asking if this argument makes sense, especially my interpretation of the RTS stack dump. Any suggestions are appreciated. Best Wishes, Greg

3 3

GADT + Newtype deriving = Erroneous errors
by Stefan O'Rear 29 Mar '07

29 Mar '07

This code causes GHC to incorrectly fail - the case *is* reachable. (I invented this technique in an attempt to directly access the internal System FC newtype coercion; it promised until a few minutes ago to solve all the EnumMap performance concerns.) stefan@stefans:/tmp$ cat A.lhs > {-# OPTIONS_GHC -fglasgow-exts #-} > > data IsIntT x where IsIntT :: IsIntT Int > > class IsIntC a where isInt :: IsIntT a > instance IsIntC Int where isInt = IsIntT > > newtype Foo = Foo Int deriving(IsIntC) > > x :: IsIntT Foo -> Int > x IsIntT = (Foo 2) + 2 > > y = x (isInt :: IsIntT Foo) stefan@stefans:/tmp$ ghci -v0 A.lhs A.lhs:11:4: Inaccessible case alternative: Can't match types `Foo' and `Int' In the pattern: IsIntT In the definition of `x': x IsIntT = (Foo 2) + 2 Stefan

7 11