July 2002 - Glasgow-haskell-users

RE: make -j2 fails on SMP with ghc-5.04
by Simon Marlow 24 Jul '02

24 Jul '02

On Monday 15. July 2002 12:38, Simon Marlow wrote: > > > I have a dual Athlon box, make -j2 fails, but make works. > > > Is that normal? > > > > I did some tweaking to the build system to help -jN builds, but I > > haven't tested a complete build this way. Whereabouts does > it fall over > > for you? > > I also experienced such a strange failure with -j2 on my > uniprocessor Athlon. > The problem was the glafp-utils/mkdependC/mkdependC script > that is generated > during the build. The build gave a seemingly senseless error > message so I > looked at the script. Appearently, to concurrent make > processes had tried to > assemble it simultaneously, generating a pretty twisted version. Which version was this? The fixes for 'make -j' went in after 5.02. I've tried 'make -j2' in glafp-utils and it appears to work fine. Cheers, Simon

2 1

RE: comparison of execution speed of array types
by Simon Marlow 24 Jul '02

24 Jul '02

> I was curious about how fast the different array > implementations were so I > decided to test it. I wrote five programs all of which take an array > containing 50001 elements, reverse it a couple of times then sum > (modulo) them finally printing the sum. The programs are as follows: > > NormalArray -- uses the standard Array package for everything > NormalArrayReplace -- same as NormalArray but builds a new array > every time it is reversed > UnboxedArray -- uses UArray > UnboxedArrayReplace -- obvious > IOMutArray -- uses the IOArray from IOExts and everything is in the IO > monad Could you try IOUArray for completeness too? (An IOUArray is the unboxed version of IOArray, it can be found in Data.Array.IO). > I've stuck the code for these at the bottom of this message, > but here are the timing results: > > NormalArray 1.65u 0.20s 0:01.89 97.8% > NormalArrayReplace 2.40u 0.08s 0:02.56 96.8% > UnboxedArray 0.80u 0.04s 0:00.87 96.5% > UnboxedArrayReplace 1.83u 0.07s 0:01.99 95.4% > IOMutArray 0.60u 0.03s 0:01.09 57.7% GHC doesn't do any analysis to detect when a pure array can be updated in place, so I'm surprised that the Replace versions of the test performed much differently from the originals - I'd have expected them to be about the same, because in both cases a new array is being built for each reverse. I suspect the difference is due to more deforestation happening in the replace case; we know there are some issues with deforestation of the array operations at the moment. > clearly IOMutArray is the best, even outperforming the > UnboxedArray. Unfortunately, writing code in the IOMutArray format is > much uglier than writing it in the UnboxedArray or > NormalArray formats, > even though I know that I'm never going to refer to an old > version of the array, so inplace updates are a-okay. You could try testing DiffArray (Data.Array.Diff) which is optimised for in-place updates, and should show a bigger difference between the normal and 'replace' versions. It might be nearly as fast as IOArray (I don't think we've ever benchmarked it), and it doesn't need to be in the IO monad. Cheers, Simon

4 3

weird (bug?) in linear implicit parameters
by Hal Daume III 23 Jul '02

23 Jul '02

(guess who finally decided to learn about imp params :P) this looks like a bug to me, but of course i may be wrong. consider these definitions: > data BLSupply = BLSupply [Bool] > > instance Splittable BLSupply where > split (BLSupply l) = (BLSupply (True:l), BLSupply (False:l)) > > newBL (BLSupply bl) = bl basically we can name things by lists of bools we name lists using this function: > number :: (%ns :: BLSupply) => [a] -> [(a,[Bool])] > number [] = [] > number (x:xs) = (x,n) : number xs > where n = newBL %ns which works fine. in ghci: *ImpParam> let %ns = BLSupply [] in number "hello" [('h',[False]),('e',[False,True]),('l',[False,True,True]),('l',[False,True,True,True]),('o',[False,True,True,True,True])] Whee. now here's the wierd thing. suppose we remove the type signature from the number function. then, in ghci: *ImpParam> :t number forall a. (%ns :: BLSupply) => [a] -> [(a, [Bool])] *ImpParam> let %ns = BLSupply [] in number "hello" [('h',[]),('e',[]),('l',[]),('l',[]),('o',[])] what gives? why did it suddenly break even though it gets the type right? - hal -- Hal Daume III "Computer science is no more about computers | hdaume(a)isi.edu than astronomy is about telescopes." -Dijkstra | www.isi.edu/~hdaume

1 0

infixed implicit parameters
by Hal Daume III 23 Jul '02

23 Jul '02

in the cannonical sort with implicit params, i we write something like: mySort :: (?cmp :: a -> a -> Ordering) => [a] -> [a] mySort [] = [] mySort (x:xs) = insert x (mySort xs) where insert x [] = [x] insert x (y:ys) | ?cmp x y == GT = y : insert x ys | otherwise = x : y : ys but we cannot write for the second-to-last line: x `?cmp` y or x ?`cmp` y any chance this will be changed (my preference would be for the first one. - hal -- Hal Daume III "Computer science is no more about computers | hdaume(a)isi.edu than astronomy is about telescopes." -Dijkstra | www.isi.edu/~hdaume

1 0

RE: Type of newForeignPtr & addForeignPtrFinalizer
by Simon Marlow 23 Jul '02

23 Jul '02

[ sorry for the delay in replying to this one, I'm just trying to clear some of my backlog... ] > I have a feeling this may be a stupid question, but why are the > types of these.. > newForeignPtr :: Ptr a -> IO () -> IO (ForeignPtr a) > addForeignPtrFinalizer :: ForeignPtr a -> IO () -> IO () > (second arg being the finalizer) > > Won't a finaliser almost always take a pointer to the thing being > finalised as an argument? If so it would be more convienient > to have newForeignPtr.. > newForeignPtr :: Ptr a -> (Ptr a -> IO ()) -> IO (ForeignPtr a) > or maybe.. > newForeignPtr :: Ptr a -> (ForeignPtr a -> IO ()) -> IO > (ForeignPtr a) I think the reason was simplicity - I seem to recall we originally had the versions you suggested, but realised that it isn't *necessary* for the finalizer to take a Ptr as an argument. Anyway, there's an ongoing discussion on the FFI list about whether having arbitrary Haskell finalizers is really a plausible design, so it may be that the point is moot anyway. > The first of these is easy to implement yourself I suppose.. > myNewForeignPtr :: Ptr a -> (Ptr a -> IO ()) -> IO (ForeignPtr a) > myNewForeignPtr p fin = newForeignPtr p (fin p) > > The second seems to require this bit of weirdness.. > myNewForeignPtr :: Ptr a -> (ForeignPtr a -> IO ()) -> IO > (ForeignPtr a) > myNewForeignPtr p fin = do > newfp <- newForeignPtr p (return ()) > addForeignPtrFinalizer newfp (fin newfp) > return newfp You can do this more easily using fixIO: myNewForeignPtr p fin = do fixIO (\fp -> newForeignPtr p (fin fp)) Cheers, Simon

3 2

RE: Type of newForeignPtr & addForeignPtrFinalizer
by Simon Marlow 23 Jul '02

23 Jul '02

> On Monday 22 July 2002 12:33 pm, Simon Marlow wrote: > > > The second seems to require this bit of weirdness.. > > > myNewForeignPtr :: Ptr a -> (ForeignPtr a -> IO ()) -> IO > > > (ForeignPtr a) > > > myNewForeignPtr p fin = do > > > newfp <- newForeignPtr p (return ()) > > > addForeignPtrFinalizer newfp (fin newfp) > > > return newfp > > > > You can do this more easily using fixIO: > > > > myNewForeignPtr p fin = do > > fixIO (\fp -> newForeignPtr p (fin fp)) > > Thanks, neat (I think:-). I wonder if I might indulge myself with > another stupid question related to this, that is, why make the > distinction between Ptr and ForeignPtr at all? > > By definition a ForeignPtr has a non-zero number of finalisers > and Ptr has no finalisers. Couldn't you just allow ForeignPtr's > to have no finalisers and dispense with Ptr alltogether? Then > you could just add finalisers as required rather than converting > between types. It seems that when making a foreign binding you have > to make what seems (to me) an arbitrary choice between Ptr and > ForeignPtr arguments. I don't really understand the reason for > this extra complexity. There's a difference in representation between Ptr & ForeignPtr, which means that ForeignPtr might be significantly less efficient than Ptr. ForeignPtr is represented as (ForeignPtr ForeignPtr#), where ForeignPtr# is a real heap object of size 2. It needs to be a heap object so that the garbage collector can keep track of it and tell when it needs to be finalized. Ptr on the other hand is represented as (Ptr Addr#) where Addr# is just an unboxed address - it doesn't live on the heap, so the garbage collector can't track it. You might ask why we can't just put the finalizer on the boxed version (the (Ptr Addr#)) - and the answer is that the compiler might optimise away the box, which might cause the finalizer to run earlier than you intended. The compiler can't optimise away the box on a ForeignPtr#, because as far as the compiler is concerned a ForeignPtr# is a primitive indivisible object. It's quite a subtle point, but an important one. My advice is to use Ptr unless you really really need ForeignPtr. There are very few guarantees about the behaviour of finalizers - about the only guarantee you've got is that the finalizer will run "eventually", so it's difficult to provide any guarantees about memory use if you rely on finalizers to free memory. Cheers, Simon

1 0

RE: Foreign.destructArray
by Simon Marlow 23 Jul '02

23 Jul '02

> what happened to Foreign.destructArray? I can't seem to find it > anywhere... > > Also, has anyone thought of putting a search-engine front end > on to the > Haskell documentation? Even just a search on google using > > site:haskell.cs.yale.edu filetype:html inurl:docs > > would be nice, once google decides to index the new docs :). a closer > integration would be nice though... Good idea, I'll add that to the ToDo list. Cheers, Simon

1 0

comparison of execution speed of array types
by Hal Daume III 23 Jul '02

23 Jul '02

Hi all, I was curious about how fast the different array implementations were so I decided to test it. I wrote five programs all of which take an array containing 50001 elements, reverse it a couple of times then sum (modulo) them finally printing the sum. The programs are as follows: NormalArray -- uses the standard Array package for everything NormalArrayReplace -- same as NormalArray but builds a new array every time it is reversed UnboxedArray -- uses UArray UnboxedArrayReplace -- obvious IOMutArray -- uses the IOArray from IOExts and everything is in the IO monad I've stuck the code for these at the bottom of this message, but here are the timing results: NormalArray 1.65u 0.20s 0:01.89 97.8% NormalArrayReplace 2.40u 0.08s 0:02.56 96.8% UnboxedArray 0.80u 0.04s 0:00.87 96.5% UnboxedArrayReplace 1.83u 0.07s 0:01.99 95.4% IOMutArray 0.60u 0.03s 0:01.09 57.7% clearly IOMutArray is the best, even outperforming the UnboxedArray. Unfortunately, writing code in the IOMutArray format is much uglier than writing it in the UnboxedArray or NormalArray formats, even though I know that I'm never going to refer to an old version of the array, so inplace updates are a-okay. So my question is: how can I get better performance without wrapping everything in the IO (or some other) monad? - Hal Source code: -- NormalArray: module Main where import Data.Array testArray :: Array Int Int testArray = array (0,50000) [(i, (19*i+23) `mod` 911) | i <- [0..50000]] reverseArray :: Array Int Int -> Array Int Int reverseArray arr = arr // [(50000-i, arr!i) | i <- [0..50000]] sumArrayMod :: Array Int Int -> Int sumArrayMod arr = sumArrayMod' low 0 where sumArrayMod' pos sum | pos > high = sum | otherwise = sumArrayMod' (pos+1) ((sum + arr!pos) `mod` 911) (low,high) = bounds arr main = print $ sumArrayMod $ reverseArray $ reverseArray $ reverseArray $ reverseArray $ reverseArray $ reverseArray testArray -- NormalArrayReplace: module Main where import Data.Array testArray :: Array Int Int testArray = array (0,50000) [(i, (19*i+23) `mod` 911) | i <- [0..50000]] reverseArray :: Array Int Int -> Array Int Int reverseArray arr = array (0,50000) [(50000-i, arr!i) | i <- [0..50000]] sumArrayMod :: Array Int Int -> Int sumArrayMod arr = sumArrayMod' low 0 where sumArrayMod' pos sum | pos > high = sum | otherwise = sumArrayMod' (pos+1) ((sum + arr!pos) `mod` 911) (low,high) = bounds arr main = print $ sumArrayMod $ reverseArray $ reverseArray $ reverseArray $ reverseArray $ reverseArray $ reverseArray testArray -- UnboxedArray: module Main where import Data.Array.IArray import Data.Array.Unboxed testArray :: UArray Int Int testArray = array (0,50000) [(i, (19*i+23) `mod` 911) | i <- [0..50000]] reverseArray :: UArray Int Int -> UArray Int Int reverseArray arr = arr // [(50000-i, arr!i) | i <- [0..50000]] sumArrayMod :: UArray Int Int -> Int sumArrayMod arr = sumArrayMod' low 0 where sumArrayMod' pos sum | pos > high = sum | otherwise = sumArrayMod' (pos+1) ((sum + arr!pos) `mod` 911) (low,high) = bounds arr main = print $ sumArrayMod $ reverseArray $ reverseArray $ reverseArray $ reverseArray $ reverseArray $ reverseArray testArray -- UnboxedArrayReplace module Main where import Data.Array.IArray import Data.Array.Unboxed testArray :: UArray Int Int testArray = array (0,50000) [(i, (19*i+23) `mod` 911) | i <- [0..50000]] reverseArray :: UArray Int Int -> UArray Int Int reverseArray arr = array (0,50000) [(50000-i, arr!i) | i <- [0..50000]] sumArrayMod :: UArray Int Int -> Int sumArrayMod arr = sumArrayMod' low 0 where sumArrayMod' pos sum | pos > high = sum | otherwise = sumArrayMod' (pos+1) ((sum + arr!pos) `mod` 911) (low,high) = bounds arr main = print $ sumArrayMod $ reverseArray $ reverseArray $ reverseArray $ reverseArray $ reverseArray $ reverseArray testArray -- IOMutArray: module Main where import IOExts import Monad testArray :: IO (IOArray Int Int) testArray = newIOArray (0,50000) 0 >>= \arr -> mapM_ (uncurry (writeIOArray arr)) [(i, (19*i+23) `mod` 911) | i <- [0..50000]] >> return arr reverseArray :: IOArray Int Int -> IO () reverseArray arr = mapM_ (\i -> readIOArray arr i >>= \oldi -> readIOArray arr (50000-i) >>= \oldj -> writeIOArray arr i oldj >> writeIOArray arr (50000-i) oldi) [0..25000] sumArrayMod :: IOArray Int Int -> IO Int sumArrayMod arr = foldM (\s p -> readIOArray arr p >>= return . (`mod` 911) . (s+)) 0 [0..50000] main = testArray >>= \a -> reverseArray a >> reverseArray a >> reverseArray a >> reverseArray a >> reverseArray a >> reverseArray a >> sumArrayMod a >>= print -- Hal Daume III "Computer science is no more about computers | hdaume(a)isi.edu than astronomy is about telescopes." -Dijkstra | www.isi.edu/~hdaume

2 1

error building greencard with ghc 5.04
by Hal Daume III 22 Jul '02

22 Jul '02

probably a result of the libraries (probably = certainly), but i get: ../../src/green-card --target ghc StdDIS.gc ghc -fglasgow-exts -fno-prune-tydecls -fvia-C -package lang -package util -c StdDIS.hs -o StdDIS.o StdDIS.hs:39: Module `Foreign' does not export `ForeignObj' StdDIS.hs:39: Module `Foreign' does not export `makeStablePtr' StdDIS.hs:39: Module `Foreign' does not export `addForeignFinalizer' StdDIS.hs:39: Module `Foreign' does not export `makeForeignObj' gmake[2]: *** [StdDIS.o] Error 1 anyone have a patch? -- Hal Daume III "Computer science is no more about computers | hdaume(a)isi.edu than astronomy is about telescopes." -Dijkstra | www.isi.edu/~hdaume

2 1

RE: Deadlock detection different in ghc-5.04?
by Simon Marlow 22 Jul '02

22 Jul '02

> Hi, for some concurrency abstractions I decided to use deadlock > detection by means of exceptions which didn't work out as expected. > Below is some simplifed source code showing the problem: My assumption > was since GHC should be able to detect that the child still has a > reference to mv_should_work -- although in an exception handler -- the > RTS would never decide to kill the other thread. > > Unluckily, this assumption turned out to be false. Did I overestimate > GHC's capabilities or is this a bug? It's not a bug, but it's quite tricky to explain. The effect you're seeing is this: GHC 5.04 is a bit more general in dealing with deadlocks, in that it doesn't automatically assume that the main thread is alive (GHC 5.02 did make that assumption). In your example, at the point of the deadlock *both* threads are blocked on unreachable MVars, so both threads get an exception. The RTS has no way to know that if it woke up the child thread first, then this would also unblock the main thread. I've cut-n-pasted the trace below (you can get this by compiling the RTS with -DDEBUG and running the program with +RTS -D1; arguably this is so useful that we ought to take it out from under -DDEBUG). Cheers, Simon DEBUG (-D1): scheduler scheduler: created thread 1, stack size = f4 words scheduler: == scheduler: waiting for thread (1) scheduler: == scheduler: waiting for thread (1) scheduler: == scheduler: new main thread (1) scheduler: all threads: thread 1 @ 0x500c0000 is not blocked scheduler: -->> Running TSO 1 (0x500c0000) ThreadEnterGHC ... scheduler: created thread 2, stack size = f4 words scheduler: --<< thread 1 (0x500c0000; ThreadRunGHC) stopped, yielding scheduler: all threads: thread 2 @ 0x500c0400 is not blocked thread 1 @ 0x500c0000 is not blocked scheduler: -->> Running TSO 2 (0x500c0400) ThreadEnterGHC ... scheduler: --<< thread 2 (0x500c0400) stopped: is blocked on an MVar scheduler: all threads: thread 2 @ 0x500c0400 is blocked on an MVar thread 1 @ 0x500c0000 is not blocked scheduler: -->> Running TSO 1 (0x500c0000) ThreadRunGHC ... scheduler: --<< thread 1 (0x500c0000; ThreadRunGHC) stopped, yielding scheduler: all threads: thread 2 @ 0x500c0400 is blocked on an MVar thread 1 @ 0x500c0000 is not blocked scheduler: -->> Running TSO 1 (0x500c0000) ThreadRunGHC ... scheduler: --<< thread 1 (0x500c0000; ThreadRunGHC) stopped, yielding scheduler: all threads: thread 2 @ 0x500c0400 is blocked on an MVar thread 1 @ 0x500c0000 is not blocked scheduler: -->> Running TSO 1 (0x500c0000) ThreadRunGHC ... scheduler: --<< thread 1 (0x500c0000) stopped: is blocked on an MVar scheduler: all threads: thread 2 @ 0x500c0400 is blocked on an MVar thread 1 @ 0x500c0000 is blocked on an MVar scheduler: deadlocked, forcing major GC... scheduler: resurrecting thread 1 scheduler: raising exception in thread 1. scheduler: raising exception in thread 1. scheduler: resurrecting thread 2 scheduler: raising exception in thread 2. scheduler: raising exception in thread 2. scheduler: -->> Running TSO 2 (0x500bf000) ThreadEnterGHC ... scheduler: --++ thread 2 (0x500bf000) finished scheduler: all threads: thread 2 @ 0x500bf000 has completed thread 1 @ 0x500bf400 is not blocked scheduler: -->> Running TSO 1 (0x500bf400) ThreadEnterGHC ... main caught: thread blocked indefinitely scheduler: --++ thread 1 (0x500bf400) finished scheduler: all threads: thread 2 @ 0x500bf000 has completed thread 1 @ 0x500bf400 has completed == scheduler: main thread (1) finished scheduler: created thread 3, stack size = f4 words scheduler: == scheduler: waiting for thread (3) scheduler: == scheduler: waiting for thread (3) scheduler: == scheduler: new main thread (3) scheduler: all threads: thread 3 @ 0x500bd028 is not blocked thread 2 @ 0x500bf000 has completed thread 1 @ 0x500bf400 has completed scheduler: -->> Running TSO 3 (0x500bd028) ThreadEnterGHC ... scheduler: --++ thread 3 (0x500bd028) finished scheduler: all threads: thread 3 @ 0x500bd028 has completed thread 2 @ 0x500bf000 has completed thread 1 @ 0x500bf400 has completed == scheduler: main thread (3) finished

1 0