July 2010 - Glasgow-haskell-users

behaviour of ghci on .c modules that are part of a library
by Axel Simon 16 Jul '10

16 Jul '10

Hi all, I'm trying to debug a segfault relating to the memory management in Gtk2Hs. Rather than make you read the ticket http://hackage.haskell.org/trac/gtk2hs/ticket/1183 , I'll describe the problem: - compiler 6.12.1 or 6.12.3 - darcs head of Gtk2Hs with #define DEBUG instead of #undef DEBUG in gtk/Graphics/UI/Gtk/General/hsthread.c - platform Ubuntu Linux, x86-64 - to reproduce: cd gtk2hs/gtk/demo/hello and run ghci World.hs and type 'main' A window with the "Hello World" button appears. After a few seconds, the GC runs and the finaliser of the GtkButton is run since the Haskell program no longer holds a reference to that object (only the GtkWindow in C land has). Thus, the GC calls a C function gtk2hs_g_object_unref_from_mainloop which is supposed to enqueue the object into a global data structure from which objects are later taken and g_object_unref is called on them. This global data structure is protected by a mutex, which is acquired using g_static_mutex_lock: void gtk2hs_g_object_unref_from_mainloop(gpointer object) { int mutex_locked = 0; if (threads_initialised) { #ifdef DEBUG printf("acquiring lock to add a %s object at %lx\n", g_type_name(G_OBJECT_TYPE(object)), (unsigned long) object); printf("value of lock function is %lx\n", (unsigned long) g_thread_functions_for_glib_use.mutex_lock); #endif g_rand_new(); #if defined( WIN32 ) EnterCriticalSection(&gtk2hs_finalizer_mutex); #else g_static_mutex_lock(&gtk2hs_finalizer_mutex); #endif mutex_locked = 1; } [..] The program prints: acquiring lock to add a GtkButton object at 22d8020 value of lock function is 0 zsh: segmentation fault ghci World Now the debugging weirdness starts. Whatever I do, I cannot get gdb to find the symbol gtk2hs_g_object_unref_from_mainloop. Since the function above is contained in a C file that comes with our Haskell library, I tried to add "cc-options: -g" and "cc-options: - ggdb -O0", but maybe somewhere symbols are stripped. So I added the bogus function call to "g_rand_new()" which is not called anywhere else and gdb stops as follows: acquiring lock to add a GtkButton object at 2105020 value of lock function is 0 [Switching to Thread 0x7ffff41ff710 (LWP 15735)] Breakpoint 12, 0x00007ffff115bfa0 in g_rand_new () from /usr/lib/ libglib-2.0.so This all seems reasonable, but: (gdb) bt #0 0x00007ffff115bfa0 in g_rand_new () from /usr/lib/libglib-2.0.so #1 0x00000000419b3792 in ?? () #2 0x00007ffff678f078 in ?? () i.e. the calling context is broken. I'm very, very sure that the caller is indeed the above mentioned function and since g_rand_new isn't called anywhere in my Haskell program (and otherwise the calling context would be sane). I'm also passing the address of gtk2hs_g_object_unref_from_mainloop as FinalizerPtr to all my ForeignPtrs, so there is no inlining going on. Back to the culprit, the call to g_static_mutex_lock. This is a macro that expands to *g_thread_functions_for_glib_use.mutex_lock where g_thread_functions_for_glib is a global variable that contains a lot of function pointers. At the break point, it contains this: (gdb) print g_thread_functions_for_glib_use $33 = {mutex_new = 0x7ffff0cd9820 <g_mutex_new_posix_impl>, mutex_lock = 0x7ffff6c8b3c0 <__pthread_mutex_lock>, mutex_trylock = 0x7ffff0cd97b0 <g_mutex_trylock_posix_impl>, mutex_unlock = 0x7ffff6c8ca00 <__pthread_mutex_unlock>, mutex_free = 0x7ffff0cd9740 <g_mutex_free_posix_impl>, [..] So the call to g_mutex_lock should call the function __pthread_mutex_lock but it calls NULL. I hoped that writing this email would give me a bit more insight into the problem, but for now I suspect that something overwrites either the stack or the code of the function. On the same platform, the compiled version prints: acquiring lock to add a GtkButton object at 1b05820 value of lock function is 7f7adcabd3c0 within mutex: adding finalizer to a GtkButton object! On Mac OS or i386, using ghci or ghc, version 6.10.4, it works as well. Now for the fun bit: on i386 using ghci version 6.12.1 it works too. So it's an x86-64 and ghc 6.12.1 bug. According to Christian Maeder who submitted the ticket, the problem persists in 6.12.3. Any hints and help appreciated, Cheers, Axel

2 4

Casting + eta reduction
by Louis Wasserman 16 Jul '10

16 Jul '10

Consider newtype Foo = Foo Int lift :: (Int -> a) -> Foo -> a lift f (Foo x) = f x Now, I'd expect this to compile with -O2 down to something like lift f = f `cast` (Foo -> a) but it doesn't. It seems that GeneralizedNewtypeDeriving assumes that these two things *are* equivalent, and it just directly casts the class dictionary. The implication would be that that GeneralizedNewtypeDeriving gives more efficient instances than you could *possibly* get if you wrote them by hand, which is very sad. Louis Wasserman wasserman.louis(a)gmail.com http://profiles.google.com/wasserman.louis

3 5

GADT related bug in GHC type checker
by George Giorgidze 13 Jul '10

13 Jul '10

Hi, I have encountered a bug in GHC type checker. I have stripped down my code to small manageable example that illustrates the bug: {-# LANGUAGE GADTs #-} {-# OPTIONS -Wall #-} module StrangeGADT where data Q a where ToQ :: (QA a) => a -> Q a Sum :: (QA a, Num a) => Q [a] -> Q a class QA a where toQ :: a -> Q a fromQ :: Q a -> a instance QA Int where toQ = ToQ fromQ q = case q of ToQ a -> a Sum as -> sum (fromQ as) instance QA a => QA [a] where toQ = ToQ fromQ q = case q of ToQ a -> a -- Sum _ -> ([] + 13) The above program typechecks but GHC wrongly warns that last pattern match is not exhaustive. Furthermore, if I uncomment the last line of the code it typechecks (without warnings) and does not reject ([] + 13) as type incorrect expression. It would be very much appreciated if someone could suggest how to circumvent the problem. Is there a version of GHC that behaves correctly in this case? Is this yet another instance of GADT related bugs already reported in GHC trac? or it is unrelated and I better report it as a separate ticket. Cheers, George

2 1

mallocForeignPtr vs. C
by Evan Laforge 13 Jul '10

13 Jul '10

So I was planning to pass a StorableVector to C. StorableVector uses ForeignPtrs to manage its memory, so it should just be a pointer pass. I happily wrote up a reference counting scheme so I could attach decref to the finalizer and have haskell and C cooperate about when to delete the pointer. However, then I found out that StorableVector uses mallocForeignPtr, which says it already has the equivalent of a free() finalizer attached, without actually using a finalizer. So it looks like I can't safely pass it to C unless I also guarantee that haskell still has a reference to it. So the only thing I thought of to deal with that is to pass C a FunPtr to a "wrapper" style haskell function. That function is never called, but it returns the vector so the haskell GC can't free the vector while the function is still alive. Then when C is done with the vector, it can call freeHaskellFunPtr via the usual freeHaskellFunPtr wrapper trick. But I'm not convinced that's actually enough because the C code is still running outside of a withForeignPtr. I would have to do something really hairy like call back to C from the haskell callback, wrapping it in withForeignPtr. Is there a better way to convince the GC that the pointer should remain alive until I explicitly release it from C? The ideal I think would be a ref count that would keep the pointer alive while >0, that could be decremented from C. Since one of the advantages of ForeignPtr using types is to pass them to C, this must be a problem someone has dealt with. A synchronous call that uses the foreign pointer temporarily is easy enough via withForeignPtr, but what about when I want to share ownership with C?

7 14

Massive slowdown in mwc-random after switching to use of primitive package
by Bryan O'Sullivan 13 Jul '10

13 Jul '10

Hi, Roman — Recently, I switched the mwc-random package ( http://hackage.haskell.org/package/mwc-random) over from running in the ST monad to using your primitive package. I didn't notice initially, but this caused a huge performance regression. mwc-random 0.4.1.1 uses ST internally, and runs the benchmarks/Quickie benchmark in 0.017 seconds. mwc-random 0.5.1.3 uses PrimMonad internally, and takes 1.328 seconds to run the same benchmark. That's about a factor of 80 slowdown. This causes a massive knock-on performance loss in packages such as criterion that need fast PRNGs. The problem is very easy to reproduce: cabal install mwc-random-0.4.1.1 cabal install mwc-random-0.5.1.3 darcs get http://darcs.serpentine.com/mwc-random/ cd mwc-random/benchmarks ghc -fforce-recomp -O -package mwc-random-0.4.1.1 --make Quickie -o quickie-411 ghc -fforce-recomp -O -package mwc-random-0.5.1.3 --make Quickie -o quickie-513 time ./quickie-411 time ./quickie-513 If you could shed any light, I'd be most grateful, as this has me a bit confounded.

4 11

Re: unsafeCoerce# between integral and floating point types
by John Meacham 09 Jul '10

09 Jul '10

On Fri, Jul 09, 2010 at 07:14:58AM -1000, Lennart Augustsson wrote: > That might not work if the alignment constraints differ between the two types. Easy enough to fix, just compare the alignments and alloca with the greater of the two. the optimizer should be able to optimize out the test. of course, this bug[1] makes it sort of moot in ghc at least. [1] http://hackage.haskell.org/trac/ghc/ticket/r2917 John -- John Meacham - ⑆repetae.net⑆john⑈ - http://notanumber.net/

1 0

Lazy IO and asynchronous callbacks?
by J. Garrett Morris 09 Jul '10

09 Jul '10

Hello everyone, I'm currently in the process of wrapping a C API, and I've run across an interesting possibility. Basically, the C API exposes non-blocking versions of some potentially long-running operations, and will invoke a callback to indicate that the long running operation has finished. For instance, I have something like: int longRunningReadOperation(int length, byte * buf, void (*callback)()) When callback is called, then there are length bytes in buf. What I'd like to do is wrap this in Haskell function like: longRunningReadOperation :: Int -> IO [Byte] such that the function returns immediately, and only blocks when someone pulls on the results if the callback hasn't been triggered yet. I figure I'm going to need unsafeInterleaveIO, but I'm not sure what to do in the computation I pass to it. I had a small hope that if the callback put the results into an MVar (say var), and I returned (unsafeInterleaveIO (readMVar var)) that might work, but it seems like if the readMVar blocks then the callback doesn't execute either. This doesn't surprise me, but it does leave me needing another option. I don't see a way to get the callback into a new Haskell thread, so that it can do the putMVar from there... am I missing something obvious? Thanks in advance! /g

4 3

unsafeCoerce# between integral and floating point types
by Daniel Fischer 09 Jul '10

09 Jul '10

The docs for unsafeCoerce# say: "The following uses of unsafeCoerce# are supposed to work (i.e. not lead to spurious compile-time or run-time crashes): # Casting any lifted type to Any # Casting Any back to the real type # Casting an unboxed type to another unboxed type of the same size (but not coercions between floating-point and integral types) ..." My experience so far is consistent with the assumption that e.g. unsafeCoerce# :: Word64 -> Double is like a cast from (uint64_t *) to (double *), i.e. a bit-pattern- preserving transformation (although by the docs, that use is undefined). Would that assumption generally hold for unsafeCoerce :: a -> b where a and b are single constructor data types wrapping unboxed types of the same bit-size and hence the use of unsafeCoerce# between such types would produce reliable results *on the same machine with the same OS (and GHC version?)* ? And what about e.g. unsafeCoerce# :: Word64# -> Double# ? By the docs, that isn't supposed to work. Is it not supposed to work only because it's not value-preserving (unsafeCoerce# 1## /=## 1.0##) or are there more pitfalls? If there are more pitfalls, is there any chance of getting a function which reinterprets the bit-patterns? Thanks, Daniel

4 4

Avoiding O(n^2) instances when using associated data types to unpack values into constructors
by Johan Tibell 06 Jul '10

06 Jul '10

Hi, I'm trying to create a data type for maps where both keys and values are unpacked into the data type constructors (see code at the end of this email). I achieve this using an associated data type of two arguments (`Map` in the code below). The problem I have is that this definition requires O(n^2) instances. I use a CPP macro to make it easier to create these instances but it doesn't address the real problem that the number of instances grows quadratically. Is there a better way to have GHC unpack the keys and values into the data constructors? Cheers, Johan \begin{code} {-# LANGUAGE CPP, MultiParamTypeClasses, TypeFamilies #-} -- | This module defines a type for maps of unboxed keys and values. module Data.AdaptMap (Unbox()) where -- | The size of the map. type Size = Int -- | A map of unboxed keys and values. class Unbox k a where data Map k a :: * -- Constructors/destructors used to implement functions on the map -- in a generic manner. tipCon :: Map k a binCon :: Size -> k -> a -> Map k a -> Map k a -> Map k a unMap :: Map k a -> b -> (Size -> k -> a -> Map k a -> Map k a -> b) -> b #define primMap(map,key,val,tipcon,bincon) \ instance Unbox key val where { \ data Map key val = tipcon \ | bincon {-# UNPACK #-} !Size \ {-# UNPACK #-} !key \ {-# UNPACK #-} !val \ !(Map key val) \ !(Map key val) \ ; tipCon = tipcon \ ; {-# INLINE tipCon #-} \ ; binCon = bincon \ ; {-# INLINE binCon #-} \ ; unMap t tk bk = case t of { \ tipcon -> tk \ ; bincon sz k v l r -> bk sz k v l r } \ ; {-# INLINE unMap #-} \ } -- Example instance with keys and values of type Int. primMap(IntIntMap,Int,Int,IntIntTip,IntIntBin) ------------------------------------------------------------------------ -- Construction empty :: Unbox k a => Map k a empty = tipCon singleton :: Unbox k a => k -> a -> Map k a singleton k x = binCon 1 k x tipCon tipCon \end{code}

2 1

BlockedIndefinitelyOnMVar exception
by Neil Mitchell 05 Jul '10

05 Jul '10

Hi, I have a very big and highly threaded program that generates a BlockedIndefinitelyOnMVar exception when run. I have spent a reasonable amount of time pouring over the source code, as has Max Bolingbroke. Neither of us have the slightest idea why it raises the exception. Some questions: * Does anyone know the exact sequence of actions that causes this exception to be thrown? I couldn't find it written down. * How confident are people that this exception does really mean that it is in a blocked state? Is there any chance the error could be raised incorrectly? * Any debugging tips for this problem? Thanks, Neil

3 10