[second attempt, this time from my bulk mailinglist address instead of the normal one.] Hi all, I recently tried to create a ffi-binding to gmp in ghc, and failed miserably. After a few days of debugging, simplifying the code and tearing my hear out, I'm slightly completely stumped, and crying for help ;) In short: calling gmp-functions from GHCI *with a prompt between* them seems to do Really Bad Things. (read: memory corruption) The long story: --------------- mpz_t p; str_test() { gmp_printf("%Zd\n", p); } void mpz_new() { mpz_init_set_si(p, 1); } foreign import ccall mpz_new :: IO () foreign import ccall str_test :: IO () Prelude Main> mpz_new Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 142833060 Prelude Main> str_test 142833060 Using other flags, importing extra modules, using CVS 6.3 (a few weeks old) or not compiling it before loading it in GHCI slightly changes the symptoms (other wrong numbers or make it happen later/earlier) but copypasting the code from main some 10 to 20 times seems to be a sure way to reproduce it. Simply running main doesn't seem to expose the problem. Now of course, GHCI uses Integer-ops during it's REPL, which I suspect is exactly what causes/exposes the problem. Am I doing (Un)Officially Forbidden Things? Is it time for a bug-report? Do I finally have to learn drinking coffee? ;) I'd be delighted to know. The full code is attached. TIA, Remi -- Nobody can be exactly like me. Even I have trouble doing it.
Hi,
please be aware that the RTS uses GMP as well, and upon
initialisation it sets GMP's 'memory functions' to allocate memory
from the RTS' heap. So, in the code below, the global variable
'p' will end up having components pointing into the heap.
Which is fine, until a GC occurs and the pointed-to
GMP allocated value is eventually stomped on by the storage
manager for some other purpose.
I'm _guessing_ that's the reason for the behaviour you're seeing.
--sigbjorn
----- Original Message -----
From: "Remi Turk"
[second attempt, this time from my bulk mailinglist address instead of the normal one.]
Hi all,
I recently tried to create a ffi-binding to gmp in ghc, and failed miserably. After a few days of debugging, simplifying the code and tearing my hear out, I'm slightly completely stumped, and crying for help ;)
In short: calling gmp-functions from GHCI *with a prompt between* them seems to do Really Bad Things. (read: memory corruption)
The long story: ---------------
mpz_t p;
str_test() { gmp_printf("%Zd\n", p); }
void mpz_new() { mpz_init_set_si(p, 1); }
foreign import ccall mpz_new :: IO () foreign import ccall str_test :: IO ()
Prelude Main> mpz_new Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 142833060 Prelude Main> str_test 142833060
Using other flags, importing extra modules, using CVS 6.3 (a few weeks old) or not compiling it before loading it in GHCI slightly changes the symptoms (other wrong numbers or make it happen later/earlier) but copypasting the code from main some 10 to 20 times seems to be a sure way to reproduce it.
Simply running main doesn't seem to expose the problem. Now of course, GHCI uses Integer-ops during it's REPL, which I suspect is exactly what causes/exposes the problem.
Am I doing (Un)Officially Forbidden Things? Is it time for a bug-report? Do I finally have to learn drinking coffee? ;) I'd be delighted to know.
The full code is attached.
TIA, Remi
-- Nobody can be exactly like me. Even I have trouble doing it.
---------------------------------------------------------------------------- ----
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
On Sun, Aug 08, 2004 at 07:34:04AM -0700, Sigbjorn Finne wrote:
Hi,
please be aware that the RTS uses GMP as well, and upon initialisation it sets GMP's 'memory functions' to allocate memory from the RTS' heap. So, in the code below, the global variable 'p' will end up having components pointing into the heap. Which is fine, until a GC occurs and the pointed-to GMP allocated value is eventually stomped on by the storage manager for some other purpose.
I'm _guessing_ that's the reason for the behaviour you're seeing.
Hm, I _was_ aware of mp_set_memory_functions being used by the RTS. I've seen it often enough in ltrace's ;) It does indeed sound rather plausible (and making big allocations and such does indeed cause it to happen earlier). At which point my next question is: what now? I don't feel really confident about my GHC-hacking skills (huh? skills? where? ;) so does that mean I'm out of luck? *looks* Am I correct that I'd have to copy any GMP-allocated memory to my own memory before returning from C and vice-versa? I hope not :( Happy hacking, Remi "3212th unfinished project" Turk -- Nobody can be exactly like me. Even I have trouble doing it.
FWIW, I couldn't reproduce this problem on my system (i.e. str_test
always printed "1"). GHC 6.2.1, libgmp 4.1.3, debian unstable
Abe
On Mon, 9 Aug 2004 17:57:14 +0200, Remi Turk
On Sun, Aug 08, 2004 at 07:34:04AM -0700, Sigbjorn Finne wrote:
Hi,
please be aware that the RTS uses GMP as well, and upon initialisation it sets GMP's 'memory functions' to allocate memory from the RTS' heap. So, in the code below, the global variable 'p' will end up having components pointing into the heap. Which is fine, until a GC occurs and the pointed-to GMP allocated value is eventually stomped on by the storage manager for some other purpose.
I'm _guessing_ that's the reason for the behaviour you're seeing.
Hm, I _was_ aware of mp_set_memory_functions being used by the RTS. I've seen it often enough in ltrace's ;) It does indeed sound rather plausible (and making big allocations and such does indeed cause it to happen earlier).
At which point my next question is: what now? I don't feel really confident about my GHC-hacking skills (huh? skills? where? ;) so does that mean I'm out of luck? *looks* Am I correct that I'd have to copy any GMP-allocated memory to my own memory before returning from C and vice-versa? I hope not :(
Happy hacking, Remi "3212th unfinished project" Turk
-- Nobody can be exactly like me. Even I have trouble doing it. _______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
On Mon, Aug 09, 2004 at 01:09:40PM -0400, Abraham Egnor wrote:
FWIW, I couldn't reproduce this problem on my system (i.e. str_test always printed "1"). GHC 6.2.1, libgmp 4.1.3, debian unstable
Abe
Same versions here, on an old heavily-patched/FUBAR rock linux 1.4 system. Does the following make any difference? (trying to cause GCing) Haskell/Mpz/weird% make ghci util.o -#include util.h PrimMpz.hs ___ ___ _ / _ \ /\ /\/ __(_) / /_\// /_/ / / | | GHC Interactive, version 6.2.1, for Haskell 98. / /_\\/ __ / /___| | http://www.haskell.org/ghc/ \____/\/ /_/\____/|_| Type :? for help. Loading package base ... linking ... done. Loading object (static) util.o ... done final link ... done Compiling Main ( PrimMpz.hs, interpreted ) Ok, modules loaded: Main. *Main> mpz_new *Main> sum (replicate (200*1000) 0) 0 *Main> str_test 1076535944 *Main> Groeten, Remi -- Nobody can be exactly like me. Even I have trouble doing it.
Ah, that triggers the bug for me as well. Oddly, calling
System.Mem.performGC once isn't enough, but twice is:
Ok, modules loaded: Main.
*Main> mpz_new
*Main> System.Mem.performGC
*Main> str_test
1
vs.
Ok, modules loaded: Main.
*Main> mpz_new
*Main> System.Mem.performGC
*Main> System.Mem.performGC
*Main> str_test
139406720
On Mon, 9 Aug 2004 19:36:37 +0200, Remi Turk
On Mon, Aug 09, 2004 at 01:09:40PM -0400, Abraham Egnor wrote:
FWIW, I couldn't reproduce this problem on my system (i.e. str_test always printed "1"). GHC 6.2.1, libgmp 4.1.3, debian unstable
Abe
Same versions here, on an old heavily-patched/FUBAR rock linux 1.4 system.
Does the following make any difference? (trying to cause GCing)
Haskell/Mpz/weird% make ghci util.o -#include util.h PrimMpz.hs ___ ___ _ / _ \ /\ /\/ __(_) / /_\// /_/ / / | | GHC Interactive, version 6.2.1, for Haskell 98. / /_\\/ __ / /___| | http://www.haskell.org/ghc/ \____/\/ /_/\____/|_| Type :? for help.
Loading package base ... linking ... done. Loading object (static) util.o ... done final link ... done Compiling Main ( PrimMpz.hs, interpreted ) Ok, modules loaded: Main. *Main> mpz_new *Main> sum (replicate (200*1000) 0) 0 *Main> str_test 1076535944 *Main>
Groeten, Remi
-- Nobody can be exactly like me. Even I have trouble doing it.
On Sun, Aug 08, 2004 at 07:34:04AM -0700, Sigbjorn Finne wrote:
Hi,
please be aware that the RTS uses GMP as well, and upon initialisation it sets GMP's 'memory functions' to allocate memory from the RTS' heap.
What about linking Haskell programs with C libraries that use GMP internally? Can it cause similar problems? Best regards, Tom -- .signature: Too many levels of symbolic links
participants (4)
-
Abraham Egnor -
Remi Turk -
Sigbjorn Finne -
Tomasz Zielonka