On 09 August 2004 16:57, Remi Turk wrote:
On Sun, Aug 08, 2004 at 07:34:04AM -0700, Sigbjorn Finne wrote:
Hi,
please be aware that the RTS uses GMP as well, and upon initialisation it sets GMP's 'memory functions' to allocate memory from the RTS' heap. So, in the code below, the global variable 'p' will end up having components pointing into the heap. Which is fine, until a GC occurs and the pointed-to GMP allocated value is eventually stomped on by the storage manager for some other purpose.
I'm _guessing_ that's the reason for the behaviour you're seeing.
Hm, I _was_ aware of mp_set_memory_functions being used by the RTS. I've seen it often enough in ltrace's ;) It does indeed sound rather plausible (and making big allocations and such does indeed cause it to happen earlier).
At which point my next question is: what now? I don't feel really confident about my GHC-hacking skills (huh? skills? where? ;) so does that mean I'm out of luck? *looks* Am I correct that I'd have to copy any GMP-allocated memory to my own memory before returning from C and vice-versa? I hope not :(
GHC's use of GMP does cause problems if you want to use GMP for your own purposes, or if you link with external code that wants to use GMP. The real problem is that GMP has internal state, which means it can't be used in a modular way. But there's not much we can do about that. Possibilities: - Rename all the symbols in our GMP to be unique. (sounds hard) - Replace GMP with something else (someone is working on this, I believe). - try to get two copies of GMP into your program by pre-linking the RTS with one copy, then linking the rest of the program with the other copy. I'm being intentionally vague here - I feel that this ought to be possible, but it's an ugly hack at best. - reset GMP's memory allocators before calling it from your code, and set them back to the RTS allocators afterward. Slow, but it should work. It doesn't solve the problem properly though: external libraries which use GMP are still broken. Cheers, Simon
Hi! On Tue, Aug 10, 2004 at 12:59:46PM +0100, Simon Marlow wrote:
On 09 August 2004 16:57, Remi Turk wrote:
Hm, I _was_ aware of mp_set_memory_functions being used by the RTS. I've seen it often enough in ltrace's ;) It does indeed sound rather plausible (and making big allocations and such does indeed cause it to happen earlier).
At which point my next question is: what now? I don't feel really confident about my GHC-hacking skills (huh? skills? where? ;) so does that mean I'm out of luck? *looks* Am I correct that I'd have to copy any GMP-allocated memory to my own memory before returning from C and vice-versa? I hope not :(
GHC's use of GMP does cause problems if you want to use GMP for your own purposes, or if you link with external code that wants to use GMP. The real problem is that GMP has internal state, which means it can't be used in a modular way. But there's not much we can do about that.
Possibilities:
[...]
- reset GMP's memory allocators before calling it from your code, and set them back to the RTS allocators afterward. Slow, but it should work. It doesn't solve the problem properly though: external libraries which use GMP are still broken.
I would be careful though: http://www.swox.com/gmp/manual/Custom-Allocation.html Be sure to call mp_set_memory_functions only when there are no active GMP objects allocated using the previous memory functions! Usually that means calling it before any other GMP function. Greetings, Carsten -- Carsten Schultz (2:38, 33:47), FB Mathematik, FU Berlin http://carsten.codimi.de/ PGP/GPG key on the pgp.net key servers, fingerprint on my home page.
- Rename all the symbols in our GMP to be unique. (sounds hard)
Nah, easy. First get a list of symbols: nm libgmp.a | grep gmp [Supply a few flags to nm to get a better list.] Then turn it into a list of #defines: #define gmp_blah ghc_gmp_blah ... #include your list of #defines into some header file that everyone eats. make clean; make You can generate the list by hand or have it rebuilt by make every time. -- Alastair Reid
On Tue, Aug 10, 2004 at 12:59:46PM +0100, Simon Marlow wrote:
GHC's use of GMP does cause problems if you want to use GMP for your own purposes, or if you link with external code that wants to use GMP. The real problem is that GMP has internal state, which means it can't be used in a modular way. But there's not much we can do about that.
Possibilities:
- Rename all the symbols in our GMP to be unique. (sounds hard) and ugly
- Replace GMP with something else (someone is working on this, I believe). Do you have a pointer? It sounds interesting. *see below*
- try to get two copies of GMP into your program by pre-linking the RTS with one copy, then linking the rest of the program with the other copy. I'm being intentionally vague here - I feel that this ought to be possible, but it's an ugly hack at best. I'm not sure I share your feelings about that ;) It sounds like symbol-clash-hell. But quite possibly I'm just being ignorant.
- reset GMP's memory allocators before calling it from your code, and set them back to the RTS allocators afterward. Slow, but it should work. It doesn't solve the problem properly though: external libraries which use GMP are still broken. It does indeed seem to work, after a quick test. (new util.c attached)
And it does solve _my_ immediate "problem": I can surround every gmp-operation with a gmp_begin()/gmp_end() and pretend to be happy. (and anyway, I'm just messing around, failure merely means I've got yet another unfinished project ;)) Part of the reason for all this messy FFIing is your post: http://www.haskell.org/pipermail/glasgow-haskell-users/2004-June/006767.html If Integers where implemented via the FFI that would make it quite a bit easier to special-case e.g. (^) and Show for Integer. (IIRC, GMP's mpz-to-string recently got a huge speedup, it would be nice if GHC would automagically profit of that..)
Cheers, Simon
Happy hacking & keep up the good work ;) Remi -- Nobody can be exactly like me. Even I have trouble doing it.
participants (4)
-
Alastair Reid -
Carsten Schultz -
Remi Turk -
Simon Marlow