August 2004 - Glasgow-haskell-users

RE: GHCI/FFI/GMP/Me madness
by Simon Marlow 10 Aug '04

10 Aug '04

On 09 August 2004 16:57, Remi Turk wrote: > On Sun, Aug 08, 2004 at 07:34:04AM -0700, Sigbjorn Finne wrote: >> Hi, >> >> please be aware that the RTS uses GMP as well, and upon >> initialisation it sets GMP's 'memory functions' to allocate memory >> from the RTS' heap. So, in the code below, the global variable >> 'p' will end up having components pointing into the heap. >> Which is fine, until a GC occurs and the pointed-to >> GMP allocated value is eventually stomped on by the storage >> manager for some other purpose. >> >> I'm _guessing_ that's the reason for the behaviour you're seeing. > > Hm, I _was_ aware of mp_set_memory_functions being used by the RTS. > I've seen it often enough in ltrace's ;) > It does indeed sound rather plausible (and making big allocations > and such does indeed cause it to happen earlier). > > At which point my next question is: what now? I don't feel really > confident about my GHC-hacking skills (huh? skills? where? ;) so > does that mean I'm out of luck? > *looks* Am I correct that I'd have to copy any GMP-allocated > memory to my own memory before returning from C and vice-versa? > I hope not :( GHC's use of GMP does cause problems if you want to use GMP for your own purposes, or if you link with external code that wants to use GMP. The real problem is that GMP has internal state, which means it can't be used in a modular way. But there's not much we can do about that. Possibilities: - Rename all the symbols in our GMP to be unique. (sounds hard) - Replace GMP with something else (someone is working on this, I believe). - try to get two copies of GMP into your program by pre-linking the RTS with one copy, then linking the rest of the program with the other copy. I'm being intentionally vague here - I feel that this ought to be possible, but it's an ugly hack at best. - reset GMP's memory allocators before calling it from your code, and set them back to the RTS allocators afterward. Slow, but it should work. It doesn't solve the problem properly though: external libraries which use GMP are still broken. Cheers, Simon

4 3

RE: GHCI/FFI/GMP/Me madness
by Simon Marlow 10 Aug '04

10 Aug '04

On 10 August 2004 14:08, Alastair Reid wrote: >> - Rename all the symbols in our GMP to be unique. (sounds hard) > > Nah, easy. > > First get a list of symbols: nm libgmp.a | grep gmp > [Supply a few flags to nm to get a better list.] > > Then turn it into a list of #defines: > > #define gmp_blah ghc_gmp_blah > ... > > #include your list of #defines into some header file that everyone > eats. > > make clean; make > > > You can generate the list by hand or have it rebuilt by make every > time. Hmmm... well maybe. We'd have to be sure to always use the version of GMP that comes with GHC - currently we use an installed dynamic version if one is available. Statically linking GMP runs into licensing problems, so I'd have to figure out how to build it dynamically. The native codegen has to use the revised names, too. (the #defines won't work there because it mentions the names inside strings). I'd bet there are almost certainly some other gotchas. Cheers, SImon

1 0

RE: GHCI/FFI/GMP/Me madness
by MR K P SCHUPKE 10 Aug '04

10 Aug '04

Re GMP, Why not provide more GMP functions as primitives on the Integer type, and avoid the need to call out to GMP via the FFI? Keean.

1 0

GHCI/FFI/GMP/Me madness
by Remi Turk 09 Aug '04

09 Aug '04

[second attempt, this time from my bulk mailinglist address instead of the normal one.] Hi all, I recently tried to create a ffi-binding to gmp in ghc, and failed miserably. After a few days of debugging, simplifying the code and tearing my hear out, I'm slightly completely stumped, and crying for help ;) In short: calling gmp-functions from GHCI *with a prompt between* them seems to do Really Bad Things. (read: memory corruption) The long story: --------------- mpz_t p; str_test() { gmp_printf("%Zd\n", p); } void mpz_new() { mpz_init_set_si(p, 1); } foreign import ccall mpz_new :: IO () foreign import ccall str_test :: IO () Prelude Main> mpz_new Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 142833060 Prelude Main> str_test 142833060 Using other flags, importing extra modules, using CVS 6.3 (a few weeks old) or not compiling it before loading it in GHCI slightly changes the symptoms (other wrong numbers or make it happen later/earlier) but copypasting the code from main some 10 to 20 times seems to be a sure way to reproduce it. Simply running main doesn't seem to expose the problem. Now of course, GHCI uses Integer-ops during it's REPL, which I suspect is exactly what causes/exposes the problem. Am I doing (Un)Officially Forbidden Things? Is it time for a bug-report? Do I finally have to learn drinking coffee? ;) I'd be delighted to know. The full code is attached. TIA, Remi -- Nobody can be exactly like me. Even I have trouble doing it.

4 6

RE: FFI, Ghc and Libs
by Simon Marlow 09 Aug '04

09 Aug '04

GHCi supports loading .o and .DLL files only, I'm afriad. Although I believe it might be possible to link against .LIB files when linking a standalone binary. Cheers, Simon ________________________________ From: glasgow-haskell-users-bounces(a)haskell.org [mailto:glasgow-haskell-users-bounces@haskell.org] On Behalf Of Dominik Oesterreich Sent: 02 August 2004 16:22 To: glasgow-haskell-users(a)haskell.org Subject: FFI, Ghc and Libs Hello, i have one Problem with ghc (and FFI). I wrote some functions in C and compiled that file. After that I typed: ghci -fffi komplett.obj test.lhs -l apr.lib and got the following error meassage: Loading package base ... linking ... done Warning: ignoring unrecognised input 'apr.lib' Loading object (static) kompett.obj ... done Loading object (dynamic) ... failed. Dynamic Linker error message was: addDLL: unknown error Whilst trying to load (dynamic) Directories to search are: C:\ghc\ghc-6.2.1\bin ghc.exe: user specified .o/.so/.DLL could not be loaded I have to use different Libraries from Subversion (subversion.tigris.org => http://subversion.tigris.org/files/documents/15/14892/svn-win32-1.0.6_de v.zip). Dominik

1 0

RE: ghci: catching up with hugs?-)
by Simon Marlow 09 Aug '04

09 Aug '04

On 29 July 2004 17:48, Claus Reinke wrote: > - especially when working with gui libs, I often find myself wanting > to know which instances some type belongs to (as that determines > the attributes/properties/etc one may use with that type). Absolutely. This is on my ToDo list, but I haven't got around to it yet. > - since ghc now keeps better source location info, how about ":find > <name>"? Yes, another good suggestion. I don't have the time to tackle these right now, but they're good small projects for anyone looking for something to contribute to GHC. > - the ":set <something>" command in ghci "feeds" the ghc command > line, but how can I figure out the current settings (especially > paths and packages)? This is really a bug - and it's another thing that's been lurking on my ToDo list for a long time. Cheers, Simon

1 0

Double -> CDouble, realToFrac doesn't work
by Hal Daume III 08 Aug '04

08 Aug '04

I need to convert a Double to a CDouble to pass to a C function. In the past, when I've asked how to do this, I'm told: realToFrac. This works in most cases, but recently gave me a problem that took me *forever* to track down: Prelude Foreign.C> (0 :: CDouble) / 0 NaN Prelude Foreign.C> (0 :: Double) / 0 NaN Prelude Foreign.C> realToFrac ((0 :: Double) / 0) :: CDouble -Infinity yikes! the NaN got turned into a -Infinity!!! aside from manually checking for 'strange' Double/CDouble values and wrapping realToFrac, is there a better way? also, does this count as a bug? -- Hal Daume III | hdaume(a)isi.edu "Arrest this man, he talks in maths." | www.isi.edu/~hdaume

2 1

getDirectoryContents fails under Windows
by Krasimir Angelov 07 Aug '04

07 Aug '04

The getDirectoryContents function from CVS sources fails under Windows. It seems like in the __hscore_end_of_dir function from HsBase.h should return 0 under Windows and ENOENT under other platforms. Cheers, Krasimir __________________________________ Do you Yahoo!? Yahoo! Mail - 50x more storage than other providers! http://promotions.yahoo.com/new_mail

1 0

GHCI/FFI/GMP/Me madness
by Remi Turk 05 Aug '04

05 Aug '04

Hi all, I recently tried to create a ffi-binding to gmp in ghc, and failed miserably. After a few days of debugging, simplifying the code and tearing my hear out, I'm slightly completely stumped, and crying for help ;) In short: calling gmp-functions from GHCI *with a prompt between* them seems to do Really Bad Things. (read: memory corruption) The long story: --------------- mpz_t p; str_test() { gmp_printf("%Zd\n", p); } void mpz_new() { mpz_init_set_si(p, 1); } foreign import ccall mpz_new :: IO () foreign import ccall str_test :: IO () Prelude Main> mpz_new Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 1 Prelude Main> str_test 142833060 Prelude Main> str_test 142833060 Using other flags, importing extra modules, using CVS 6.3 (a few weeks old) or not compiling it before loading it in GHCI slightly changes the symptoms (other wrong numbers or make it happen later/earlier) but copypasting the code from main some 10 to 20 times seems to be a sure way to reproduce it. Simply running main doesn't seem to expose the problem. Now of course, GHCI uses Integer-ops during it's REPL, which I suspect is exactly what causes/exposes the problem. Am I doing (Un)Officially Forbidden Things? Is it time for a bug-report? Do I finally have to learn drinking coffee? ;) I'd be delighted to know. The full code is attached. TIA, Remi -- Nobody can be exactly like me. Even I have trouble doing it.

1 0

RE: hPutBuf synchronous?
by Simon Marlow 04 Aug '04

04 Aug '04

On 27 July 2004 11:17, David Roundy wrote: > I've been trying to track down a "Bus error" that one of my users is > getting on the macintosh, and am at the point of having just one idea > as to what might be causing it. > > The error happens during garbage collection, as far as I can tell. He > compiled darcs using ghc 6.2.1. > > My only theory is that perhaps hGetBuf (or maybe hPutBuf) isn't > actually synchronous on some platforms? If this were the case, of > course, using h{Get,Put}Buf on a pointer within withForeignPtr might > be dangerous, since the ForeignPtr could be GCed before the hGetBuf > actually takes place. It's hard to imagine that this actually is the > case, but I'm really running out of ideas as to what could cause this > bus error he's seeing. It's not 100% reproducible, but goes away if > he makes the GC triggered less often (in darcs I trigger GCs manually > before doing a large mallocForeignPtr to keep memory usage down), and > when we added a debug message finalizer, he sees it considerably less > often. Alas, I've been unable to reproduce it myself, and his > repository that displays this behavior contains private data. > > Any suggestions and/or insight would be welcome. hGetBuf is synchronous; the system will not write to the buffer after hGetBuf returns. hPutBuf is also synchronous, in the sense that the data to be written is copied out of the buffer passed in, and either written right away or placed in the Handle's buffer ready to be written at a later time. So I don't think your hypothetical scenario is happening, sounds like it must be something else. There has been one GC bug found since 6.2.1, which could perhaps be the cause. Try getting your user to compile up an RTS from the latest CVS sources of the 6.2 branch and see if that helps. Cheers, Simon

2 1