RE: Effect of large binaries on garbage collection

GHC does not copy big objects, so don't worry about the copying cost. (Instead of copying, it allocates big objects to (a contiguous series of) heap blocks, with no other objects in those blocks. Then the object can "move" simply by swizzling the heap-block descriptor.) Simon | -----Original Message----- | From: Adrian Hey [mailto:ahey@iee.org] | Sent: 04 March 2003 11:05 | To: glasgow-haskell-users@haskell.org | Subject: Effect of large binaries on garbage collection | | Hello, | | I'm writing a library which will require many blocks of binary data | of various sizes (some very large) to be stored in the heap. I'm a | little worried about the effect this will have on the efficiency of | garbage collection. I'm not sure how ghc gc works these days, but | I seem to remember it's a copying collector by default. If so it seems | a pity to waste time copying 10's of MBytes of binaries at each | collection. | | The options I'm considering are.. | | (1) Use Haskell heap space | Pros: Easy for me | Cons: May slow down gc | AFAICS I can't use anything like realloc | Current FFI proposals seem to prevent me from directly | accessing Haskell heap objects from C land (or have I | misunderstood?). | | (2) Use C heap space | Pros: Easy(ish) to use from C and Haskell ffi | Cons: Unless C heaps have improved a lot since I last looked | (which I doubt), it seems likely I will suffer from slow | allocation and fragmentation problems. | | (3) Write my own "sliding" heap manager and use finalisers for | garbage collection. | Pros: Can tailor it to work exactly the way I want. | Cons: More work for me, especially if I want the | result to be portable across OS's. | Might be a complete waste of time if my worries | about ghc heap management are groundless :-) | | Any advice? | | Thanks | -- | Adrian Hey | _______________________________________________ | Glasgow-haskell-users mailing list | Glasgow-haskell-users@haskell.org | http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

On Tuesday 04 March 2003 12:36, Simon Peyton-Jones wrote:
GHC does not copy big objects, so don't worry about the copying cost. (Instead of copying, it allocates big objects to (a contiguous series of) heap blocks, with no other objects in those blocks. Then the object can "move" simply by swizzling the heap-block descriptor.)
Thanks, looks like it's option (1) then. Could you tell me what Haskell type I have to use be able to pass a pointer to this binary to C land (genuine address, not StablePtr). I don't think the standard FFI allows this at all but, IIRC, the old ghc libraries allowed you to do this with a mutable byte array. Regards -- Adrian Hey

On Thursday 06 March 2003 10:55, Adrian Hey wrote:
On Tuesday 04 March 2003 12:36, Simon Peyton-Jones wrote:
GHC does not copy big objects, so don't worry about the copying cost. (Instead of copying, it allocates big objects to (a contiguous series of) heap blocks, with no other objects in those blocks. Then the object can "move" simply by swizzling the heap-block descriptor.)
Thanks, looks like it's option (1) then. Could you tell me what Haskell type I have to use be able to pass a pointer to this binary to C land (genuine address, not StablePtr). I don't think the standard FFI allows this at all but, IIRC, the old ghc libraries allowed you to do this with a mutable byte array.
Does anybody know the answer to this? please.. pretty please..:-) Sorry if this is a case of me failing to RTFM, but I don't think it is. Paragraph 8.1.1 of the users guide says.. "The types ByteArray and MutableByteArray may be used as basic foreign types (see FFI Addendum, Section 3.2). In C land, they map to (char *)." I can't find any way in the Base libs (or what's left of the old libs) to create a ByteArray or MutableByteArray, which leads me to suspect that they no longer exist. Should I use something else instead? Thanks -- Adrian Hey
participants (2)
-
Adrian Hey
-
Simon Peyton-Jones