the new GC, anyone got a bitset?
So I am exceedingly pleased with what has been happening with the new GC in jhc, enough that I am going to push for a major release in not long. With some careful profiling and tweaking I have been able to ensure enabling GC is a fairly dramatic win in terms of performance and memory for the majority of haskell programs. There is just one thing that is bothering me, I still rely on libJudy in the runtime for its extremely fast bitset operation. Basically, in order to determine if an address is allocated on the heap vs on the stack or in the bss or data segments, I use a bitmap, adding an entry for each 4M chunk of memory dedicated to heap pages. on a 64 bit system, using a plain bit array is just not feasable, libJudy has a very fast bitset that I utilize, but it is not exactly lightweight and is LGPL which may cause issues for creating statically linked executables from jhc. So, I am looking for a very fast (in querying, updates need not be that fast) bitset that maps a 'uintptr_t' to either true or false, chances are it will be very sparse. I'd like something I can embed in the runtime, so it should not be very big. jhc executables are on the order of 8k maybe compiled, so adding 4k of bitmap code for just this isn't really ideal. So, anyone have something in the public domain sitting around? John -- John Meacham - ⑆repetae.net⑆john⑈ - http://notanumber.net/
* John Meacham:
There is just one thing that is bothering me, I still rely on libJudy in the runtime for its extremely fast bitset operation. Basically, in order to determine if an address is allocated on the heap vs on the stack or in the bss or data segments, I use a bitmap, adding an entry for each 4M chunk of memory dedicated to heap pages. on a 64 bit system, using a plain bit array is just not feasable, libJudy has a very fast bitset that I utilize, but it is not exactly lightweight and is LGPL which may cause issues for creating statically linked executables from jhc.
You could use mmap with PROT_NONE to get a large chunk of contiguous address space, and allocate the parts you actually use with mprotect, setting PROT_READ and PROT_WRITE. Then it's far easier to tell which addresses are part of the heap and which aren't.
On Fri, Jun 04, 2010 at 09:51:53PM +0200, Florian Weimer wrote:
* John Meacham:
There is just one thing that is bothering me, I still rely on libJudy in the runtime for its extremely fast bitset operation. Basically, in order to determine if an address is allocated on the heap vs on the stack or in the bss or data segments, I use a bitmap, adding an entry for each 4M chunk of memory dedicated to heap pages. on a 64 bit system, using a plain bit array is just not feasable, libJudy has a very fast bitset that I utilize, but it is not exactly lightweight and is LGPL which may cause issues for creating statically linked executables from jhc.
You could use mmap with PROT_NONE to get a large chunk of contiguous address space, and allocate the parts you actually use with mprotect, setting PROT_READ and PROT_WRITE. Then it's far easier to tell which addresses are part of the heap and which aren't.
Yeah, there are various schemes to use mprotect to form write/read barriers for incremental GCs. For now, I wanted to stick to producing portable code, however that may be something I explore later. A portable fallback would still be needed though, so might as well start with that rather than making assumptions about the target platform. John -- John Meacham - ⑆repetae.net⑆john⑈ - http://notanumber.net/
participants (2)
-
Florian Weimer -
John Meacham