Hey guys, Recent changes to malloc on OpenBSD mean that nhc98 builds now die with the "OS allocated a heap in high memory" error, as malloc is putting things above the 0x80000000 line. Switching the heap allocation to use mmap with a hint to allocate somewhere lower fixes it at least on x86/openbsd. The ghc testsuite runs, for example. The mmap code is stolen from MBlock.c in ghc. Any options on this? Is it reasonable? (I'd like to use it as a custom patch in the OpenBSD port of nhc98, but want to check I'm not doing anything hideously wrong :) -- Don Here's what I'm doing: --- src/runtime/Kernel/collector.c.orig Tue Mar 8 03:53:38 2005 +++ src/runtime/Kernel/collector.c Sat Oct 22 19:05:22 2005 @@ -7,6 +7,11 @@ /*#include "runtime.h" -- already included in node.h */ #include "mark.h" +#include <errno.h> +#include <sys/types.h> +#include <sys/mman.h> + + /*#define HEAPSIZE 100000 -- defined in top-level Makefile at config-time */ #define GCSTACKSIZE 20000 @@ -45,11 +50,22 @@ void initGc(Int hpSize,NodePtr *ihp,Int { Int totalSize = hpSize+spSize; Int tableSize = (totalSize+WORDSIZE)/(WORDSIZE+1)+1; /* Last one for end of marked */ + int size = (int)totalSize * sizeof(Node); - if(NULL == (hpStart = malloc ((int)totalSize * sizeof(Node)))) { - fprintf(stderr,"Not enough memory for heap and stack.\n"); - exit(-1); - } + /* Ask for memory well below the 0x80000000 line. The nhc runtime + * reserves the high bit on addresses for its own purposes */ + hpStart = mmap((void *)0x40000000, size, + PROT_EXEC|PROT_READ|PROT_WRITE, + MAP_PRIVATE|MAP_ANON, -1 , 0); + + if (hpStart == MAP_FAILED) { + if (errno == ENOMEM) { + fprintf(stderr,"initGc: out of memory (requested %lu bytes)", size); + exit(-1); + } else { + fprintf(stderr, "initGc: mmap failed: %s", strerror(errno)); + } + } hpEnd = hpStart + totalSize;
Also, there's some gnu-isms here that add an unnecessary build dependency. Thanks to Marc Espie for this patch. Cheers, Don --- script/tprofprel.orig Sun Mar 13 11:56:03 2005 +++ script/tprofprel Sun Mar 13 11:56:38 2005 @@ -7,20 +7,8 @@ case $# in 1) exit 1;; esac -case $BASH in - "") if ( bash --version -e >/dev/null 2>&1 ) - then exec bash $0 "$@" - fi ;; - *) ;; -esac +increment() { echo $1 + 1 | bc; } - -# Nasty compatibility stuff between /bin/sh/ and bash for arithmetic. -if sh --version 2>/dev/null | grep -i gnu >/dev/null -then increment() { ( let x=$1+1; echo $x; ); } -else increment() { echo $1 + 1 | bc; } -fi - BUILTIN='Builtin' #Runtime Module added to beginning MODNAMES='' nummods=0 @@ -100,7 +88,7 @@ then echo ' char *strc;' echo ' int i = strlen(str)+1;' echo ' if(0==(strc = (char *)malloc(i))) {' - echo ' fprintf(stderr,"No space to duplicate \"%s\"\n",str);' + echo ' fprintf(stderr,"No space to duplicate \"%s\"\\n",str);' echo ' exit(-1);' echo ' }' echo ' strcpy(strc,str);'
dons@cse.unsw.edu.au (Donald Bruce Stewart) writes:
Switching the heap allocation to use mmap with a hint to allocate somewhere lower fixes it at least on x86/openbsd.
Thanks for the fix(es). Eventually the high-memory problem will be fixed by not using the top bit for GC, but until then, your approach will be a useful workaround. I'll check it into CVS. Regards, Malcolm
Malcolm.Wallace:
dons@cse.unsw.edu.au (Donald Bruce Stewart) writes:
Switching the heap allocation to use mmap with a hint to allocate somewhere lower fixes it at least on x86/openbsd.
Thanks for the fix(es). Eventually the high-memory problem will be fixed by not using the top bit for GC, but until then, your approach will be a useful workaround. I'll check it into CVS.
How do we deal with portability issues in nhc? The code I submitted will run find on the BSDs and Linux, but for Solaris, Irix, HPUS, and Darwin when need slight variations (which ghc currently has) (and we're stuck with malloc on Windows). Does nhc have some equivalent of the #if defined(solaris2_HOST_OS) || defined(irix_HOST_OS) macros? -- Don
Am Sonntag, 23. Oktober 2005 03:47 schrieb Donald Bruce Stewart:
How do we deal with portability issues in nhc? The code I submitted will run find on the BSDs and Linux, but for Solaris, Irix, HPUS, and Darwin when need slight variations (which ghc currently has) (and we're stuck with malloc on Windows).
Does nhc have some equivalent of the #if defined(solaris2_HOST_OS) || defined(irix_HOST_OS) macros?
[ putting on my autotools hat again... ] Unless you *really* care about the OS, never use #ifdefs like the ones above, they are a source of constant maintenance trouble. Feature-based #ifdefs are the way to go, and this is e.g. the central point of the highly successful autotools toolchain. In our case at hand we should check for the availability of mmap (and perhaps its flags) and use the code you gave if it is available, falling back to malloc if it is not. Even if we are lazy and don't want to do a real feature detection at configuration time, using feature-based #ifdefs in the code is still the right way to go, because then the build system can #define the feature-based #ifdefs in a single place depending on the OS, which is much easier to maintain. Cheers, S.
dons@cse.unsw.edu.au (Donald Bruce Stewart) writes:
Does nhc have some equivalent of the #if defined(solaris2_HOST_OS) || defined(irix_HOST_OS) macros?
Bearing in mind Sven's good advice to use feature tests in preference to OS tests where possible, you can test any of the symbols ordinarily defined by your C compiler. See include/HsFFI.h and include/newmacros.h for examples. #if defined(__sparc__) || defined (__sgi) || etc Regards, Malcolm
participants (3)
-
dons@cse.unsw.edu.au -
Malcolm Wallace -
Sven Panne