On Thu, Jul 05, 2018 at 09:32:31AM -0400, Ben Gamari wrote:

...

While there's no binary distribution for FreeBSD at haskell.org, as of June 30 there's a FreeBSD ghc-8.4.3 port.

https://svnweb.freebsd.org/ports/head/lang/ghc/Makefile?view=log

Ahh, great! Thanks for doing this. I'll try to replicate this while doing the builds for 8.6.

For the record, in case anyone gets the wrong impression, I am not one of the port maintainers (who deserve the credit for creating the port). I'm just a happy user of the port, which was upgraded from 8.4.2 to 8.4.3 on Jun 30th. No issues with either version on FreeBSD 11.1-RELEASE-p10. My meagre contribution amounts to reporting success with: # cd /usr/ports/lang/ghc # make config # make package # pkg install work/pkg/ghc-8.4.3.txz The "make config" options include: BOOT Use installed GHC for bootstrapping which speeds up upgrade builds. -- Viktor.

On Thu, Jul 05, 2018 at 08:13:33PM -0400, Viktor Dukhovni wrote:

On Thu, Jul 05, 2018 at 09:32:31AM -0400, Ben Gamari wrote:

...

We use a much different memory allocation strategy on FreeBSD which (AFAIK) does not allow one to reserve address space without also committing.

Here's a demo of mapping in a 1TB heap on FreeBSD 11.1, on a machine with "just" 64GB of RAM and 160GB swap. I would expect anon memory mappings to behave similarly on older systems.

$ cat foo.c #include #include #include #include

int main(int argc, char **argv) { size_t heaplen = 1ULL << 40; /* 1TB */ unsigned char *heap;

heap = mmap(NULL, heaplen, PROT_READ|PROT_WRITE, MAP_ANON, -1, 0); heap[0] = 'A'; heap[heaplen-1] = 'z'; printf ("%p(%zd) %c %c\n", heap, heaplen, heap[0], heap[heaplen-1]); sleep(2); return 0; }

I also get essentially the same results with posix_memalign(3): #include #include #include #include #include int main(int argc, char **argv) { size_t heaplen = 1ULL << 40; void *p; unsigned char *heap; if (posix_memalign(&p, getpagesize(), heaplen) != 0) err(1, "posix_memalign"); heap = p; heap[0] = 'A'; heap[heaplen-1] = 'z'; printf ("%p(%zd) %c %c\n", heap, heaplen, heap[0], heap[heaplen-1]); sleep(2); return 0; } $ ./foo & sleep 1; ps -o vsz= -o rss= $! ; wait [1] 76003 0x801000000(1099511627776) A z 1073750140 2080 [1]+ Done ./foo -- Viktor.

On Thu, Jul 05, 2018 at 10:20:30PM -0400, Ben Gamari wrote:

Hmm, it's possible this could be fixed fairly easily in that case. The original reason for disabling the two-step allocator is #12695, where GHC failed to build due to MAP_NORESERVE being undefined. I had assumed that this meant that reservation-only mappings weren't defined however now that I look again at the mmap(2) manpage it looks like I am probably wrong. MAP_NORESERVE merely flags to the system that swap space should not be reserved for the mapping.

Right, absence of MAP_NORESERVE does not mean that all the pages will be reserved in advance. However, the VM behaviour may be configuration-dependent: https://wiki.freebsd.org/SystemTuning#SYSCTL_TUNING The vm.overcommit sysctl defines the overcommit behaviour of the vm subsystem. The virtual memory system always does accounting of the swap space reservation, both total for system and per-user. Corresponding values are available through sysctl vm.swap_total, that gives the total bytes available for swapping, and vm.swap_reserved, that gives number of bytes that may be needed to back all currently allocated anonymous memory. Setting bit 0 of the vm.overcommit sysctl causes the virtual memory system to return failure to the process when allocation of memory causes vm.swap_reserved to exceed vm.swap_total. Bit 1 of the sysctl enforces RLIMIT_SWAP limit (see getrlimit(2)). Root is exempt from this limit. Bit 2 allows to count most of the physical memory as allocatable, except wired and free reserved pages (accounted by vm.stats.vm.v_free_target and vm.stats.vm.v_wire_count sysctls, respectively). I have the default vm.overcommmit setting: $ sysctl vm.overcommit vm.overcommit: 0 in which "bit 0" is off, so processes can allocate more anonymous memory than available swap. With vm.overcommit=1 the 1TB mmap() fails (surprisingly EINVAL rather than ENOMEM), while a 1MB mmap() succeeds.

My only question is how one performs reservation-and-commit mappings on FreeBSD. This is important since without committing any access to the mapping may crash the program in the case of OOM. Does FreeBSD simply not provide a way to safely map known-good memory?

With vm.overcommit=0, I don't off-hand know of a way to map anonymous memory that's guaranteed to not later segfault on first access. One way to avoid issues when vm.overcommit=1 is to use the MAP_GUARD flag to reserve a mapping without allocating any pages. And then (as needed) explicitly map pages within the reserved range. $ cat foo.c #include #include #include #include #include int main(int argc, char **argv) { size_t heapmax = 1ULL << 40; /* 1TB */ size_t heaplen = 1ULL << 20; /* 1MB */ unsigned char *heap; unsigned char *heap2; /* Reserve 1 TB of address-space */ heap = mmap(NULL, heapmax, PROT_NONE, MAP_GUARD, -1, 0); if (heap == MAP_FAILED) err(1, "mmap"); printf ("%p(%zd)\n", heap, heapmax); /* Allocate last 1MB of the 1TB address-space */ heap2 = &heap[heapmax - heaplen]; heap2 = mmap(heap2, heaplen, PROT_READ|PROT_WRITE, MAP_ANON|MAP_PRIVATE|MAP_FIXED, -1, 0); if (heap2 == MAP_FAILED) err(1, "mmap"); /* Use the allocated space */ heap2[0] = 'A'; heap2[heaplen-1] = 'z'; printf ("%p(%zd) %c %c\n", heap, heaplen, heap2[0], heap2[heaplen-1]); sleep(20); return 0; } $ ./foo & sleep 1; pid=$! ; cat /proc/$pid/map; ps -o vsz= -o rss= $pid ; wait [1] 84867 0x800e00000(1099511627776) 0x800e00000(1048576) A z 0x400000 0x401000 1 0 0xfffff80f04f75780 r-x 2 0 0x9210 COW NC vnode /tmp/foo NCH -1 0x600000 0x601000 1 0 0xfffff80b872ed1e0 rw- 1 0 0x3000 NCOW NNC default - CH 1001 0x800600000 0x800621000 33 0 0xfffff80011df3780 r-x 173 76 0x1000 COW NC vnode /libexec/ld-elf.so.1 NCH -1 0x800621000 0x800642000 22 0 0xfffff80b872ed690 rw- 1 0 0x3000 NCOW NNC default - CH 1001 0x800820000 0x800821000 1 0 0xfffff80d348e3c30 rw- 1 0 0x3000 COW NNC vnode /libexec/ld-elf.so.1 CH 1001 0x800821000 0x800822000 1 0 0xfffff80f432df690 rw- 1 0 0x3000 NCOW NNC default - CH 1001 0x800822000 0x8009b4000 402 0 0xfffff8000d720e10 r-x 173 76 0x1000 COW NC vnode /lib/libc.so.7 NCH -1 0x8009b4000 0x800bb4000 0 0 0xfffff8035779ea50 --- 1 0 0x2000 NCOW NNC default - NCH -1 0x800bb4000 0x800bc0000 12 0 0xfffff8084feace10 rw- 1 0 0x3000 COW NNC vnode /lib/libc.so.7 CH 1001 0x800bc0000 0x800bda000 5 0 0xfffff80682dcfc30 rw- 2 0 0x3000 NCOW NNC default - CH 1001 0x800c00000 0x800e00000 9 0 0xfffff80682dcfc30 rw- 2 0 0x3000 NCOW NNC default - CH 1001 0x800e00000 0x10800d00000 0 0 0 --- 0 0 0x0 COW NC none - NCH -1 0x10800d00000 0x10800e00000 2 0 0xfffff8018644c1e0 rw- 1 0 0x3000 NCOW NNC default - CH 1001 0x10800e00000 0x10801000000 6 0 0xfffff80b76b20d20 rw- 1 0 0x3000 NCOW NNC default - CH 1001 0x7fffdffff000 0x7ffffffdf000 0 0 0 --- 0 0 0x0 NCOW NNC none - NCH -1 0x7ffffffdf000 0x7ffffffff000 3 0 0xfffff80c6b72c1e0 rw- 1 0 0x3000 NCOW NNC default - CH 1001 0x7ffffffff000 0x800000000000 0 0 0xfffff800117b6000 r-x 100 0 0x6 NCOW NNC default - NCH -1 9340 2072 [1]+ Done ./foo Note the 1TB-1MB hole in the process memory map: 0x800e00000 0x10800d00000 0 0 0 --- 0 0 0x0 COW NC none - NCH -1 followed by a 1MB mapping: 0x10800d00000 0x10800e00000 2 0 0xfffff8018644c1e0 rw- 1 0 0x3000 NCOW NNC default - CH 1001 Note also that the "vsz" does not include the hole. So we are guaranteed exclusive use of a virtual address range, but it is not initially allocated, and later allocations can proceed incrementally. Those later allocation may or may not be subject to overcommit with OOM segfaults depending on vm.overcommit. -- Viktor.