[GHC] #15348: Enable two-step allocator on FreeBSD
#15348: Enable two-step allocator on FreeBSD ----------------------------------------+--------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Keywords: | Operating System: FreeBSD Architecture: Unknown/Multiple | Type of failure: None/Unknown Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: ----------------------------------------+--------------------------------- Currently the two-step allocator is disabled on FreeBSD as the `MEM_NORESERVE` macro is undefined. It seems that FreeBSD provided this macro until 2014, when it was [[https://reviews.freebsd.org/D848|removed]] as it wasn't implemented in the kernel. Regardless, Viktor Dukhovni reports empirical evidence on `ghc-devs` that just plain `mmap` does what we want. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => patch * differential: => Phab:D4939 Comment: Could someone test Phab:D4939 on FreeBSD? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by vdukhovni): Specifically, one can reserve a contiguous address range with: {{{ heap = mmap(NULL, heapmax, PROT_NONE, MAP_GUARD, -1, 0); }}} and then incrementally populate segments of that range: {{{ mmap(base, len, PROT_READ|PROT_WRITE, MAP_ANON|MAP_PRIVATE|MAP_FIXED, -1, 0); }}} With `base == heap` for the initial block of pages, and then to the first address not yet mapped for subsequent increments. What we don't get is protection from overcommit OOM, unless the system is configured to disallow overcommit. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): vdukhovni, I've updated Phab:D4939 to account for your comment. Do you suppose you could test? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by vdukhovni): I can try to test, but the patch is not ready yet. FreeBSD does not support combining `MAP_GUARD` with other flags such as `MAP_ANON` or `MAP_PRIVATE`, it must stand alone. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by vdukhovni): From `kern_mmap()` in `/usr/src/sys/vm/vm_mmap.c`: {{{ if ((flags & MAP_GUARD) != 0 && (prot != PROT_NONE || fd != -1 || pos != 0 || (flags & (MAP_SHARED | MAP_PRIVATE | MAP_PREFAULT | MAP_PREFAULT_READ | MAP_ANON | MAP_STACK)) != 0)) return (EINVAL); }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD
-------------------------------------+-------------------------------------
Reporter: bgamari | Owner: (none)
Type: feature request | Status: patch
Priority: normal | Milestone: 8.6.1
Component: Compiler | Version: 8.4.3
Resolution: | Keywords:
Operating System: FreeBSD | Architecture:
| Unknown/Multiple
Type of failure: None/Unknown | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s): Phab:D4939
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Changes (by vdukhovni): * Attachment "patch-rts__posix__OSMem.c" added. OSMem.c patch with MAP_GUARD used alone rather than in combination with MAP_ANON|MAP_PRIVATE -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by vdukhovni): I modified the proposed OSMem.c patch as attached ,rebuilt GHC-8.4.3 and am now running my DANE scanner (runs for ~8 hours, using around 300MB of VM space, and doing a total of around 4TB of allocations). So far so good. And it did allocate a 1TB hole in its address space, partly replaced with actually allocated pages. Because this is a reserved hole, and not a mapping, the reported virtual size is still modest. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by vdukhovni): Small change for HPUX, instead of {{{ #undef MAP_ANON #define MAP_ANON ... }}} safer to {{{ #ifndef MAP_ANON #define MAP_ANON ... }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Changes (by vdukhovni): * Attachment "patch-rts__posix__OSMem.2.c" added. [Updated] OSMem.c patch with MAP_GUARD used alone rather than in combination with MAP_ANON|MAP_PRIVATE -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by vdukhovni): So far, this works just fine. The scanner ran for ~7 hours, with reasonable graphs of rss/vsz over time (attached). For the next run, I'll throw in RTS memory stats. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Changes (by vdukhovni): * Attachment "mem.pdf" added. Memory footprint of allocation-intensive application over 7 hours of run- time. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Changes (by vdukhovni): * Attachment "patch-rts__posix__OSMem.3.c" added. [Further code cleanup] OSMem.c patch with MAP_GUARD used alone rather than in combination with MAP_ANON|MAP_PRIVATE -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by vdukhovni): I found the OSMem.c code a bit of an `#ifdef` maze. The latest patch attached should improve clarity, by better separating the logic from CPP conditionals. One side-effect is that `madvise(MADV_WILLNEED)` will now be called on `COMMIT` not only for Linux, but also for FreeBSD and other non-Darwin `mmap()` API platforms. Will test this shortly... -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by vdukhovni): The latest patch runs fine... I'm done. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Thanks for finishing this up! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): vdukhovni, I'm having a bit of trouble applying the patch you attached. Could you either rebase against `master` and upload a new patch or push a branch somewhere? Thanks! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Pinging vdukhovni. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Changes (by vdukhovni): * Attachment "0001-Enable-two-step-allocator-on-FreeBSD.patch" added. FreeBSD patch for master (git am format) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by vdukhovni): Updated patch attached. (Further questions/comments would be best via direct email, I don't seem to have notifications enabled here) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD
-------------------------------------+-------------------------------------
Reporter: bgamari | Owner: (none)
Type: feature request | Status: patch
Priority: normal | Milestone: 8.6.1
Component: Compiler | Version: 8.4.3
Resolution: | Keywords:
Operating System: FreeBSD | Architecture:
| Unknown/Multiple
Type of failure: None/Unknown | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s): Phab:D4939
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: merge Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: patch => merge -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD
-------------------------------------+-------------------------------------
Reporter: bgamari | Owner: (none)
Type: feature request | Status: merge
Priority: normal | Milestone: 8.6.1
Component: Compiler | Version: 8.4.3
Resolution: | Keywords:
Operating System: FreeBSD | Architecture:
| Unknown/Multiple
Type of failure: None/Unknown | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s): Phab:D4939
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by bgamari):
vdukhovni, the patch has been merged.
That being said, I'm a bit confused; I have tried building it under a
FreeBSD 11 VM but it doesn't appear that `MAP_GUARD` is defined. I checked
in both the `mmap(2)` manpage as well as `
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: merge Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Ahh, the problem is that I'm on 11.0 yet apparently `MAP_GUARD` is only supported in 11.1 and later (judging by the manpages on freebsd.org) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: merge Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Comment (by vdukhovni): Yes, I'm on a FreeBSD 11.1 system: {{{ $ uname -sr FreeBSD 11.1-RELEASE-p10 $ grep -rw MAP_GUARD /usr/include/sys /usr/include/sys/mman.h:#define MAP_GUARD 0x00002000 /* reserve but don't map address range */ }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15348: Enable two-step allocator on FreeBSD -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: feature request | Status: closed Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: fixed | Keywords: Operating System: FreeBSD | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4939 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: merge => closed * resolution: => fixed Comment: Indeed things look better with FreeBSD 11.2. Merged to `ghc-8.6` with 79e136104922aa4dcb555084731a890294cda106. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15348#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC