#14492: Tiered memory allocation restricts available memory -------------------------------------+------------------------------------- Reporter: unode | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 8.0.2 Resolution: | Keywords: memory ulimit Operating System: Linux | Architecture: x86_64 Type of failure: Poor/confusing | (amd64) error message | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by luispedro): The current behaviour is to start at 1TB and divide by 2 until success (or failure if it cannot allocate at least 1MB): https://github.com/ghc/ghc/blob/25f36bd7ba6899be6c720292528c56bc35e0f089/rts... If the memory available is in between two multiples of 2, then this can lead to a lot of unused memory. A (not so implausible, I think our systems might run something similar) worst-case scenario is running on a system with 1T of RAM, but where the sysadmin has wisely chosen to reserve 1GB for a monitoring process. Now, a whole 511GB of RAM are wasted! Btw, that code mentions https://ghc.haskell.org/trac/ghc/ticket/10877 with a previous discussion on this issue. A simple solution is to replace `*len /= 2` by `*len -= *len/8`. The max number of calls to mmap() goes up from 14 to 67, but wastage is now at most 1/8th of the final allocated memory. (Another question is whether a GHC compiled programme could access >1TB of RAM? We now have a few 4TB machines and more on the way...) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14492#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler