1TB vsize for all haskell processes?

Hey, haskell-cafe, I realize that this should not be a problem, but it is very strange to me that *every* haskell process on my system has a 1TB VSIZE. If I sort my `ps auwx` output on vsize, the top of the list looks like: mdorman 1208 0.0 0.1 1074011536 12500 ? S Oct14 0:04 /home/mdorman/.xmonad/xmonad-x86_64-linux mdorman 21900 0.0 0.2 1074111408 19712 pts/1 Sl+ 09:06 0:00 cabal run melpa2nix -- --output melpa-generated.nix --melpa /home/mdorman/src/melpa --work /home/mdorman/src/emacs2nix/.workdir mdorman 1363 0.0 0.1 1074383784 13052 ? Sl Oct14 1:14 /home/mdorman/.cache/taffybar/taffybar-linux-x86_64 +RTS -I0 -V0 --RTS --dyre-master-binary=/nix/store/6skvglp84w5xzqx8dxxydazk1zj8h2ih-taffybar-0.4.6/bin/taffybar mdorman 21981 8.6 0.5 1076223012 42308 pts/1 Sl+ 09:06 0:19 /home/mdorman/src/emacs2nix/dist/build/melpa2nix/melpa2nix --output melpa-generated.nix --melpa /home/mdorman/src/melpa --work /home/mdorman/src/emacs2nix/.workdir Each of those haskell processes has a vsize 5 orders of magnitude more than their resident set size, whether it's the two I always have running (xmonad and taffybar), or in this case, cabal itself, as well as the compiled executable it's running. I've spent a reasonable amount of time attempting to google this, to no avail. Is this perhaps a peculiarity of how GHC was built? I'm running on nixos-unstable, using ghc-8.0.1---is there some compilation that we should or shouldn't be setting? Thanks for any guidance, Mike.

This was introduced here: https://ghc.haskell.org/trac/ghc/ticket/9706
Basically, if you can have a huge virtual address space (which you can
on a 64bit architecture) then you can simplify the memory management
logic.
On 17 October 2016 at 14:23, Michael Alan Dorman
Hey, haskell-cafe,
I realize that this should not be a problem, but it is very strange to me that *every* haskell process on my system has a 1TB VSIZE. If I sort my `ps auwx` output on vsize, the top of the list looks like:
mdorman 1208 0.0 0.1 1074011536 12500 ? S Oct14 0:04 /home/mdorman/.xmonad/xmonad-x86_64-linux mdorman 21900 0.0 0.2 1074111408 19712 pts/1 Sl+ 09:06 0:00 cabal run melpa2nix -- --output melpa-generated.nix --melpa /home/mdorman/src/melpa --work /home/mdorman/src/emacs2nix/.workdir mdorman 1363 0.0 0.1 1074383784 13052 ? Sl Oct14 1:14 /home/mdorman/.cache/taffybar/taffybar-linux-x86_64 +RTS -I0 -V0 --RTS --dyre-master-binary=/nix/store/6skvglp84w5xzqx8dxxydazk1zj8h2ih-taffybar-0.4.6/bin/taffybar mdorman 21981 8.6 0.5 1076223012 42308 pts/1 Sl+ 09:06 0:19 /home/mdorman/src/emacs2nix/dist/build/melpa2nix/melpa2nix --output melpa-generated.nix --melpa /home/mdorman/src/melpa --work /home/mdorman/src/emacs2nix/.workdir
Each of those haskell processes has a vsize 5 orders of magnitude more than their resident set size, whether it's the two I always have running (xmonad and taffybar), or in this case, cabal itself, as well as the compiled executable it's running.
I've spent a reasonable amount of time attempting to google this, to no avail. Is this perhaps a peculiarity of how GHC was built? I'm running on nixos-unstable, using ghc-8.0.1---is there some compilation that we should or shouldn't be setting?
Thanks for any guidance,
Mike. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
-- Michael Walker (http://www.barrucadu.co.uk)

I, too, found this change a bit problematic btw: It means I can no longer run Haskell on systems where memory overcommit is disabled. For example, I used to run my shell with an appropriate `ulimit -v` to guarantee that a single program can't force me into swapping; I can no longer do that.

As far as I know, this behaviour should not be affected by overcommit as
the unused pages are all mapped with PROT_NONE and thus do not count
towards the commit limit as they cannot be used without the mapping being
changed. `ulimit -v` (aka RLIMIT_AS) however limits the actual address
space size, and so this does count towards that (as do mmap()ed files and
other such virtual mappings that do not count towards the commit limit).
On Tue, 18 Oct 2016 at 11:39 Niklas Hambüchen
I, too, found this change a bit problematic btw: It means I can no longer run Haskell on systems where memory overcommit is disabled.
For example, I used to run my shell with an appropriate `ulimit -v` to guarantee that a single program can't force me into swapping; I can no longer do that. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Indeed, I have overcommit 48GB max. No problem running haskell programs. On 10/18/2016 12:31 PM, Tristan Seligmann wrote:
As far as I know, this behaviour should not be affected by overcommit as the unused pages are all mapped with PROT_NONE and thus do not count towards the commit limit as they cannot be used without the mapping being changed. `ulimit -v` (aka RLIMIT_AS) however limits the actual address space size, and so this does count towards that (as do mmap()ed files and other such virtual mappings that do not count towards the commit limit).
On Tue, 18 Oct 2016 at 11:39 Niklas Hambüchen
mailto:mail@nh2.me> wrote: I, too, found this change a bit problematic btw: It means I can no longer run Haskell on systems where memory overcommit is disabled.
For example, I used to run my shell with an appropriate `ulimit -v` to guarantee that a single program can't force me into swapping; I can no longer do that. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Hi Tristan and Branimir, this is interesting, I didn't know that ulimit makes this distinction and that the way GHC maps the pages doesn't count towards the accounting limit (the limit as described e.g. on https://www.etalabs.net/overcommit.html in the section starting with "The approach taken in reality"). Is it possible to set the the overcommit settings per shell / per program like `ulimit -v` allows? On 18/10/16 12:31, Tristan Seligmann wrote:
As far as I know, this behaviour should not be affected by overcommit as the unused pages are all mapped with PROT_NONE and thus do not count towards the commit limit as they cannot be used without the mapping being changed. `ulimit -v` (aka RLIMIT_AS) however limits the actual address space size, and so this does count towards that (as do mmap()ed files and other such virtual mappings that do not count towards the commit limit).
participants (5)
-
Branimir Maksimovic
-
Michael Alan Dorman
-
Michael Walker
-
Niklas Hambüchen
-
Tristan Seligmann