- ghc-devs - Haskell.org

Issues with the large address space allocator and HPC systems with resource limits
by Luis Pedro Coelho 05 Jul '18

05 Jul '18

Dear GHC devs, I hope this is the right forum to bring this up. I am the lead developer of NGLess (https://github.com/ngless-toolkit/ngless a bioinformatics tool, written in Haskell). Several users have complained about not being able to easily use NGLess in an academic cluster environment due to the fact that it allocates 1TB of address space (e.g., https://groups.google.com/forum/#!topic/ngless/9su2E0EdeCc and I have also gotten several private emails on this issue). In particular, many systems are set up with a limit on the address space so that if the job allocates more than the given limit, it is immediately killed. This appears to be the default way to set up SGE, the most widely used batch system. Users are dependent on their sysadmins and lack the permissions to change these settings easily (and may not always be cognizant of the difference between "allocating address space" and "allocating memory"). Using ulimit seem to make the issue disappear on most, but not all, user setups. I have now built NGLess with a version of GHC that was compiled without the large address allocator (using ./configure --disable-large-address-space). At least locally, this seems to run correctly and solve the issue. I assume that there are performance or other reasons to use the large address space allocator as the default, but, right now, for the problem space I am working in, disabling it seems to be a better trade-off. In principle, the RTS that is used for GHC and the one that is used for the programme being linked do not need to be the same. Is there any possibility of making this choice when a programme is linked and not when GHC is compiled? Thank you for all your effort! Luis -- Luis Pedro Coelho | Fudan University | http://luispedro.org PI of Big Data Biology Lab at Fudan University (start mid-2018) http://big-data-biology.org

3 3

Not possible to build debug + prof, oversight or expected?
by Ömer Sinan Ağacan 04 Jul '18

04 Jul '18

I just realized GHC 8.4.2 doesn't ship a debug + prof RTS, and HEAD doesn't build it with release build (no build.mk) or with prof flavor. It seems to me that this is a bug, because some debug files in RTS (e.g. Printer.c) actually check PROFILING macro in a few places and print different stuff depending on that. Anyone know if this is intentional? It's really inconvenient that I can't use debug runtime to debug profiling build issues.. (I also realized that we dont' generate .sos for some ways, which means -dynamic is also don't combine with e.g. -prof) Thanks Ömer

2 1

Loading GHC into GHCi (and ghcid)
by Matthew Pickering 04 Jul '18

04 Jul '18

Hi all, Csongor has informed me that he has worked out how to load GHC into GHCi which can then be used with ghcid for a more interactive development experience. 1. Put this .ghci file in compiler/ https://gist.github.com/mpickering/73749e7783f40cc762fec171b879704c 2. Run "../inplace/bin/ghc-stage2 --interactive -odir tmp -hidir tmp" from inside compiler/ It may take a while and require a little bit of memory but in the end all 500 or so modules will be loaded. It can also be used with ghcid. ghcid -c "../inplace/bin/ghc-stage2 --interactive -odir tmp -hidir tmp" Hopefully someone who has more RAM than I. Can anyone suggest the suitable place on the wiki for this information? Cheers, Matt

9 17

req_interp tests
by Ömer Sinan Ağacan 02 Jul '18

02 Jul '18

Hi, I have a few problems with req_interp tests. First, req_interp doesn't actually skip the test, it runs it but expects it to fail. This causes problems when testing stage 1 compiler because 629 req_interp tests are run for no reason. Ideally I think req_interp would skip the test, and for the error messages ("not build for interactive use" etc.) we'd have a few stage 1 tests (maybe we alrady have this). This would make the testsuite much faster for testing stage 1. Second, combination of req_interp and compile_fail currently doesn't work, because req_interp makes a failing test pass, but compile_fail expects the test to fail. See T3953 as an example. Making req_interp skip the test fixes this problem as well. So I'd like to make req_interp skip the test instead of expecting it to fail. Any objections to this? Ömer

2 1

Booting ghc with system-wide installed Cabal?
by Ömer Sinan Ağacan 02 Jul '18

02 Jul '18

Currently we have to build Cabal from scratch after every make clean. Ideally I should be able to skip this step by installing the correct versions of Cabal and cabal-install system-wide, but as far as I can see we currently doesn't support this. Any ideas on how to make this work? Thanks, Ömer

3 4

Phabricator post-mortem
by Ben Gamari 02 Jul '18

02 Jul '18

Hi everyone, As you may have noticed, Phabricator was down briefly on Thursday and for much of yesterday. The reason for both outages was a full root volume. My response to Thursday outage was to free enough space to buy time for a transition away from Rackspace (which would need to happen in the next few months anyways). Unfortunately, the database grew faster than I could put the Rackspace transition into action so we again ran out of space on Sunday when I was away on a hike. This morning I provisioned a new volume and moved the database to it. This should hold us over until we have an opportunity to move away from Rackspace. Cheers, - Ben

1 0

Phab is unhealthy
by Simon Peyton Jones 02 Jul '18

02 Jul '18

I'm getting the stuff below from Phabricator, e.g. https://phabricator.haskell.org/D4909 What should I do? Simon Error 503 Backend is unhealthy Backend is unhealthy Guru Mediation: Details: cache-lcy19226-LCY 1530521354 1115923518 ________________________________ Varnish cache server

2 1

Should we have primitive fill-once variables?
by David Feuer 01 Jul '18

01 Jul '18

IVars (write-once variables) can be useful for various purposes. In Haskell, they can be implemented using MVars. But MVars are not the lightest things in the world. I'm wondering if it might pay to implement something much lighter primitively. Here's a sketch of some things I'll tentatively call QVars. A QVar has two fields: a value (possibly null) and a stack (no need for a queue) of waiting threads. newEmptyQVar: Create a QVar with a null value and an empty queue. tryReadQVar: Just look at the value. readQVar: Check if the value is null (simple memory read). If not, use it. If so, push yourself onto the waiting stack (CAS loop). The code that will run when you're awakened will try to awaken the next thread if there is one (CAS loop). putQVar: Install a new value and get the old one (exchange). If the old value was null, mark the QVar dirty. Awaken the first thread if there is one (CAS loop). Return the old value if it was non-null (this can be used in library code to make duplicate writes, or non-equal duplicate writes, an error). I think we'd probably also want atomic modification operations, but I haven't figured out which ones yet. Implementation differences from MVars: * We have two fields instead of three (because we can get away with a stack instead of a queue). * We never need to lock the QVar closure. MVars do: they can change freely between empty and full, so it's necessary to coordinate between value and queue modifications.

5 11

Re: Plan for GHC 8.6.1
by Ben Gamari 29 Jun '18

29 Jun '18

Moritz Angermann <moritz(a)lichtzwerge.com> writes: > Hi, > > I would almost go as far as saying 8.6 will work with LLVM4-6. GHC llvm codegen > is really only dependent on the textual IR, and even there only on the parts we > use. This used to be an issue, where LLVMs textual IR changed quite a bit, but > it looks like it hasn't for the last few releases. > > This of course does not insulate us from bugs in LLVM itself, which might or > might not affect us. > > Maybe we can be a bit more lenient with respect to LLVM versions? > I would really prefer not to be. We put in place the current policy for a few good reasons: not only was keeping up with the syntactic changes in a compatible way tiresome, but various sets of LLVM bugs meant significantly more work during ticket triage. While the relative syntactic stability of the last few LLVM releases may reduce the relevance of the first reason, the second reason holds just as well today as it did two years ago. Cheers, - Ben

3 3

Harbormaster: Build failure on OS/X build. How to fix it as a non Mac user
by Roland Senn 29 Jun '18

29 Jun '18

Hi all, Phabricator informed me, that the builds on Harbourmaster for my changes failed. Looking at the log at https://phabricator.haskell.org/B21368 I can see, that the builds on Linux and Windows succeeded, however the tests on OS/X failed. Looking at the failed tests at https://phabricator.haskell.org/harborma ster/build/48354/ it tells me that the 2 tests TEST="T5631 T6048" failed. Now, I don't own a machine with OS/X. How can I look, what really failed, and how can I try to fix it? Many thanks Roland

3 2