July 2016 - ghc-devs - Haskell.org

RE: [commit: ghc] master: Use deterministic maps for FamInstEnv (9858552)
by Simon Peyton Jones 07 Jul '16

07 Jul '16

| | -type FamInstEnv = UniqFM FamilyInstEnv -- Maps a family to its | instances | +type FamInstEnv = UniqDFM FamilyInstEnv -- Maps a family to its | +instances | -- See Note [FamInstEnv] Bartosz, could you add your comment from the commit (or variant thereof) as a comment to this type definition, so we know WHY you are using UniqDFM here? Thanks Simon We turn FamInstEnvs into lists in some places which don't directly affect the ABI. That happens in family consistency checks and when producing output for `:info`. Unfortunately that nondeterminism is nonlocal and it's hard to tell locally what it affects. Furthermore the envs should be relatively small, so it should be free to use deterministic maps here. Testing with nofib and ./validate detected no difference between UniqFM and UniqDFM. GHC Trac: #4012

1 0

Windows is still broken
by Simon Peyton Jones 05 Jul '16

05 Jul '16

Simon, Windows is still broken. Do you want me to revert both patches, or will you? Simon "inplace/bin/ghc-stage1.exe" -optc-fno-stack-protector -optc-Wall -optc-Werror -optc-Wall -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc-Waggregate-return -optc-Wpointer-arith -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls -optc-Iincludes -optc-Iincludes/dist -optc-Iincludes/dist-derivedconstants/header -optc-Iincludes/dist-ghcconstants/header -optc-Irts -optc-Irts/dist/build -optc-DCOMPILING_RTS -optc-fno-strict-aliasing -optc-fno-common -optc-Irts/dist/build/./autogen -optc-Wno-error=inline -optc-O2 -optc-fomit-frame-pointer -optc-g -optc-fno-omit-frame-pointer -optc-g -optc-O0 -optc-DRtsWay=\"rts_thr_debug\" -optc-DWINVER=0x06000100 -static -optc-DTHREADED_RTS -optc-DDEBUG -O0 -H64m -Wall -fllvm-fill-undef-with-garbage -Werror -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Irts -Irts/dist/build -DCOMPILING_RTS -this-unit-id rts -dcmm-lint -i -irts -irts/dist/build -Irts/dist/build -irts/dist/build/./autogen -Irts/dist/build/./autogen -O2 -O0 -Wnoncanonical-monad-instances -c rts/Linker.c -o rts/dist/build/Linker.thr_debug_o In file included from rts\Linker.c:13:0: error: rts\Linker.c: In function 'ocTryLoad': rts\Linker.c:2566:55: error: error: pointer type mismatch in conditional expression [-Werror] oc->archiveMemberName : oc->fileName)); ^ includes\Rts.h:300:53: error: note: in definition of macro 'IF_DEBUG' #define IF_DEBUG(c,s) if (RtsFlags.DebugFlags.c) { s; } ^ rts\Linker.c:2564:33: error: error: format '%ls' expects argument of type 'wchar_t *', but argument 2 has type 'void *' [-Werror=format=] IF_DEBUG(linker, debugBelch("Resolving %" PATH_FMT "\n", ^ includes\Rts.h:300:53: error: note: in definition of macro 'IF_DEBUG' #define IF_DEBUG(c,s) if (RtsFlags.DebugFlags.c) { s; } ^ cc1.exe: all warnings being treated as errors `gcc.exe' failed in phase `C Compiler'. (Exit code: 1) rts/ghc.mk:255: recipe for target 'rts/dist/build/Linker.debug_o' failed make[1]: *** [rts/dist/build/Linker.debug_o] Error 1 make[1]: *** Waiting for unfinished jobs.... In file included from rts\Linker.c:13:0: error: rts\Linker.c: In function 'ocTryLoad': rts\Linker.c:2566:55: error: error: pointer type mismatch in conditional expression [-Werror] oc->archiveMemberName : oc->fileName)); ^ includes\Rts.h:300:53: error: note: in definition of macro 'IF_DEBUG' #define IF_DEBUG(c,s) if (RtsFlags.DebugFlags.c) { s; } ^ rts\Linker.c:2564:33: error: error: format '%ls' expects argument of type 'wchar_t *', but argument 2 has type 'void *' [-Werror=format=] IF_DEBUG(linker, debugBelch("Resolving %" PATH_FMT "\n", ^ includes\Rts.h:300:53: error: note: in definition of macro 'IF_DEBUG' #define IF_DEBUG(c,s) if (RtsFlags.DebugFlags.c) { s; } ^ cc1.exe: all warnings being treated as errors `gcc.exe' failed in phase `C Compiler'. (Exit code: 1) rts/ghc.mk:255: recipe for target 'rts/dist/build/Linker.thr_debug_o' failed make[1]: *** [rts/dist/build/Linker.thr_debug_o] Error 1 Makefile:129: recipe for target 'all' failed make: *** [all] Error 2 /cygdrive/c/code/HEAD$ git log -2 rts/Linker.c commit 01f449f4ffd2c4f23bfe5698b9f1b98a86276900 Author: Simon Marlow <marlowsd(a)gmail.com> Date: Mon Jul 4 10:56:04 2016 +0100 Fix 32-bit build failures commit 6377757918c1e7f63638d6f258cad8d5f02bb6a7 Author: Simon Marlow <marlowsd(a)gmail.com> Date: Wed Jun 29 21:50:18 2016 +0100 Linker: some extra debugging / logging

2 1

Heads up LLVM 3.8
by Erik de Castro Lopo 05 Jul '16

05 Jul '16

HI all, This is just a heads up that git HEAD will soon be switch to LLVM 3.8. The Phab patch is here: https://phabricator.haskell.org/D2382 I suspect the ghc-8.0 branch will continue to use LLVM 3.7 for all the 8.0 releases. I also suspect that LLVM 3.9 will be released (and that we will switch to it) before the first RC release of 8.2. Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/

1 0

Linker.c broken
by Simon Peyton Jones 04 Jul '16

04 Jul '16

Aargh! Windows is broken /again/. Some mess-up in Linker.c. I have not yet tried reverting recent patches. Might someone fix please? It’s really helpful to validate on Windows when making RTS changes. Simon rts\Linker.c: In function 'ghci_find': rts\Linker.c:1482:52: error: error: pointer type mismatch in conditional expression [-Werror] oc->archiveMemberName : oc->fileName); ^ rts\Linker.c:1480:28: error: error: format '%ls' expects argument of type 'wchar_t *', but argument 3 has type 'void *' [-Werror=format=] debugBelch("%p is in %" PATH_FMT, addr, ^ "inplace/bin/ghc-stage1.exe" -optc-fno-stack-protector -optc-Wall -optc-Werror -optc-Wall -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc-Waggregate-return -optc-Wpointer-arith -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls -optc-Iincludes -optc-Iincludes/dist -optc-Iincludes/dist-derivedconstants/header -optc-Iincludes/dist-ghcconstants/header -optc-Irts -optc-Irts/dist/build -optc-DCOMPILING_RTS -optc-fno-strict-aliasing -optc-fno-common -optc-Irts/dist/build/./autogen -optc-Wno-error=inline -optc-O2 -optc-fomit-frame-pointer -optc-g -optc-fno-omit-frame-pointer -optc-g -optc-O0 -optc-DRtsWay=\"rts_debug\" -optc-DWINVER=0x06000100 -static -optc-DDEBUG -ticky -DTICKY_TICKY -O0 -H64m -Wall -fllvm-fill-undef-with-garbage -Werror -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Irts -Irts/dist/build -DCOMPILING_RTS -this-unit-id rts -dcmm-lint -i -irts -irts/dist/build -Irts/dist/build -irts/dist/build/./autogen -Irts/dist/build/./autogen -O2 -O0 -Wnoncanonical-monad-instances -c rts/RaiseAsync.c -o rts/dist/build/RaiseAsync.debug_o rts\Linker.c:1483:28: error: error: format '%lx' expects argument of type 'long unsigned int', but argument 3 has type 'long long unsigned int' [-Werror=format=] debugBelch(", section %d, offset %lx\n", i, ^ In file included from rts\Linker.c:13:0: error: rts\Linker.c: In function 'ocTryLoad': rts\Linker.c:2563:55: error: error: pointer type mismatch in conditional expression [-Werror] oc->archiveMemberName : oc->fileName)); ^ includes\Rts.h:300:53: error: note: in definition of macro 'IF_DEBUG' #define IF_DEBUG(c,s) if (RtsFlags.DebugFlags.c) { s; } ^ rts\Linker.c:2561:33: error: error: format '%ls' expects argument of type 'wchar_t *', but argument 2 has type 'void *' [-Werror=format=] IF_DEBUG(linker, debugBelch("Resolving %" PATH_FMT "\n", ^ includes\Rts.h:300:53: error: note: in definition of macro 'IF_DEBUG' #define IF_DEBUG(c,s) if (RtsFlags.DebugFlags.c) { s; } ^ cc1.exe: all warnings being treated as errors `gcc.exe' failed in phase `C Compiler'. (Exit code: 1) rts/ghc.mk:255: recipe for target 'rts/dist/build/Linker.debug_o' failed make[1]: *** [rts/dist/build/Linker.debug_o] Error 1 make[1]: *** Waiting for unfinished jobs.... Makefile:129: recipe for target 'all' failed make: *** [all] Error 2 /cygdrive/c/code/HEAD$

8 20

Interruptible exception wormholes kill modularity
by Edward Z. Yang 04 Jul '16

04 Jul '16

In 2010, in the thread "Asynchronous exception wormholes kill modularity" [1], Bas van Dijk observed that 'unblock :: IO a -> IO a' broke modularity, as the sequence of calls 'block . block . unblock $ io' would result in 'io' being run with asynchronous exceptions unblocked, despite the outer 'block' "expecting" that asynchronous exceptions cannot be thrown. I would like to make two claims: 1. The new mask/restore interface is insufficient to "solve" this modularity problem, as *interruptible* operations can still be used to catch asynchronous exceptions. 2. Thus, we should provide an unblock combinator which can be used to catch asynchronous exceptions from a 'mask' (though not an 'uninterruptibleMask')--though it is doubtful if anyone should ever use 'mask' in the first place. Claim 1: Here is some code which reimplements 'unblock': import Control.Exception import Control.Concurrent import Control.Concurrent.MVar unblock :: IO a -> IO a unblock io = do m <- newEmptyMVar _ <- forkIO (io >>= putMVar m) takeMVar m The main idea is that 'takeMVar' is an interruptible operation: when it blocks, the thread can now receive asynchronous exceptions. In general, a thread can unmask exceptions by blocking. Here is a simple test-case: main = do let x = 10000000 -- Just do a bit of work tid <- myThreadId forkIO $ (threadDelay 10000 >> killThread tid) r <- mask $ \restore -> do -- restore $ do -- unblock $ do -- do something non-blocking evaluate (f x []) -- If the exception is delivered in a timely manner, -- shouldn't get here. print r f 0 r = r f n r = f (n-1) (n:r) With both restore and unblock commented, the ThreadKilled exception is delayed; uncommenting either restore or unblock causes the exception to be delivered. This admonition does not apply to uninterruptibleMask, for which there are no interruptible exceptions. Claim 2: Thus, I come to the conclusion that we were wrong to remove 'unblock', and that it is no worse than the ability for interruptible actions to catch asynchronous exceptions. You could very well argue that interruptible actions are a design flaw. Then you should use 'uninterruptibleMask' instead, which effectively removes the concept of interruptibility--and is thus modular. Indeed, Eyal Lotem proposed [2] that 'bracket' should instead use 'uninterruptibleMask', for precisely the reason that it is too easy to reenable asynchronous exceptions in 'mask'. But assuming that interruptible masks are a good idea (Simon Marlow has defended them as "a way avoid reasoning about asynchronous exceptions except at specific points, i.e., where you might block"), there should be an 'unblock' for this type of mask. It should be said that the absence of 'unblock' for 'uninterruptibleMask' only implies that a passed in IO action (e.g., the cleanup action in bracket) does not have access to the exceptions thrown to the current thread; it doesn't actually guarantee uninterruptibility, since the passed in IO action could always raise a normal exception. Haskell's type system is not up to the task of enforcing such invariants. Cheers, Edward [1] https://mail.haskell.org/pipermail/libraries/2010-March/013310.html https://mail.haskell.org/pipermail/libraries/2010-April/013420.html [2] https://mail.haskell.org/pipermail/libraries/2014-September/023675.html P.S. You were CC'ed to this mail because you participated in the original "Asynchronous exception wormholes kill modularity" discussion. P.P.S. I have some speculations about using uninterruptibleMask more frequently: it seems to me that there ought to be a variant of uninterruptibleMask that immediately raises an exception if the "uninterruptible" action blocks. This would probably of great assistance of noticing and eliminating blocking in uninterruptible code.

3 6

Template Haskell determinism
by Bartosz Nitka 01 Jul '16

01 Jul '16

Template Haskell with its ability to do arbitrary IO is non-deterministic by design. You could for example embed the current date in a file. There is however one kind of non-deterministic behavior that you can trigger accidentally. It has to do with how Names are reified. If you take a look at the definition of reifyName you can see that it puts the assigned Unique in a NameU: reifyName :: NamedThing n => n -> TH.Name reifyName thing | isExternalName name = mk_varg pkg_str mod_str occ_str | otherwise = TH.mkNameU occ_str (getKey (getUnique name)) ... NameFlavour which NameU is a constructor of has a default Ord instance, meaning that it ends up comparing the Uniques. The relative ordering of Uniques is not guaranteed to be stable across recompilations [1], so this can lead to ABI-incompatible binaries. This isn't an abstract problem and it actually happens in practice. The microlens package keeps Names in a Set and later turns that set into a list. The results have different orders of TyVars resulting in different ABI hashes and can potentially be optimized differently. I believe it's worth to handle this case in a deterministic way and I have a solution in mind. The idea is to extend NameU (and potentially NameL) with an ordering key. To be more concrete: - | NameU !Int + | NameU !Int !Int This way the Ord instance can use a stable key and the problem reduces to ensuring the keys are stable. To generate stable keys we can use the fact that reify traverses the expressions in the same order every time and sequentially allocate new keys based on traversal order. The way I have it implemented now is to add a new field in TcGblEnv which maps Uniques to allocated keys: + tcg_th_names :: TcRef (UniqFM Int, Int), Then the reifyName and qNewName do the necessary bookkeeping and translate the Uniques on the fly. This is a breaking change and it doesn't fix the problem that NameFlavour is not abstract and leaks the Uniques. It would break at least: - singletons - th-lift - haskell-src-meta - shakespeare - distributed-closure I'd like to get feedback if this is an acceptable solution and if the problem is worth solving. Cheers, Bartosz [1] https://ghc.haskell.org/trac/ghc/wiki/DeterministicBuilds#NondeterministicU…

5 11