July 2018 - ghc-tickets

[GHC] #15136: High CPU when asynchronous exception and unblocking retry on TVar raced
by GHC 16 Jul '18

16 Jul '18

#15136: High CPU when asynchronous exception and unblocking retry on TVar raced -------------------------------------+------------------------------------- Reporter: nshimaza | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: 8.6.1 Component: Runtime | Version: 8.4.2 System | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Runtime crash Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Detail: https://github.com/nshimaza/race-tmvar-async-exception Runtime falls into high CPU under racing condition between async exception and unblocking retry on TVar. * Reproduces with +RTS -Nx where x > 1 * Does NOT reproduce with +RTS -N1 * Program stalls at `killThread` * High CPU based on given -Nx * CPU won't be 100% if you gave x smaller than available hardware threads of your platform. * Does NOT reproduce if TVar/retry is replaced by MVar * Reproduced with GHC 8.4.2 (macOS High Sierra (10.13.4)) * Reproduced with GHC 8.4.2 (Docker for Mac Version 18.03.1-ce-mac65) * Reproduced with ghc-8.5.20180506 (Docker for Mac Version 18.03.1-ce- mac65) Minimal reproducing code here. (You can find more verbose code on the above github repo.) {{{#!hs main :: IO () main = do let volume = 1000 forM_ [1..1000] $ \i -> do putStrFlush $ show i ++ " " -- Spawn massive number of threads. threads <- replicateM volume $ do trigger <- newTVarIO False tid <- forkIO $ void $ atomically $ do t <- readTVar trigger if t then pure t else retry pure (trigger, tid) -- Make sure all threads are spawned. threadDelay 30000 -- Let threads start to exit normally. forkIO $ forM_ threads $ \(trigger, _) -> threadDelay 1 *> atomically (writeTVar trigger True) -- Concurrently kill threads in order to create race. -- TMVar operation and asynchronous exception can hit same thread simultaneously. -- Adjust threadDelay if you don't reproduce very well. threadDelay 1000 forM_ threads $ \(_, tid) -> do putCharFlush 'A' killThread tid -- When the issue reproduced, this killThread doesn't return. putCharFlush '\b' }}} This program intentionally creates race condition between asynchronous exception and unblocking operation of `retry` on TVar. From one side, a `writeTVar trigger True` is attempted from external thread while target thread is blocking at `retry` on the same `TVar`. On the other side, an asynchronous exception `ThreadKilled` is thrown by yet another external thread to the same target thread. In other word, it attempts to kill a thread about to unblock. I guess when the above two operation hit the same thread at the same time in parallel in SMP environment, GHC runtime falls into high CPU. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15136> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 17

[GHC] #15401: Weird GHCi bug
by GHC 16 Jul '18

16 Jul '18

#15401: Weird GHCi bug -------------------------------------+------------------------------------- Reporter: vagarenko | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHCi | Version: 8.4.3 Keywords: | Operating System: Unknown/Multiple Architecture: x86_64 | Type of failure: None/Unknown (amd64) | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Take a look at this GHCi session: {{{#!hs $ ghci GHCi, version 8.4.3: http://www.haskell.org/ghc/ :? for help }}} type `do {_}` {{{#!hs Prelude> do {_} <interactive>:1:5: error: * Found hole: _ :: t Where: `t' is a rigid type variable bound by the inferred type of it :: t at <interactive>:1:1-6 * In a stmt of a 'do' block: _ In the expression: do _ In an equation for `it': it = do _ * Relevant bindings include it :: t (bound at <interactive>:1:1) Valid substitutions include undefined :: forall (a :: TYPE r). GHC.Stack.Types.HasCallStack => a (imported from `Prelude' (and originally defined in `GHC.Err')) }}} looks ok type `()` {{{#!hs Prelude> () () }}} ok type `do {_}` again {{{#!hs Prelude> do {_} <interactive>:3:5: error: * Found hole: _ :: t Where: `t' is a rigid type variable bound by the inferred type of it :: t at <interactive>:3:1-6 * In a stmt of a 'do' block: _ In the expression: do _ In an equation for `it': it = do _ * Relevant bindings include it :: t (bound at <interactive>:3:1) Valid substitutions include undefined :: forall (a :: TYPE r). GHC.Stack.Types.HasCallStack => a (imported from `Prelude' (and originally defined in `GHC.Err')) }}} ok, same message type `()` second type {{{#!hs Prelude> () () }}} ok, type `do {_}` third time: {{{#!hs Prelude> do {_} <interactive>:1:1: error: GHC internal error: `Ghci1.it' is not in scope during type checking, but it passed the renamer tcl_env of environment: [] Prelude> }}} what's happening? -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15401> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 1

[GHC] #15246: -fghci-leak-cheak causes many testsuite failures with the quick build flavour
by GHC 16 Jul '18

16 Jul '18

#15246: -fghci-leak-cheak causes many testsuite failures with the quick build flavour -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHCi | Version: 8.5 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- As [https://mail.haskell.org/pipermail/ghc-devs/2018-May/015812.html originally reported] on the ghc-devs mailing list, the `-fghci-leak-check` flag is causing several tests to fail when GHC is built with the quick build flavour. On my machine, here are all the tests that fail: {{{ Unexpected failures: ghci/prog001/prog001.run prog001 [bad stdout] (ghci) ghci/prog002/prog002.run prog002 [bad stdout] (ghci) ghci/prog003/prog003.run prog003 [bad stdout] (ghci) ghci/prog010/ghci.prog010.run ghci.prog010 [bad stdout] (ghci) ghci/prog013/prog013.run prog013 [bad stdout] (ghci) ghci/prog012/prog012.run prog012 [bad stdout] (ghci) ghci/prog009/ghci.prog009.run ghci.prog009 [bad stdout] (ghci) ghci/scripts/ghci025.run ghci025 [bad stdout] (ghci) ghci/scripts/ghci038.run ghci038 [bad stdout] (ghci) ghci/scripts/ghci057.run ghci057 [bad stdout] (ghci) ghci/scripts/T2182ghci.run T2182ghci [bad stdout] (ghci) ghci/scripts/ghci058.run ghci058 [bad stdout] (ghci) ghci/scripts/T6106.run T6106 [bad stdout] (ghci) ghci/scripts/T8353.run T8353 [bad stdout] (ghci) ghci/scripts/T9293.run T9293 [bad stdout] (ghci) ghci/scripts/T10989.run T10989 [bad stdout] (ghci) ghci/should_run/T13825-ghci.run T13825-ghci [bad stdout] (ghci) ghci.debugger/scripts/print007.run print007 [bad stdout] (ghci) ghci.debugger/scripts/break009.run break009 [bad stdout] (ghci) ghci.debugger/scripts/break008.run break008 [bad stdout] (ghci) ghci.debugger/scripts/break026.run break026 [bad stdout] (ghci) perf/space_leaks/T4029.run T4029 [bad stdout] (ghci) }}} [https://gist.githubusercontent.com/RyanGlScott/f920737287049b82947e1c47cdbc… Here] is the full test output. Generally, the test output discrepancies are all of the form of extra lines of: {{{ +-fghci-leak-check: HomeModInfo for D is still alive! +-fghci-leak-check: ModIface is still alive! +-fghci-leak-check: ModDetails is still alive! +-fghci-leak-check: Linkable is still alive! }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15246> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 6

[GHC] #15397: Linking Issue on Ubuntu and Fedora with Provided Bindists (GHC-8.4.2)
by GHC 16 Jul '18

16 Jul '18

#15397: Linking Issue on Ubuntu and Fedora with Provided Bindists (GHC-8.4.2) -------------------------------------+------------------------------------- Reporter: _recursion | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.4.4 Component: Compiler | Version: 8.4.2 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Incorrect result Unknown/Multiple | at runtime Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- It appears that the bindists being given to both ubuntu and fedora users are subtly broken. When attempting to link a binary depending on libffi, the linker picks up the copy of `libffi.so.7` found in the rts folder of the distribution. This means that at runtime, despite the systems in question having a copy of `libffi.so.6`, the binary can't find the correct shared library to link against. This does not happen on Arch Linux or Gentoo, as `/usr/lib` or `/usr/lib64` are included in their linker invocations respectively, allowing the linker to pick up the correct version of the dependency. You can reproduce the issue quickly by cloning [https://github.com/luna /luna-core Luna Core] and executing the following commands. I suggest doing this on an Ubuntu or Fedora system as I know it fails on those two distros. {{{ stack build luna-shell --fast stack exec luna }}} You should see something along the lines of the following. {{{ error while loading shared libraries: libffi.so.7: cannot open shared object file: No such file or directory }}} In essence, the reproduction steps are as follows: 1. Create a project depending on libffi using ghc-8.4.2 2. Build the project 3. Execute it The ''expected'' result is that the binary links against the system copy of `libffi.so.6`, rather than the `libffi.so.7` provided in the ghc distribution. I can't guarantee that Ubuntu and Fedora are the only affected systems, but I know that Arch Linux and Gentoo were both fine with the provided bindists. All tested systems were x64, so I cannot confirm if the issue affects x86 bindists. The actual symptom appears to be a result of the linker never being given the path to the system library directory (e.g. `/usr/lib` or `/usr/lib64`), whereas on Arch and Gentoo, the linkline does contain that directory. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15397> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 0

[GHC] #15373: core-spec.pdf contains parse errors
by GHC 16 Jul '18

16 Jul '18

#15373: core-spec.pdf contains parse errors -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.8.1 Component: Compiler | Version: 8.4.3 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- After commit 2fbe0b5171fd5639845b630faccb9a0c3b564df7, `core-spec.pdf` now contains several gnarly looking parse errors (which I'll attach screenshots of separately). This is because `nth` coercion typing rule was extended to take a role as an argument, but the uses of `nth` in the `S_Push`, `S_CPush`, and `S_CasePush` rules were never updated to reflect this. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15373> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 5

[GHC] #15372: GHCi leaks in DEBUG mode
by GHC 16 Jul '18

16 Jul '18

#15372: GHCi leaks in DEBUG mode -------------------------------------+------------------------------------- Reporter: goldfire | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- The solution to ticket #15111 fixes some space leaks in GHCi. That ticket also installs `-fghci-leak-check` to the testsuite, so that we'll know when these leaks return. This is all good. However, the `DEBUG` compile still leaks in GHCi. This means that many ghci tests fail in the testsuite when using a `DEBUG` compiler. I thus have two requests: 1. (hopefully quick) Teach the testsuite not to use `-fghci-leak-check` on a `DEBUG` compiler. I tried to do this myself but got lost, never having worked on the testsuite harness. 2. Fix the leaks. If someone could post the fix to (1) in the short term (even if you don't commit it), I'd be grateful. This is holding me up. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15372> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 7

[GHC] #15395: Make StaticPtr (more) robust to code changes and recompilation
by GHC 16 Jul '18

16 Jul '18

#15395: Make StaticPtr (more) robust to code changes and recompilation -------------------------------------+------------------------------------- Reporter: richardw | Owner: (none) Type: feature | Status: new request | Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Discussion migrated from [https://www.reddit.com/r/haskell/comments/8z86q6/stability_of_staticptr/ this] thread on Reddit. The [http://hackage.haskell.org/package/base-4.11.1.0/docs/GHC- StaticPtr.html documentation] for GHC.StaticPtr states that "Only processes launched from the same program binary are guaranteed to use the same set of keys." On the other hand, this [https://ghc.haskell.org/trac/ghc/blog/simonpj/StaticPointers blog post] by Simon suggests that the only sensible implementation of StaticPtr would be as (essentially) a package/module/identifier name triple, which sounds right to me. In this case two StaticPtrs should share the same key as long as they are in the same package/module and assigned to the same identifier. My use case is similar to the one described [http://neilmitchell.blogspot.com/2017/09/existential-serialisation.html here], where StaticPtr is used for (de)serialization of an existential. As pointed out in the comments to that post, this approach could break upon any recompilation of the code base. Having static pointers stable only across instances of the same binary also seems like it would be a potential problem for the applications like CloudHaskell that StaticPointers was developed for. Presumably this means means all nodes need to be based on the same architecture and updates need to happen on every node simultaneously. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15395> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 1

[GHC] #15394: GHC doesn't come with dynamic object files/libraries compiled with profiling
by GHC 16 Jul '18

16 Jul '18

#15394: GHC doesn't come with dynamic object files/libraries compiled with profiling -------------------------------------+------------------------------------- Reporter: mrkkrp | Owner: (none) Type: feature | Status: new request | Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- GHC doesn't come with dynamic object files/libraries compiled with profiling thus it's not possible to compile code with `-dynamic` or `-dynamic-too` and `-prof`. This must be a known issue, but I couldn't find a ticket opened specially for it. I also don't know why these files are not included by default in the bindists. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15394> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 1

[GHC] #15391: Maybe ghc-pkg register should unregister packages with "incompatible" signatures
by GHC 15 Jul '18

15 Jul '18

#15391: Maybe ghc-pkg register should unregister packages with "incompatible" signatures -------------------------------------+------------------------------------- Reporter: ezyang | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: ghc-pkg | Version: 8.4.3 Keywords: backpack | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Consider the following situation: 1. I register package 'p-0.1-inplace' instantiated with 'P = base:Data.Map' 2. I register package 'p-0.1-inplace' instantiated with 'P = base:Control.Monad' Even though the IPID of these two packages is the same, we clearly want to keep both of them in the database. This thus motivates the following lines in ghc-pkg {{{ removes = [ RemovePackage p | not multi_instance, p <- packages db_to_operate_on, mungedId p == mungedId pkg, -- Only remove things that were instantiated the same way! instantiatedWith p == instantiatedWith pkg ] }}} OK... now consider the following situation 1. I register package 'p-0.1-inplace' instantiated with 'P = base:Data.Map' 2. I register package 'p-0.1-inplace', instantiated with nothing (I have edited 'p' such that it has no more signatures) So... ghc-pkg is going to keep both of these packages (per the logic above). But is that really what we want? Now the package database is in a half-consistent state, where the instances of 'p-0.1-inplace' are not consistent with the number of holes they are supposed to have. So let us suggest an invariant: INVARIANT: All packages which have the same IPID in a package database are self-consistent with each other. OK, so given this invariant, what do we have to do in step 2? It would seem that when we get a prospective package to register, we must *find all packages which are incompatible with it*, and remove them from the package database. So, given a package with a set of holes H, we must remove all packages with holes H' such that H = H' I'm not completely convinced this is a good idea. Usually ghc-pkg only removes a single package in response to a new registration, and this behavior could result in lots of packages getting unregistered, more than you might expect. On the other hand, if you think of the package database in a more hierarchical manner, it makes sense that changing the root registration would invalidate all the children. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15391> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 0

[GHC] #13812: deriveConstants: no objdump program given (OpenBSD)
by GHC 15 Jul '18

15 Jul '18

#13812: deriveConstants: no objdump program given (OpenBSD) -------------------------------------+------------------------------------- Reporter: roland | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Build System | Version: Keywords: | Operating System: OpenBSD Architecture: | Type of failure: Building GHC Unknown/Multiple | failed Test Case: | Blocked By: Blocking: | Related Tickets: #9549 Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Building HEAD on OpenBSD 6.1 fails with: {{{ inplace/bin/deriveConstants --gen-header -o includes/dist- derivedconstants/header/DerivedConstants.h --tmpdir includes/dist- derivedconstants/header/ --gcc-program "gcc" --gcc-flag -W all --gcc-flag -std=gnu99 --gcc-flag -fno-stack-protector --gcc-flag -Iincludes --gcc-flag -Iincludes/dist --gcc-flag -Iincludes/dist- derivedconstants/header --gcc-flag -Iincludes/di st-ghcconstants/header --gcc-flag -Irts --gcc-flag -fcommon --nm-program "nm" --target-os "openbsd" deriveConstants: no objdump program given }}} This can be fixed in configure.ac (see attached patch). -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13812> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 4