November 2016 - ghc-tickets

[GHC] #12808: For primitive (Addr#) operations, Loop Invariant Code Flow not lifted outside the loop...
by GHC 11 Jan '17

11 Jan '17

#12808: For primitive (Addr#) operations, Loop Invariant Code Flow not lifted outside the loop... -------------------------------------+------------------------------------- Reporter: GordonBGood | Owner: Type: bug | Status: new Priority: normal | Milestone: 8.2.1 Component: Compiler | Version: 8.0.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Runtime Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- '''Background:''' I've been intrigued investigating whether GHC can produce code "as fast as Cee (C/C++/Rust/etc.)" by-any-means-possible, and have been using the very tight inner composite culling loops (purely integer operations) of a basic Sieve of Eratosthenes implementation as a test vehicle. '''Synopsis:''' This is a follow-on of the work leading to finding the efficiency problem described in ticket #12798, but involves pushing the speed even further as per the method described for "primesieve as per [http://primesieve.org/] in the "Highly optimized inner loop" section. '''Description of test code:''' Essentially, this method involves extreme loop unrolling with very imperative code although coded functionally; in the case of the following code it means that, recognizing that for all odd primes (which they all are other than two), and that all word sizes are of an even number of bits, there will be a repeating pattern of composite number culls that repeats every word size number of bits. Thus for a word size of one eight-bit byte, we can unroll to eight composite culls in the body of one loop, with loops cases for the primes modulo 8 of 1, 3, 5, and 7, and for the eight bit start positions (b0..b7) meaning there are four times eight is 32 loop cases. When there are no longer a full eight culls left, the culling reverts to conventional single-cull-per-loop as per the test program of ticket #12798. To do this using GHC we need pointer arithmetic, and the only way to implement pointer arithmetic in GHC is to use the Addr# primitive. GHC/Haskell has one other slight overhead over Cee languages in that we need to move the culling array to a pinned array to avoid having it moved while the culling is going on and then move it back when finished but this takes a negligible amount of time (one percent or so) as compared to the culling. As usual for test programs, the culling operations are repeated in a loop for a number of times to give more accurate timing not influenced by execution not related to the culling. All of this is included in the following code (truncated as to loop coses for inclusion here): {{{#!hs -- EfficiencyBug.hs -- showing that there is a register loop invariant bug in generation of assembler code... -- LLVM shows the bug clearer since the code is generally a little faster... {-# LANGUAGE FlexibleContexts, BangPatterns, MagicHash, UnboxedTuples #-} {-# OPTIONS_GHC -O2 -rtsopts -keep-s-files #-} -- or -O2 -fllvm import Data.Word import Data.Bits import Data.Array.ST (runSTUArray) import Data.Array.Base import GHC.ST ( ST(..) ) import GHC.Exts numLOOPS = 10000 :: Int -- Uses a very simple Sieve of Eratosthenes for fixed 2 ^ 18 range (so one L1 cache size) to prove it. twos :: UArray Int Word32 twos = listArray (0, 31) [1 `shiftL` i | i <- [0 .. 31]] soep1 :: () -> [Word32] soep1() = 2 : [fromIntegral i * 2 + 3 | (i, False) <- assocs bufb] where bufb = runSTUArray $ do let bfBts = (256 * 1024) `div` 2 -- to 2^18 + 2 is 128 KBits = 16 KBytes bf <- newArray (0, bfBts - 1) False :: ST s (STUArray s Int Bool) cullb bf cullb bf@(STUArray l u n marr#) = ST $ \s0# -> case getSizeofMutableByteArray# marr# s0# of { (# s1#, n# #) -> let loop t mr# s0# = -- cull a number of times to test timing if t <= 0 then (# s0#, STUArray l u n mr# #) else case getSizeofMutableByteArray# mr# s0# of { (# s1#, n# #) -> case newPinnedByteArray# n# s1# of { (# s2#, marr'# #) -> case copyMutableByteArray# mr# 0# marr'# 0# n# s2# of { s3# -> case unsafeFreezeByteArray# marr'# s3# of { (# s4#, arr# #) -> -- must do this case byteArrayContents# arr# of { adr# -> -- to obtain the addr# here let cullp i@(I# i#) sp# = let !p@(I# p#) = i + i + 3 in let !s@(I# s#) = (p * p - 3) `div` 2 in if s >= n then case copyMutableByteArray# marr'# 0# mr# 0# n# sp# of so# -> (# so#, mr# #) else let !(UArray _ _ _ tarr#) = twos in case readWord64Array# marr# (i# `uncheckedIShiftRL#` 6#) sp# of { (# sp0#, v0# #) -> case (v0# `and#` ((int2Word# 1#) `uncheckedShiftL#` (i# `andI#` 63#))) `eqWord#` (int2Word# 0#) of 0# -> cullp (i + 1) sp0# -- not prime _ -> -- is prime -- most program execution time spent in the following tight loops. -- the following code implments extream loop unrolling... let !pi@(I# pi#) = fromIntegral p in let !sw@(I# sw#) = s `shiftR` 3 in let !sb@(I# sb#) = s .&. 7 in let p1 = sb + pi in let !(I# r1#) = p1 `shiftR` 3 in let p2 = p1 + pi in let !(I# r2#) = p2 `shiftR` 3 in let p3 = p2 + pi in let !(I# r3#) = p3 `shiftR` 3 in let p4 = p3 + pi in let !(I# r4#) = p4 `shiftR` 3 in let p5 = p4 + pi in let !(I# r5#) = p5 `shiftR` 3 in let p6 = p5 + pi in let !(I# r6#) = p6 `shiftR` 3 in let p7 = p6 + pi in let !(I# r7#) = p7 `shiftR` 3 in let !lmt@(I# lmt#) = (fromIntegral n `shiftR` 3) - p7 in let !lmt1# = plusAddr# adr# lmt# in let !strt# = plusAddr# adr# sw# in let !(I# n#) = n in let (# !so#, !sco# #) = case ((((p - 1) `div` 2) .&. 3) `shiftL` 3) + sb of { 0 -> let cull c# sp# = case c# `ltAddr#` lmt1# of 0# -> (# c#, sp# #) _ -> case readWord8OffAddr# c# 0# sp# of { (# sp0#, v0# #) -> case writeWord8OffAddr# c# 0# (v0# `or#` (int2Word# 1#)) sp0# of { sp1# -> case readWord8OffAddr# c# r1# sp1# of { (# sp2#, v1# #) -> case writeWord8OffAddr# c# r1# (v1# `or#` (int2Word# 2#)) sp2# of { sp3# -> case readWord8OffAddr# c# r2# sp3# of { (# sp4#, v2# #) -> case writeWord8OffAddr# c# r2# (v2# `or#` (int2Word# 4#)) sp4# of { sp5# -> case readWord8OffAddr# c# r3# sp5# of { (# sp6#, v3# #) -> case writeWord8OffAddr# c# r3# (v3# `or#` (int2Word# 8#)) sp6# of { sp7# -> case readWord8OffAddr# c# r4# sp7# of { (# sp8#, v4# #) -> case writeWord8OffAddr# c# r4# (v4# `or#` (int2Word# 16#)) sp8# of { sp9# -> case readWord8OffAddr# c# r5# sp9# of { (# sp10#, v5# #) -> case writeWord8OffAddr# c# r5# (v5# `or#` (int2Word# 32#)) sp10# of { sp11# -> case readWord8OffAddr# c# r6# sp11# of { (# sp12#, v6# #) -> case writeWord8OffAddr# c# r6# (v6# `or#` (int2Word# 64#)) sp12# of { sp13# -> case readWord8OffAddr# c# r7# sp13# of { (# sp14#, v7# #) -> case writeWord8OffAddr# c# r7# (v7# `or#` (int2Word# 128#)) sp14# of { sp15# -> cull (plusAddr# c# pi#) sp15# }}}}}}}}}}}}}}}} in cull strt# sp0#; 1 -> let cull c# sp# = case c# `ltAddr#` lmt1# of 0# -> (# c#, sp# #) _ -> case readWord8OffAddr# c# 0# sp# of { (# sp0#, v0# #) -> case writeWord8OffAddr# c# 0# (v0# `or#` (int2Word# 2#)) sp0# of { sp1# -> case readWord8OffAddr# c# r1# sp1# of { (# sp2#, v1# #) -> case writeWord8OffAddr# c# r1# (v1# `or#` (int2Word# 4#)) sp2# of { sp3# -> case readWord8OffAddr# c# r2# sp3# of { (# sp4#, v2# #) -> case writeWord8OffAddr# c# r2# (v2# `or#` (int2Word# 8#)) sp4# of { sp5# -> case readWord8OffAddr# c# r3# sp5# of { (# sp6#, v3# #) -> case writeWord8OffAddr# c# r3# (v3# `or#` (int2Word# 16#)) sp6# of { sp7# -> case readWord8OffAddr# c# r4# sp7# of { (# sp8#, v4# #) -> case writeWord8OffAddr# c# r4# (v4# `or#` (int2Word# 32#)) sp8# of { sp9# -> case readWord8OffAddr# c# r5# sp9# of { (# sp10#, v5# #) -> case writeWord8OffAddr# c# r5# (v5# `or#` (int2Word# 64#)) sp10# of { sp11# -> case readWord8OffAddr# c# r6# sp11# of { (# sp12#, v6# #) -> case writeWord8OffAddr# c# r6# (v6# `or#` (int2Word# 128#)) sp12# of { sp13# -> case readWord8OffAddr# c# r7# sp13# of { (# sp14#, v7# #) -> case writeWord8OffAddr# c# r7# (v7# `or#` (int2Word# 1#)) sp14# of { sp15# -> cull (plusAddr# c# pi#) sp15# }}}}}}}}}}}}}}}} in cull strt# sp0#; -- and so on for 30 more cases... _ -> (# strt#, sp0# #) {- normally never taken case, all cases covered -} } in let !ns# = ((minusAddr# so# adr#) `uncheckedIShiftL#` 3#) +# sb# in -- extreme loop unrolling ends here; remaining primes culled conventionally... let cull j# sc# = -- very tight inner loop case j# <# n# of 0# -> cullp (i + 1) sc# _ -> let i# = j# `andI#` 31# in let !sh# = indexWord32Array# tarr# i# in -- (1 `shiftL` (j .&. 31))) let w# = j# `uncheckedIShiftRL#` 5# in case readWord32Array# marr'# w# sc# of { (# sc0#, ov# #) -> case writeWord32Array# marr'# w# (ov# `or#` sh#) sc0# of { sc1# -> cull (j# +# pi#) sc1# }} in cull ns# sp0# } in case cullp 0 s4# of (# sp#, mrp# #) -> loop (t - 1) mrp# sp# }}}}} in loop numLOOPS marr# s1# } main = print $ length $ soep1() }}} '''The problem:''' The problem is in the innermost loop of the cases, for which case "0" the following assembly code (using -fllvm) is produced: {{{ seGU_info$def: # BB#0: # %cgRL cmpq %r14, 70(%rbx) jbe .LBB35_1 .align 16, 0x90 .LBB35_2: # %cgRJ # =>This Inner Loop Header: Depth=1 movq 14(%rbx), %rcx movq 22(%rbx), %rdx movq 30(%rbx), %rsi movq 38(%rbx), %rdi movq 46(%rbx), %r8 movq 54(%rbx), %r9 movq 62(%rbx), %r10 movq 6(%rbx), %rax addq %r14, %rax orb $1, (%r14) orb $2, (%rcx,%r14) orb $4, (%rdx,%r14) orb $8, (%rsi,%r14) orb $16, (%rdi,%r14) orb $32, (%r8,%r14) orb $64, (%r9,%r14) orb $-128, (%r10,%r14) cmpq 70(%rbx), %rax movq %rax, %r14 jb .LBB35_2 jmp .LBB35_3 .LBB35_1: movq %r14, %rax .LBB35_3: # %cgRK movq (%rbp), %rcx movq %rax, %rbx rex64 jmpq *%rcx # TAILCALL }}} One can readily see that the compiler is not lifting the Loop Invariant Code Flow as in initializing the registers to outside the inner loop, meaning there are many register loads from memory which are not necessary. '''Desired results:''' The desired assembly code is something like the following, which is similar to what is produced by Cee (C/C++/Rust/etc.): {{{ seGU_info$def: # BB#0: # %cgRL movq 14(%rbx), %rcx movq 22(%rbx), %rdx movq 30(%rbx), %rsi movq 38(%rbx), %rdi movq 46(%rbx), %r8 movq 54(%rbx), %r9 movq 62(%rbx), %r10 movq 6(%rbx), %rax movq 70(%rbx), %rbx cmpq %r14, %rbx jbe .LBB35_1 .align 16, 0x90 .LBB35_2: # %cgRJ # =>This Inner Loop Header: Depth=1 orb $1, (%r14) orb $2, (%rcx,%r14) orb $4, (%rdx,%r14) orb $8, (%rsi,%r14) orb $16, (%rdi,%r14) orb $32, (%r8,%r14) orb $64, (%r9,%r14) orb $-128, (%r10,%r14) addq %rax, %r14 cmpq %rbx, %rax jb .LBB35_2 jmp .LBB35_3 .LBB35_1: movq %r14, %rax .LBB35_3: # %cgRK movq (%rbp), %rcx movq %rax, %rbx # rbx clobbered here anyway rex64 jmpq *%rcx # TAILCALL }}} '''Full testing:''' The actual unrolled loop code is too long to post here, but to verify the result is correct (23000) and the performance, the full actual file is attached here. Due to the magic of modern CPU instruction fusion, the code is not as slow as it would indicate by the number of increased instructions, but while it is about twice as fast as when culled conventionally (Intel Skylake), it is about half again as slow as Cee. On an Intel Sky Lake i5-6500 (running at 3.5 GHz for single threading), this takes about one second, about two seconds culled conventionally, but only about 0.6 seconds for Rust/LLVM (with the assembly code output essentially identical to the "desired" code). '''Other back ends and targets:''' Although the code generated by the native NCG has other problems (not moving the loop test to the end of the loop to avoid one jump, and not combining the read and modify and store instructions into the single available instruction), it exhibits the same problem as to not lifting the Loop Invariant Code Flow register initialization. Although this code is x86_64, the problem also applies to x86 code even though the x86 architecture doesn't have enough registers to do this in one loop and needs to be split into two loops culling only four composites per loop, but there still is a significant gain in speed. Although not tested, it probably also applies to other targets such as ARM (which has many general purpose registers). '''Conclusion:''' The use of Addr# primitives is probably not a frequent use case, but as shown here that when one needs their use, they should be efficient. I considered that GHC may intentionally limit the performance of these unsafe primitives to limit their use unless absolutely necessary as in marshalling, something as C## does for the use of unsafe pointers, but surely GHC would not do that as the target programmers are different. '''If this and ticket #12798 were fixed, for this use case the GHC code would be within a percent or two of the performance of Cee.''' -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/12808> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 21

[GHC] #11592: Self-kinded type varialbe accepted
by GHC 10 Jan '17

10 Jan '17

#11592: Self-kinded type varialbe accepted -------------------------------------+------------------------------------- Reporter: simonpj | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Found when chasing #11407 {{{ data T (a::a) = Blah -- Accepted data S (b :: a) a = ... -- Rejected with: variable ‘a’ used as a kind variable before being bound -- as a type variable. Perhaps reorder your variables? }}} But `T` should be rejected too, for the same reason as `S` -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/11592> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 6

[GHC] #12707: Add contributed performance testcase
by GHC 10 Jan '17

10 Jan '17

#12707: Add contributed performance testcase -------------------------------------+------------------------------------- Reporter: bgamari | Owner: bgamari Type: task | Status: new Priority: normal | Milestone: 8.2.1 Component: Compiler | Version: 8.0.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- phadej was kind enough to offer a compile-time performance testcase. Add it to the testsuite or perhaps nofib. https://gist.github.com/phadej/554c2cc7e06b98142dc0fbadb6d3a47f -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/12707> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 3

[GHC] #11040: Split SRT tables on Mac OS
by GHC 10 Jan '17

10 Jan '17

#11040: Split SRT tables on Mac OS ----------------------------------------+--------------------------------- Reporter: olsner | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Keywords: | Operating System: MacOS X Architecture: Unknown/Multiple | Type of failure: None/Unknown Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: ----------------------------------------+--------------------------------- Split off from https://phabricator.haskell.org/D1242#inline-11490 Mac uses subsections via symbols to allow dead code stripping, but doesn't seem to do anything special with the SRT tables (in `HscMain.doCodeGen`), so either I'm missing something or Mac doesn't actually get much useful dead code stripping at all from the subsections feature because the SRT tables would keep everything alive. Perhaps Mac should just always do SRT splitting. Looks like you need to manually add `-optl-Wl,-dead_strip` to test this, since that's not enabled by default. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/11040> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 4

[GHC] #11551: Get doctests into testsuite
by GHC 10 Jan '17

10 Jan '17

#11551: Get doctests into testsuite -------------------------------------+------------------------------------- Reporter: lwm | Owner: Type: task | Status: new Priority: normal | Milestone: Component: Core | Version: 7.10.3 Libraries | Keywords: | Operating System: Unknown/Multiple documentation, doctest | Architecture: | Type of failure: Documentation Unknown/Multiple | bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- It seems like a bit of a shame not to have Mr.Orlitzky's lovely doctests not running with the testsuite. Or perhaps to at least have some sort of `make doctest`. If there is some interest to get this working, I'd like to take a look at submitting a patch. Here are the previous documentation patches I can find: https://phabricator.haskell.org/p/mjo/ I can see the following files that have doctests: {{{ ruby-2.2.1 ~/work/contributing/ghc (master) [12:35:44] lwm$ grep -R "=== __Examples__" libraries/base/ | uniq grep: libraries/base/Data/Maybe.hs:-- ==== __Examples__ libraries/base/Data/Either.hs:-- ==== __Examples__ libraries/base/Data/List.hs:-- ==== __Examples__ libraries/base/Data/Functor.hs:-- ==== __Examples__ libraries/base/Data/Bool.hs:-- ==== __Examples__ libraries/base/Data/Char.hs:-- ==== __Examples__ libraries/base/GHC/Unicode.hs:-- ==== __Examples__ libraries/base/Text/Printf.hs:-- ==== __Examples__ }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/11551> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 7

[GHC] #8761: Make pattern synonyms work with Template Haskell
by GHC 10 Jan '17

10 Jan '17

#8761: Make pattern synonyms work with Template Haskell ------------------------------------+------------------------------------- Reporter: goldfire | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Template Haskell | Version: 7.8.1-rc1 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- Template Haskell has no support for pattern synonyms. This should be added. Specifically: * Create syntax in `Language.Haskell.TH.Syntax` and corresponding functions in `Language.Haskell.TH.Lib` * Support in !DsMeta for desugaring pattern synonym quotations. See `rep_bind`. * Support in Convert for splicing in pattern synonyms. * Support in !TcSplice for reifying pattern synonyms. See `reifyThing`. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8761> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 26

[GHC] #12511: template-haskell's Language.Haskell.Syntax module should be Trustworthy
by GHC 07 Jan '17

07 Jan '17

#12511: template-haskell's Language.Haskell.Syntax module should be Trustworthy -------------------------------------+------------------------------------- Reporter: int-e | Owner: Type: feature | Status: new request | Priority: normal | Milestone: Component: libraries | Version: 8.0.1 (other) | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- The reason why this module isn't inferred safe is that it uses `unsafePerformIO` to create a global `IORef`. From a SafeHaskell perspective, this module isn't very useful in itself, but its unsafety prevents other modules that import it to provide class instances from being inferred as safe. A concrete example is https://hackage.haskell.org/package/exceptions-0.8.3/docs/src/Control- Monad-Catch.html#line-186 -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/12511> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 6

[GHC] #12896: Consider using compact regions in GHC itself to reduce GC overhead
by GHC 06 Jan '17

06 Jan '17

#12896: Consider using compact regions in GHC itself to reduce GC overhead -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: Type: feature | Status: new request | Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Compile-time Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- In make mode, GHC keeps a lot of in-memory data structures about modules it has built and interface files it has read. These data structures are long-lived and immutable (at least, per module/mutually recursive module group/package), so they may be candidates for storing in compact regions to reduce the work done by the GC. Potential problem: if we rely heavily on lazy evaluation in these structures for the purpose of avoiding work that will never be needed, then we can't store them in a compact region (since the contents of a compact region must be fully evaluated). -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/12896> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 2

[GHC] #9748: Disambiguate IO actions in GHCi with :set +t
by GHC 06 Jan '17

06 Jan '17

#9748: Disambiguate IO actions in GHCi with :set +t -------------------------------------+------------------------------------- Reporter: Iceland_jack | Owner: Type: feature request | Status: new Priority: low | Milestone: Component: GHCi | Version: 7.8.2 Keywords: | Operating System: Architecture: Unknown/Multiple | Unknown/Multiple Difficulty: Easy (less than 1 | Type of failure: hour) | None/Unknown Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Currently GHCi's `:set +t` displays the same type for the following: {{{#!hs % ghci -ignore-dot-ghci GHCi, version 7.8.2: http://www.haskell.org/ghc/ :? for help Loading package ghc-prim ... linking ... done. Loading package integer-gmp ... linking ... done. Loading package base ... linking ... done. Prelude> :set +t Prelude> 'a' 'a' it :: Char Prelude> return 'a' 'a' it :: Char Prelude> return 'a' :: IO Char 'a' it :: Char }}} A user at `#haskell-beginners` ran into a problem where they mistook an expression that defaulted to IO for a pure expression (`Id . even <$> pure 5`), `:set +t` was suggested but ended up showing the incorrect type: {{{#!hs ghci> Id . even <$> pure 5 Id False it :: Id Bool }}} This is bad for pedagogical reasons, it would be an improvement to display the following: {{{#!hs Prelude> :set +t Prelude> 'a' 'a' it :: Char Prelude> return 'a' 'a' it :: Char <- return 'a' :: IO Char Prelude> return 'a' :: IO Char 'a' it :: Char <- return 'a' :: IO Char }}} mirroring existing syntax: {{{#!hs Prelude> :set -XScopedTypeVariables Prelude> a :: Char <- return 'a' :: IO Char a :: Char }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9748> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 8

[GHC] #10352: Properly link Haskell shared libs on ELF systems
by GHC 05 Jan '17

05 Jan '17

#10352: Properly link Haskell shared libs on ELF systems -------------------------------------+------------------------------------- Reporter: duncan | Owner: Type: task | Status: new Priority: normal | Milestone: Component: Package | Version: 7.11 system | Operating System: Linux Keywords: | Type of failure: None/Unknown Architecture: | Blocked By: Unknown/Multiple | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Since the early days of building Haskell shared libs on Linux we have been using a scheme that is really a bit of a hack. We should do it properly. In my blog post on this from 2009 (http://www.well-typed.com/blog/30/) I said: > If we use ldd again to look at the libfoo.so that we've made we will notice that it is missing a dependency on the rts library. This is problem that we've yet to sort out, so for the moment we can just add the dependency ourselves: > > $ ghc --make -dynamic -shared -fPIC Foo.hs -o libfoo.so \ > -lHSrts-ghc6.11 -optl-Wl,-rpath,/opt/ghc/lib/ghc-6.11/ > > The reason it's not linked in yet is because we need to be able to switch which version of the rts we're using without having to relink every library. For example we want to be able to switch between the debug, threaded and normal rts versions. It's quite possible to do this and it just needs a bit more rearranging in the build system to sort it out. Once it's done you'll even be able to switch rts at runtime, eg: > > $ LD_PRELOAD=/opt/ghc/lib/ghc-6.11/libHSrts_debug-ghc6.11.so > $ ./Hello So in general, if a shared lib requires symbols from another shared lib then it should depend on it. In ELF terminology that means a NEEDED entry to say this lib needs that other lib. This is important to be able to link and load these shared libraries, otherwise they can have dangling dependencies. But we don't do this. For the specific case of the RTS we do not link Haskell shared libs against the RTS. So they have lots of dangling symbols. These libraries cannot be loaded on their own, e.g. with `dlopen()`. This is bad, and has other knock-on consequences. Why don't we link to the RTS? It's because historically (with static linking) GHC had had the ability to select the flavour of the RTS when final executables are linked, not when intermediate libraries are created. This works because the RTS flavours share a common ABI. This is a useful feature as it lets us select the SMP or debug or other RTS at final link time. So when we made up the first shared lib scheme on ELF we had to support this. Our initial scheme was like this: don't link Haskell library DSOs against the RTS, only like the final exe against the RTS. Each RTS flavour has a separate SONAME, e.g. `libHSrts_thr-ghc7.8.4.so` or `libHSrts_debug- ghc7.8.4.so`. This works because the runtime linker looks at the final exe first and loads the RTS, and then when other libs are loaded the symbols all resolve. Why can't we link all the libraries against the RTS? Currently each RTS flavour has a different SONAME, which is the key that the dynamic linker uses to identify each library. So if we did link all the Haskell libs against "the" RTS we would have to pick which one at the point at which we create the library, and that'd stop us from being able to choose later. So, can we use a better scheme? We want one that doesn't leave dangling undefined references in intermediate Haskell libs, and is also compatible with the ability to select the flavour of the RTS at final exe link time (or even override it at load time). Yes we can! The first thing to note is that to be interchangeable, all the RTS flavours (that share a compatible ABI) need to have the same SONAME. So for example, all the (non-profiling) RTS DSO files have to have the internal SONAME of `libHSrts-ghc7.8.4.so`. Once they all have the same SONAME, then it's ok for all the Haskell libs to specify a NEEDED dependency on that rts SONAME. But if they have the same SONAME, what do the files get called, where do they live and how are they found? The trick is to make use of the search path. Put each RTS flavour in a different directory, but otherwise with the same filename, e.g. `lib/rts-1.0/thr/libHSrts-ghc7.8.4.so`, `lib/rts-1.0/debug/libHSrts-ghc7.8.4.so` etc. Each library DSO and exe has its list of NEEDED entries, and it has an RPATH entry used to find those libraries if they're not loaded yet. The key is the "if they're not loaded yet" bit. Remember that the linker uses the SONAME as the key to decide if the lib is loaded yet or not. So the libraries could all have an RPATH entry to say to look for the RTS in the directory containing the default RTS flavour. But then the top level exe (or foreign/export shared lib) can also link to the RTS directly (ie an NEEDED entry) and can specify an RPATH which can be for any of the rts flavours. When the linker loads the top level exe, it will loads the selected RTS using the exe's RPATH, and then when the linker sees other Haskell libs that have a NEEDED entry on the RTS it will ignore them because the RTS's SONAME is already loaded. So concretely, instead of: lib/ghc-${ver}/rts-1.0/libHSrts-ghc${ver}.so lib/ghc-${ver}/rts-1.0/libHSrts_thr-ghc${ver}.so lib/ghc-${ver}/rts-1.0/libHSrts_debug-ghc${ver}.so lib/ghc-${ver}/rts-1.0/libHSrts_l-ghc${ver}.so lib/ghc-${ver}/rts-1.0/libHSrts_thr_l-ghc${ver}.so lib/ghc-${ver}/rts-1.0/libHSrts_thr_debug-ghc${ver}.so each with a different SONAME we'd have lib/ghc-${ver}/rts-1.0/libHSrts-ghc${ver}.so lib/ghc-${ver}/rts-1.0/thr/libHSrts-ghc${ver}.so lib/ghc-${ver}/rts-1.0/debug/libHSrts-ghc${ver}.so lib/ghc-${ver}/rts-1.0/l/libHSrts-ghc${ver}.so lib/ghc-${ver}/rts-1.0/thr_l/libHSrts-ghc${ver}.so lib/ghc-${ver}/rts-1.0/thr_debug/libHSrts-ghc${ver}.so each with the same SONAME When linking libs we would always use `-lHSrts -rpath lib/ghc-${ver}/rts-1.0` When linking exes (or shared libs for external consumption) we would use both `-lHSrts` and `-rpath lib/ghc-${ver}/${rtsflavour}/rts-1.0`. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10352> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 22