January 2014 - ghc-devs

Re: Pattern synonyms for 7.8?
by Dr. ERDI Gergo 08 Jan '14

08 Jan '14

On Tue, 7 Jan 2014, Austin Seipp wrote: > Hi Gergo, > > Thanks for rebasing your changes. Unfortunately, they do not compile > cleanly with ./validate, which we really need to have working for all > incoming patches. I've fixed validation (but the commit history is a bit of a mess -- expect another history rewrite shortly. But the code is at the same location on github. Thanks, Gergo -- .--= ULLA! =-----------------. \ http://gergo.erdi.hu \ `---= gergo(a)erdi.hu =-------' Glob thinkally, loc actally

2 2

RE: GHC API: Using runGhc twice or from multiple threads?
by p.k.f.holzenspies＠utwente.nl 08 Jan '14

08 Jan '14

Dear Benno, Simon, > I think that both ought to be ok, but I'd welcome other input. > > GHC.runGHC calls initGhcMonad, which allocates an entirely new session (in > newHscEnv). So the two will work entirely independently. > > Unfortunately that's not 100% true. If you search for GLOBAL_VAR you'll see > a handful of disgusting global state variables, and they *will* be shared > between GHC sessions. > > There really aren't many. I'd love someone to eliminate them! (NB ghc- > devs) There is one open question about what you intend to do with the results of running those GHC-sessions. Correct me if I'm wrong (Simon, or anyone), but methinks you should be very careful with assigned Uniques, i.e. (Rdr/Occ)Name-things from one session shouldn't be used in the other and vice versa. I was wondering about this earlier; it would be nice to have some more explicit combinators for manipulating GHC API states (e.g. combining two HscEnvs, such that any NamedThing defined in either will also exist in the result). I thinks some updated documentation on some of the states (HscEnv, GblEnv, LclEnv, etc.) would help people not well-versed with the GHC-innerts to bend their mind around the API. Regards, Philip

1 0

Re: LLVM and dynamic linking
by Aaron Friel 08 Jan '14

08 Jan '14

Replying to include the email list. You’re right, the llvm backend and the gmp licensing issues are orthogonal - or should be. The problem is I get build errors when trying to build GHC with LLVM and dynamic libraries. The result is that I get a few different choices when producing a platform image for development, with some uncomfortable tradeoffs: 1. LLVM-built GHC, dynamic libs - doesn’t build. 2. LLVM-built GHC, static libs - potential licensing oddities with me shipping a statically linked ghc binary that is now gpled. I am not a lawyer, but the situation makes me uncomfortable. 3. GCC/ASM-built GHC, dynamic libs - this is the *standard* for most platforms shipping ghc binaries, but it means that one of the biggest and most critical users of the LLVM backend is neglecting it. It also bifurcates development resources for GHC. Optimization work is duplicated and already devs are getting into the uncomfortable position of suggesting to users that they should trust GHC to build your programs in a particular way, but not itself. 4. GCC/ASM-built GHC, static libs - worst of all possible worlds. Because of this, the libgmp and llvm-backend issues aren’t entirely orthogonal. Trac ticket #7885 is exactly the issue I get when trying to compile #1. From: Carter Schonwald<mailto:carter.schonwald@gmail.com> Sent: ‎Monday‎, ‎December‎ ‎30‎, ‎2013 ‎1‎:‎05‎ ‎PM To: Aaron Friel<mailto:aaron@frieltek.com> Good question but you forgot to email the mailing list too :-) Using llvm has nothing to do with Gmp. Use the native code gen (it's simper) and integer-simple. That said, standard ghc dylinks to a system copy of Gmp anyways (I think ). Building ghc as a Dylib is orthogonal. -Carter On Dec 30, 2013, at 1:58 PM, Aaron Friel <aaron(a)frieltek.com<mailto:aaron@frieltek.com>> wrote: Excellent research - I’m curious if this is the right thread to inquire about the status of trying to link GHC itself dynamically. I’ve been attempting to do so with various LLVM versions (3.2, 3.3, 3.4) using snapshot builds of GHC (within the past week) from git, and I hit ticket #7885 [https://ghc.haskell.org/trac/ghc/ticket/7885] every time (even the exact same error message). I’m interested in dynamically linking GHC with LLVM to avoid the entanglement with libgmp’s license. If this is the wrong thread or if I should reply instead to the trac item, please let me know. From: Carter Schonwald<mailto:carter.schonwald@gmail.com> Sent: ‎Friday‎, ‎December‎ ‎27‎, ‎2013 ‎2‎:‎41‎ ‎PM To: Ben Gamari<mailto:bgamari.foss@gmail.com> Cc: ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org> great work! :) On Fri, Dec 27, 2013 at 3:21 PM, Ben Gamari <bgamari.foss(a)gmail.com<mailto:bgamari.foss@gmail.com>> wrote: Simon Marlow <marlowsd(a)gmail.com<mailto:marlowsd@gmail.com>> writes: > This sounds right to me. Did you submit a patch? > > Note that dynamic linking with LLVM is likely to produce significantly > worse code that with the NCG right now, because the LLVM back end uses > dynamic references even for symbols in the same package, whereas the NCG > back-end uses direct static references for these. > Today with the help of Edward Yang I examined the code produced by the LLVM backend in light of this statement. I was surprised to find that LLVM's code appears to be no worse than the NCG with respect to intra-package references. My test case can be found here[2] and can be built with the included `build.sh` script. The test consists of two modules build into a shared library. One module, `LibTest`, exports a few simple members while the other module (`LibTest2`) defines members that consume them. Care is taken to ensure the members are not inlined. The tests were done on x86_64 running LLVM 3.4 and GHC HEAD with the patches[1] I referred to in my last message. Please let me know if I've missed something. # Evaluation ## First example ## The first member is a simple `String` (defined in `LibTest`), helloWorld :: String helloWorld = "Hello World!" The use-site is quite straightforward, testHelloWorld :: IO String testHelloWorld = return helloWorld With `-O1` the code looks reasonable in both cases. Most importantly, both backends use IP relative addressing to find the symbol. ### LLVM ### 0000000000000ef8 <rKw_info>: ef8: 48 8b 45 00 mov 0x0(%rbp),%rax efc: 48 8d 1d cd 11 20 00 lea 0x2011cd(%rip),%rbx # 2020d0 <libtestzm0zi1zi0zi0_LibTest_helloWorld_closure> f03: ff e0 jmpq *%rax 0000000000000f28 <libtestzm0zi1zi0zi0_LibTest2_testHelloWorld_info>: f28: eb ce jmp ef8 <rKw_info> f2a: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) ### NCG ### 0000000000000d58 <rH1_info>: d58: 48 8d 1d 71 13 20 00 lea 0x201371(%rip),%rbx # 2020d0 <libtestzm0zi1zi0zi0_LibTest_helloWorld_closure> d5f: ff 65 00 jmpq *0x0(%rbp) 0000000000000d88 <libtestzm0zi1zi0zi0_LibTest2_testHelloWorld_info>: d88: eb ce jmp d58 <rH1_info> With `-O0` the code is substantially longer but the relocation behavior is still correct, as one would expect. Looking at the definition of `helloWorld`[3] itself it becomes clear that the LLVM backend is more likely to use PLT relocations over GOT. In general, `stg_*` primitives are called through the PLT. As far as I can tell, both of these call mechanisms will incur two memory accesses. However, in the case of the PLT the call will consist of two JMPs whereas the GOT will consist of only one. Is this a cause for concern? Could these two jumps interfere with prediction? In general the LLVM backend produces a few more instructions than the NCG although this doesn't appear to be related to handling of relocations. For instance, the inexplicable (to me) `mov` at the beginning of LLVM's `rKw_info`. ## Second example ## The second example demonstrates an actual call, -- Definition (in LibTest) infoRef :: Int -> Int infoRef n = n + 1 -- Call site testInfoRef :: IO Int testInfoRef = return (infoRef 2) With `-O1` this produces the following code, ### LLVM ### 0000000000000fb0 <rLy_info>: fb0: 48 8b 45 00 mov 0x0(%rbp),%rax fb4: 48 8d 1d a5 10 20 00 lea 0x2010a5(%rip),%rbx # 202060 <rLx_closure> fbb: ff e0 jmpq *%rax 0000000000000fe0 <libtestzm0zi1zi0zi0_LibTest2_testInfoRef_info>: fe0: eb ce jmp fb0 <rLy_info> ### NCG ### 0000000000000e10 <rI3_info>: e10: 48 8d 1d 51 12 20 00 lea 0x201251(%rip),%rbx # 202068 <rI2_closure> e17: ff 65 00 jmpq *0x0(%rbp) 0000000000000e40 <libtestzm0zi1zi0zi0_LibTest2_testInfoRef_info>: e40: eb ce jmp e10 <rI3_info> Again, it seems that LLVM is a bit more verbose but seems to handle intra-package calls efficiently. [1] https://github.com/bgamari/ghc/commits/llvm-dynamic [2] https://github.com/bgamari/ghc-linking-tests/tree/master/ghc-test [3] `helloWorld` definitions: LLVM: 00000000000010a8 <libtestzm0zi1zi0zi0_LibTest_helloWorld_info>: 10a8: 50 push %rax 10a9: 4c 8d 75 f0 lea -0x10(%rbp),%r14 10ad: 4d 39 fe cmp %r15,%r14 10b0: 73 07 jae 10b9 <libtestzm0zi1zi0zi0_LibTest_helloWorld_info+0x11> 10b2: 49 8b 45 f0 mov -0x10(%r13),%rax 10b6: 5a pop %rdx 10b7: ff e0 jmpq *%rax 10b9: 4c 89 ef mov %r13,%rdi 10bc: 48 89 de mov %rbx,%rsi 10bf: e8 0c fd ff ff callq dd0 <newCAF@plt> 10c4: 48 85 c0 test %rax,%rax 10c7: 74 22 je 10eb <libtestzm0zi1zi0zi0_LibTest_helloWorld_info+0x43> 10c9: 48 8b 0d 18 0f 20 00 mov 0x200f18(%rip),%rcx # 201fe8 <_DYNAMIC+0x228> 10d0: 48 89 4d f0 mov %rcx,-0x10(%rbp) 10d4: 48 89 45 f8 mov %rax,-0x8(%rbp) 10d8: 48 8d 05 21 00 00 00 lea 0x21(%rip),%rax # 1100 <cJC_str> 10df: 4c 89 f5 mov %r14,%rbp 10e2: 49 89 c6 mov %rax,%r14 10e5: 58 pop %rax 10e6: e9 b5 fc ff ff jmpq da0 <ghczmprim_GHCziCString_unpackCStringzh_info@plt> 10eb: 48 8b 03 mov (%rbx),%rax 10ee: 5a pop %rdx 10ef: ff e0 jmpq *%rax NCG: 0000000000000ef8 <libtestzm0zi1zi0zi0_LibTest_helloWorld_info>: ef8: 48 8d 45 f0 lea -0x10(%rbp),%rax efc: 4c 39 f8 cmp %r15,%rax eff: 72 3f jb f40 <libtestzm0zi1zi0zi0_LibTest_helloWorld_info+0x48> f01: 4c 89 ef mov %r13,%rdi f04: 48 89 de mov %rbx,%rsi f07: 48 83 ec 08 sub $0x8,%rsp f0b: b8 00 00 00 00 mov $0x0,%eax f10: e8 1b fd ff ff callq c30 <newCAF@plt> f15: 48 83 c4 08 add $0x8,%rsp f19: 48 85 c0 test %rax,%rax f1c: 74 20 je f3e <libtestzm0zi1zi0zi0_LibTest_helloWorld_info+0x46> f1e: 48 8b 1d cb 10 20 00 mov 0x2010cb(%rip),%rbx # 201ff0 <_DYNAMIC+0x238> f25: 48 89 5d f0 mov %rbx,-0x10(%rbp) f29: 48 89 45 f8 mov %rax,-0x8(%rbp) f2d: 4c 8d 35 1c 00 00 00 lea 0x1c(%rip),%r14 # f50 <cGG_str> f34: 48 83 c5 f0 add $0xfffffffffffffff0,%rbp f38: ff 25 7a 10 20 00 jmpq *0x20107a(%rip) # 201fb8 <_DYNAMIC+0x200> f3e: ff 23 jmpq *(%rbx) f40: 41 ff 65 f0 jmpq *-0x10(%r13) _______________________________________________ ghc-devs mailing list ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org> http://www.haskell.org/mailman/listinfo/ghc-devs

5 11

[PATCH] get rid of "Just" string in __GLASGOW_HASKELL_LLVM__ define for invoked GCC The patch fixes invoked GCC command line -D parameter from -D__GLASGOW_HASKELL_LLVM__=Just <LLVM version number> to correct -D__GLASGOW_HASKELL_LLVM__=<LLVM version number>, e.g. -D__GLASGOW_HASKELL_LLVM__=Just 32 fixed to -D__GLASGOW_HASKELL_LLVM__=32 for LLVM 3.2
by Karel Gardas 07 Jan '14

07 Jan '14

--- compiler/main/DriverPipeline.hs | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/compiler/main/DriverPipeline.hs b/compiler/main/DriverPipeline.hs index 337778e..f789d44 100644 --- a/compiler/main/DriverPipeline.hs +++ b/compiler/main/DriverPipeline.hs @@ -2086,7 +2086,9 @@ doCpp dflags raw input_fn output_fn = do getBackendDefs :: DynFlags -> IO [String] getBackendDefs dflags | hscTarget dflags == HscLlvm = do llvmVer <- figureLlvmVersion dflags - return [ "-D__GLASGOW_HASKELL_LLVM__="++show llvmVer ] + return $ case llvmVer of + Just n -> [ "-D__GLASGOW_HASKELL_LLVM__="++show n ] + _ -> [] getBackendDefs _ = return [] -- 1.7.3.2

2 1

Re: Validating with Haddock
by Mateusz Kowalczyk 07 Jan '14

07 Jan '14

On 07/01/14 13:57, Simon Hengel wrote: > Hey! > Sorry for not being of much help with this right now. Regarding Haddock releases I think we updated the version used for Hackage independently of ghc before. Cheers. > Oh, if that's the case then I no longer feel that it's urgent that we get it into 7.8 considering Hackage is where the bulk of the docs are. -- Mateusz K.

1 0

Test suite regressions
by Joachim Breitner 06 Jan '14

06 Jan '14

Hi, travis-ci reports test suite failures. Unfortunately, the builds still sometimes timeout, so I cannot pin-point the precise change, but someone pushing today broke Unexpected failures: ghci/scripts T8639 [bad stdout] (ghci) polykinds T7594 [stderr mismatch] (normal) https://s3.amazonaws.com/archive.travis-ci.org/jobs/16345412/log.txt If you pushed today, please check if you might have broken them. And please validate your changes before pushing! (I have some scripts that make clean validating mostly hassle-free, based on a dedicated build host where I push to "validate/some-name", and after lunch I come back and see if the branch was renamed to "validated/some-name" or "broken/some-name" – I can share them if you are interested. Although I believe that we would benefit from a central, official solution.) Greetings, Joachim -- Joachim “nomeata” Breitner mail(a)joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata(a)joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata(a)debian.org

1 1

Re: RE: Starting GHC development.
by Kyle Van Berendonck 06 Jan '14

06 Jan '14

The build/test system could be scaring away potential developers too. Not to complain or anything, but I used to try building GHC (form scratch) every 2 months or so and It would (usually) completely fail to build on Windows and OSX, and when it does, there would be some problem with the makefile where it either wouldn't rebuild properly and required a dist-clean or it wouldn't handle anything above -j1 properly. I'm (hoping) that these issues have been resolved by now. The test suite is also really scary. Regards.

1 0

Interface loading and dynamic linking
by Ben Gamari 04 Jan '14

04 Jan '14

In trying to test my LLVM-dynamic linking changes, I've encountered some builds which fail with, Bad interface file: dist-install/build/GHC/CString.hi mismatched interface file ways (wanted "dyn", got "") during `make install`. The failing command is `ghc-stage2`, invoked by `ghc-cabal`, ../../inplace/bin/ghc-stage2 '-B/opt/exp/ghc/root-ghc-7.8-dyn/lib/ghc-7.7.20131221' '--abi-hash' '-fbuilding-cabal-package' '-O' '-outputdir' 'dist-install/build' '-odir' 'dist-install/build' '-hidir' 'dist-install/build' '-stubdir' 'dist-install/build' '-i' '-idist-install/build' '-i.' '-idist-install/build/autogen' '-Idist-install/build/autogen' '-Idist-install/build' '-optP-include' '-optPdist-install/build/autogen/cabal_macros.h' '-package-name' 'ghc-prim-0.3.1.0' '-hide-all-packages' '-no-user-package-db' '-package-id' 'builtin_rts' '-XHaskell2010' 'GHC.CString' 'GHC.Classes' 'GHC.Debug' 'GHC.IntWord64' 'GHC.Magic' 'GHC.PrimopWrappers' 'GHC.Tuple' 'GHC.Types' 'GHC.Prim' '-package-name' 'ghc-prim' The build is on an x86_64 box with `build.mk` containing, GhcDebugged = YES DYNAMIC_BY_DEFAULT = YES # Roughly BuildFlavour=quick-llvm without disabling dynamic linking SRC_HC_OPTS = -H64m -O0 -fllvm # -keep-llvm-files GhcStage1HcOpts = -O -fllvm GhcStage2HcOpts = -O0 -fllvm GhcLibHcOpts = -O -fllvm SplitObjs = NO HADDOCK_DOCS = NO BUILD_DOCBOOK_HTML = NO BUILD_DOCBOOK_PS = NO BUILD_DOCBOOK_PDF = NO I've spent the morning tracing through the interface loading code and have finally met an impasse. As far as I can tell, the relevant subset of the call graph is, LoadIface.lhs: loadInterface LoadIface.lhs : findAndReadIface Finder.hs: findExactModule Finder.hs: findHomeModule Finder.hs: homeSearchCache Finder.hs: findHomeModule findAndReadIface(read_file) findAndReadIface(checkBuildDynamicToo) whenGeneratingDynamicToo TcRnMonad.hs: withDoDynamicToo DynFlags.hs: dynamicTooMkDynamicDynFlags findAndReadIface(read_file) where findAndReadIFace(read_file) LoadIface.lhs : readIface BinIface.hs : readBinIface Up until `checkBuildDynamicToo` the dynflags have, ways dflags == [WayDyn] buildTag dflags == "dyn" hiSuf dflags == "hi" dynHiSuf dflags == "dyn_hi" Given this, it's perhaps unsurprising that `readBinIface` gets quite confused when it attempts to read `dist-install/build/GHC/CString.hi` to find that it was compiled with a null tag, not `dyn` as expected. If I add `-static` to the command line the build succeeds. However, adding `-dynamic` or `-dynamic-too` or omitting all of the above (due to `DYNAMIC_BY_DEFAULT`) all fail. It seems that `findAndReadIface` attempts to account for `-dynamic-too` but I don't see how the approach should work: it tries to first load the static interface file (but with `ways==[WayDyn]`) then tries again to load the interface using `DynFlags` and .hi file name constructed for the dynamic case. I can't see how this would ever work as `readBinIface` will expect to see a dynamic interface file due to the DynFlags' ways. It's also unclear how the `-dynamic` case (not `-dynamic-too`) is handled under this scheme. I would have thought that running with `-dynamic` would cause `hiSuf == dynHiSuf` but I can't find any code to suggest this is the case. Could someone offer some insight into how interface loading, `-dynamic`, and `-dynamic-too` are supposed to interact? Thanks, - Ben

4 9

Normal for make install to (re?)build libraries with stage1 compiler?
by Aaron Friel 04 Jan '14

04 Jan '14

Still working on getting my own development environment configured, I am seeing make install perform a lot of rebuilds of libraries: > "inplace/bin/ghc-stage1" -hisuf hi -osuf o -hcsuf hc -static -H64m -O0 -fasm -package-name base-4.7.0.0 -hide-all-packages -i -ilibraries/base/. -ilibraries/base/dist-install/build -ilibraries/base/dist-install/build/autogen -Ilibraries/base/dist-install/build -Ilibraries/base/dist-install/build/autogen -Ilibraries/base/include -optP-DOPTIMISE_INTEGER_GCD_LCM -optP-include -optPlibraries/base/dist-install/build/autogen/cabal_macros.h -package ghc-prim-0.3.1.0 -package integer-gmp-0.5.1.0 -package rts-1.0 -package-name base -XHaskell2010 -O2 -fllvm -no-user-package-db -rtsopts -odir libraries/base/dist-install/build -hidir libraries/base/dist-install/build -stubdir libraries/base/dist-install/build -split-objs -c libraries/base/./GHC/IO/Encoding/Types.hs -o libraries/base/dist-install/build/GHC/IO/Encoding/Types.o However, GHC has already been built to stage 2. Why is GHC inplace/bin/stage1 being invoked here - and hasn’t this library already been built by make?

2 1

ticket for adding ARM backend to NCG?
by Corey O'Connor 04 Jan '14

04 Jan '14

Is there an existing ticket or wiki page describing the work required for adding an ARM backend to the NCG? I tried the following search with no luck: https://ghc.haskell.org/trac/ghc/query?status=infoneeded&status=merge&statu… -Corey O'Connor coreyoconnor(a)gmail.com http://corebotllc.com/

9 13