May 2016 - ghc-devs - Haskell.org

Re: suboptimal ghc code generation in IO vs equivalent pure code case
by Dan Doel 14 May '16

14 May '16

I'm no expert on reading GHC's generated assembly. However, there may be a line you've overlooked in explaining the difference, namely: movq $stg_upd_frame_info,-16(%rbp) This appears only in the IO code, according to what you've pasted, and it appears to be pushing an update frame (I think). Update frames are used as part of lazy evaluation, to ensure that work only happens once, barring very short race conditions. So, at a guess, I would say that your IO-based code is creating extra updatable closures, which isn't free. It's sometimes difficult to see the difference at the core level. It will probably be clearer at the STG level, because the expression language is more disciplined there. But, for instance, your pure code tail calls (++), whereas your IO code returns an unboxed tuple with the same sort of expression that is in the pure tail call. However, complex expressions like that can't really be put in an unboxed tuple at the STG level, so what will happen is that the complex expression will be let (closure allocation), and the closure will be returned in the unboxed tuple. So that is the source of the difference. A more perspicuous picture would be something like: Pure: False -> let { l1 = : ww_amuh [] l2 = Data.Unicode.Internal.Normalization.decompose_$sdecompose ipv_smuv ipv1_smuD } in ++ l1 l2 IO: False -> case $sa1_sn0g ipv_smUT ipv1_smV6 ipv2_imWU of _ { (# ipv4_XmXv, ipv5_XmXx #) -> let { l1 = : sc_sn0b [] l3 = ++ l1 ipv5_XmXx } in (# ipv4_XmXv, l3 #) I can't say for certain that that's the only thing making a difference, but it might be one thing. -- Dan On Mon, May 9, 2016 at 10:23 AM, Harendra Kumar <harendra.kumar(a)gmail.com> wrote: > I have a loop which runs millions of times. For some reason I have to run it > in the IO monad. I noticed that when I convert the code from pure to IO > monad the generated assembly code in essence is almost identical except one > difference where it puts a piece of code in a separate block which is making > a huge difference in performance (4-6x slower). > > I want to understand what makes GHC to generate code in this way and if > there is anything that can be done at source level (or ghc option) to > control that. > > The pure code looks like this: > > decomposeChars :: [Char] -> [Char] > > decomposeChars [] = [] > decomposeChars [x] = > case NFD.isDecomposable x of > True -> decomposeChars (NFD.decomposeChar x) > False -> [x] > decomposeChars (x : xs) = decomposeChars [x] ++ decomposeChars xs > > The equivalent IO code is this: > > decomposeStrIO :: [Char] -> IO [Char] > > decomposeStrPtr !p = decomposeStrIO > where > decomposeStrIO [] = return [] > decomposeStrIO [x] = do > res <- NFD.isDecomposable p x > case res of > True -> decomposeStrIO (NFD.decomposeChar x) > False -> return [x] > decomposeStrIO (x : xs) = do > s1 <- decomposeStrIO [x] > s2 <- decomposeStrIO xs > return (s1 ++ s2) > > The difference is in how the code corresponding to the call to the (++) > operation is generated. In the pure case the (++) operation is inline in the > main loop: > > _cn5N: > movq $sat_sn2P_info,-48(%r12) > movq %rax,-32(%r12) > movq %rcx,-24(%r12) > movq $:_con_info,-16(%r12) > movq 16(%rbp),%rax > movq %rax,-8(%r12) > movq $GHC.Types.[]_closure+1,(%r12) > leaq -48(%r12),%rsi > leaq -14(%r12),%r14 > addq $40,%rbp > jmp GHC.Base.++_info > > In the IO monad version this code is placed in a separate block and a call > is placed in the main loop: > > the main loop call site: > > _cn6A: > movq $sat_sn3w_info,-24(%r12) > movq 8(%rbp),%rax > movq %rax,-8(%r12) > movq %rbx,(%r12) > leaq -24(%r12),%rbx > addq $40,%rbp > jmp *(%rbp) > > out of the line block - the code that was in the main loop in the previous > case is now moved to this block (see label _cn5s below): > > sat_sn3w_info: > _cn5p: > leaq -16(%rbp),%rax > cmpq %r15,%rax > jb _cn5q > _cn5r: > addq $24,%r12 > cmpq 856(%r13),%r12 > ja _cn5t > _cn5s: > movq $stg_upd_frame_info,-16(%rbp) > movq %rbx,-8(%rbp) > movq 16(%rbx),%rax > movq 24(%rbx),%rbx > movq $:_con_info,-16(%r12) > movq %rax,-8(%r12) > movq $GHC.Types.[]_closure+1,(%r12) > movq %rbx,%rsi > leaq -14(%r12),%r14 > addq $-16,%rbp > jmp GHC.Base.++_info > _cn5t: > movq $24,904(%r13) > _cn5q: > jmp *-16(%r13) > > Except this difference the rest of the assembly looks pretty similar in both > the cases. The corresponding dump-simpl output for the pure case: > > False -> > ++ > @ Char > (GHC.Types.: @ Char ww_amuh (GHC.Types.[] @ Char)) > (Data.Unicode.Internal.Normalization.decompose_$sdecompose > ipv_smuv ipv1_smuD); > > And for the IO monad version: > > False -> > case $sa1_sn0g ipv_smUT ipv1_smV6 ipv2_imWU > of _ [Occ=Dead] { (# ipv4_XmXv, ipv5_XmXx #) -> > (# ipv4_XmXv, > ++ > @ Char > (GHC.Types.: @ Char sc_sn0b (GHC.Types.[] @ Char)) > ipv5_XmXx #) > }; > > The dump-simpl output is essentially the same except the difference due to > the realworld token in the IO case. Why is the generated code different? I > will appreciate if someone can throw some light on the reason or can point > to the relevant ghc source to look at where this happens. > > I am using ghc-7.10.3 in native code generation mode (no llvm). > > Thanks, > Harendra > > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users(a)haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users >

3 6

arc stopped working
by Peter Trommler 13 May '16

13 May '16

Hi *, I am getting the following error message from arc: peter@montebre:/local/home/peter/ghc> arc diff Usage Exception: Failed to load phutil library at location '/local/home/peter/ghc/.arc-linters/arcanist-external-json-linter'. This library is specified by the "load" setting in ".arcconfig". Check that the setting is correct and the library is located in the right place. I looked at '/local/home/peter/ghc/.arc-linters/arcanist-external-json-linter' and it is an empty directory. `arc upgrade` did not fix the issue. What can I do to fix arc? Thanks, Peter

2 2

Stop rejecting Summary: Signed-off-by: ... OR patch Phabricator
by Edward Z. Yang 13 May '16

13 May '16

Hey guys, Currently the commit hooks reject commit messages that look like: Summary: Signed-off-by: Edward Z. Yang <ezyang(a)cs.stanford.edu> Unfortunately, if I do a one line commit message with -s, Phabricator will automatically reformat my message to have this. This is dumb. Either: 1) You should stop rejecting these commit messages, or 2) You should patch Phabricator to stop munging the messages this way. The message formatting is done server-side so as a client I have no way of changing it, you need to fix it. This URL has some guidance on how to do it: https://secure.phabricator.com/T6055 I'd offer to edit the hook myself but there does not seem to be any canonical location where the hooks are versioned. Thanks, Edward

2 1

Mentor for a JVM backend for GHC
by Rahul Muttineni 12 May '16

12 May '16

Hi GHC Developers, I've started working on a JVM backend for GHC [1] and I'd love to work on it as my Summer of Haskell project. Currently, the build system is setup using a mix of Shake (for the RTS build) and Stack (for the main compiler build) and I ensure that most commits build successfully. I have ported the core part of the scheduler and ported over the fundamental types (Capability, StgTSO, Task, StgClosure, etc.) taking advantage of OOP in the implementation when I could. Additionally, I performed a non-trivial refactor of the hs-java package adding support for inner classes and fields which was very cumbersome to do in the original package. On the frontend, I have tapped into the STG code from the GHC 7.10.3 library and setup a CodeGen monad for generating JVM bytecode. The main task of generating the actual bytecode, porting the more critical parts of the RTS, and adding support for the threaded RTS remain. The strategy for compilation is as follows: - Intercept the STG code in the GHC pipeline - Convert from STG->JVM bytecode [2] in a similar manner as STG->Cmm preserving semantics as best as possible [3] - Port the GHC RTS (normal & threaded) to Java [4] - Put all the generated class files + RTS into a single jar to be run directly by the JVM. My objectives for the project during the summer are: - To implement the compilation strategy mentioned above - Implement the Java FFI for foreign imports. [5] - Implement the most important [6] PrimOps that GHC supports. - Port the base package replacing the C FFI imports with equivalent Java FFI imports. [7] A little bit about myself: I spent a lot of time studying functional language implementation by reading SPJ's famous book and reading research papers on related topics last summer as self-study. I took a break and resumed a couple months ago where I spent a lot of time plowing through the STG->Cmm code generator as well as the RTS and going back and forth between them to get a clear understanding of how everything works. Moreover, I compiled simple Haskell programs and observed the STG, Cmm, and assembly output (by decompiling the final executable with objdump) to understand bits of the code generator where the source code wasn't that clear. I also spent a great deal of time studying the JVM internals, reading the JVM spec, looking for any new features that could facilitate a high performance implementation [8]. It would be great if someone with an understanding of nuances of the RTS and code generator could mentor me for this project. It has been a blast so far learning all the prerequisites and contemplating the design. I'd be very excited to take this on as a summer project. Also, given that I have hardly 5 days remaining, does anyone have suggestions on how I can structure the proposal without getting into too many details? There are still some parts of the design I haven't figured out, but I know I could find some solution when I get to it during the porting process. Thanks, Rahul Muttineni [1] http://github.com/rahulmutt/ghcvm [2] I intend to organically derive an IR at a later stage to allow for some optimizations by looking at the final working implementation without an IR and looking for patterns of repeated sequences of bytecode and assigning each sequence its own instruction in the IR. [3] Obviously, the lack of control of memory layouts (besides allocating off the JVM heap using DirectByteBuffers) and lack of general tail calls makes it tough to match the semantics of Cmm, but there are many solutions around it, as can be found in the few papers on translating STG to Java/JVM bytecode. [4] This is the GHC RTS without GC and profiling since the JVM has great support for those already. Also, lots of care must be taken to ensure that the lock semantics stays in tact during the port. [5] foreign exports will be dealt at a later stage, but I am taking care of naming the closures nicely so that in the future you don't have to type long names like the labels GHC compiles to call a Haskell function in Java. [6] Basically all the PrimOps that would be required to provide plumbing for the Prelude functions that can compile beginner-level programs found in books such as Learn You a Haskell for Great Good. [7] I know that it's a lot more complicated than just replacing FFI calls. I'd have to change around a lot of the code in base as well. [8] I found that the new "invokedynamic" instruction as well as the MethodHandle API (something like function pointers) that were introduced in JDK 7 could fit the bill. But as of now, I want to get a baseline implementation that is compatible with Java 5 so I will not be utilizing these newer features.

9 18

Determining if an instance is provided by the environment
by Edward Z. Yang 10 May '16

10 May '16

Hello all, Suppose I have a ClsInst from typechecking the following instance declaration: instance Show [Char] -- i.e. String I'd now like to answer the question: "Is this instance 'provided' by the instance environment?" For example, this instance is provided given that I have these two instances in the environment: instance Show a => Show [a] -- (1) instance Show Char -- (2) However, if I have just instance (1) in the environment, it's not provided (and if you tried to use show "foo", you'd get the error that Char is not an instance of Show.) Is there are convenient way to do this from TcM? With 'tcMatchTys' and I can easily test if there is some instance in the environment which *matches* my instance head (e.g., Show [a] matches Show [Char]) but this doesn't tell me if all the resulting constraints are solvable. Thanks, Edward

3 5

Harbourmaster is failing
by Simon Peyton Jones 10 May '16

10 May '16

Lots of early Harbourmaster fails https://phabricator.haskell.org/harbormaster/build/11545/ Simon In file included from includes/Stg.h:213:0, from includes/Rts.h:30, from includes/dist-derivedconstants/header/tmp.c:13: includes/stg/Types.h:31:2: error: #import is a deprecated GCC extension [-Werror=deprecated] #import <inttypes.h> ^ cc1: all warnings being treated as errors Executing "gcc" failed

3 3

suboptimal ghc code generation in IO vs equivalent pure code case
by Harendra Kumar 09 May '16

09 May '16

I have a loop which runs millions of times. For some reason I have to run it in the IO monad. I noticed that when I convert the code from pure to IO monad the generated assembly code in essence is almost identical except one difference where it puts a piece of code in a separate block which is making a huge difference in performance (4-6x slower). I want to understand what makes GHC to generate code in this way and if there is anything that can be done at source level (or ghc option) to control that. The pure code looks like this: decomposeChars :: [Char] -> [Char] decomposeChars [] = [] decomposeChars [x] = case NFD.isDecomposable x of True -> decomposeChars (NFD.decomposeChar x) False -> [x] decomposeChars (x : xs) = decomposeChars [x] ++ decomposeChars xs The equivalent IO code is this: decomposeStrIO :: [Char] -> IO [Char] decomposeStrPtr !p = decomposeStrIO where decomposeStrIO [] = return [] decomposeStrIO [x] = do res <- NFD.isDecomposable p x case res of True -> decomposeStrIO (NFD.decomposeChar x) False -> return [x] decomposeStrIO (x : xs) = do s1 <- decomposeStrIO [x] s2 <- decomposeStrIO xs return (s1 ++ s2) The difference is in how the code corresponding to the call to the (++) operation is generated. In the pure case the (++) operation is inline in the main loop: _cn5N: movq $sat_sn2P_info,-48(%r12) movq %rax,-32(%r12) movq %rcx,-24(%r12) movq $:_con_info,-16(%r12) movq 16(%rbp),%rax movq %rax,-8(%r12) movq $GHC.Types.[]_closure+1,(%r12) leaq -48(%r12),%rsi leaq -14(%r12),%r14 addq $40,%rbp jmp GHC.Base.++_info In the IO monad version this code is placed in a separate block and a call is placed in the main loop: the main loop call site: _cn6A: movq $sat_sn3w_info,-24(%r12) movq 8(%rbp),%rax movq %rax,-8(%r12) movq %rbx,(%r12) leaq -24(%r12),%rbx addq $40,%rbp jmp *(%rbp) out of the line block - the code that was in the main loop in the previous case is now moved to this block (see label _cn5s below): sat_sn3w_info: _cn5p: leaq -16(%rbp),%rax cmpq %r15,%rax jb _cn5q _cn5r: addq $24,%r12 cmpq 856(%r13),%r12 ja _cn5t _cn5s: movq $stg_upd_frame_info,-16(%rbp) movq %rbx,-8(%rbp) movq 16(%rbx),%rax movq 24(%rbx),%rbx movq $:_con_info,-16(%r12) movq %rax,-8(%r12) movq $GHC.Types.[]_closure+1,(%r12) movq %rbx,%rsi leaq -14(%r12),%r14 addq $-16,%rbp jmp GHC.Base.++_info _cn5t: movq $24,904(%r13) _cn5q: jmp *-16(%r13) Except this difference the rest of the assembly looks pretty similar in both the cases. The corresponding dump-simpl output for the pure case: False -> ++ @ Char (GHC.Types.: @ Char ww_amuh (GHC.Types.[] @ Char)) (Data.Unicode.Internal.Normalization.decompose_$sdecompose ipv_smuv ipv1_smuD); And for the IO monad version: False -> case $sa1_sn0g ipv_smUT ipv1_smV6 ipv2_imWU of _ [Occ=Dead] { (# ipv4_XmXv, ipv5_XmXx #) -> (# ipv4_XmXv, ++ @ Char (GHC.Types.: @ Char sc_sn0b (GHC.Types.[] @ Char)) ipv5_XmXx #) }; The dump-simpl output is essentially the same except the difference due to the realworld token in the IO case. Why is the generated code different? I will appreciate if someone can throw some light on the reason or can point to the relevant ghc source to look at where this happens. I am using ghc-7.10.3 in native code generation mode (no llvm). Thanks, Harendra

1 0

[Maintainer needed] Actively maintained Haddock are happy Haddock
by Ben Gamari 06 May '16

06 May '16

tl;dr. Our Haddock need a dedicated maintainer responsible for their care and feeding. Would you like to move forward a critical piece of infrastructure used on a daily basis by thousands of Haskellers? Let us know if so. Hello everyone, While for several years Mateusz Kowalczyk and Simon Hengel have done an admirable job in serving as comaintainers of the Haddock documentation tool, they have both indicated that they no longer have the time to continue to be active in this role. They both deserve our thanks for their service. While Matthew Pickering and I have tried to pick up some of the Haddock's maintenance load in the meantime, we really need a dedicated maintainer to ensure that Haddock continues to swim along healthfully. At the moment there are over 150 open tickets in the issue tracker [1], an open call for a stable release, and 4 patches in need of review. We currently lack the time to get on top of such a load. In particular, Haddock need someone to take ownership of the management of the project: this person would keep an eye on (and preferably pare down) the issue tracker (which typically isn't very active, happily), guide new contributors, and manage releases. This is an opportunity to drive forward a project used on a daily basis by a significant fraction of Haskellers. If you think you might be interested in serving in this role, please let us know. Thanks! Cheers, - Ben [1] https://github.com/haskell/haddock/pulls

2 2

pmcheck/T11195
by Simon Peyton Jones 06 May '16

06 May '16

Harbourmaster is failing (“Killed”) on pmcheck/should_compile/T11195. Are others finding this? What should we do? SImon

1 0

Re: [Haskell-cafe] Call for Contributions - Haskell Communities and Activities Report, May 2016 edition (30th edition)
by Ben Gamari 05 May '16

05 May '16

Mihai Maruseac <mihai.maruseac(a)gmail.com> writes: > Hi, > > Thanks for the entry. In the past, I was copying the text at > https://ghc.haskell.org/trac/ghc/wiki/Status/May16 (with the > corresponding different date at the end) and formatting it in LaTeX. > Now I got a tex file which looks exactly like the text in the Trac > wiki. If not mentioned otherwise, I guess I'll use the .tex file, > since it will be easier. > Ahh, I see; thanks for letting me know! I think this time around it would be better you use the TeX source I provided as I made a few formatting improvements over the Trac document. Cheers, - Ben

1 0