November 2013 - ghc-devs

simd branch ready for review/merge
by Geoffrey Mainland 07 Nov '13

07 Nov '13

The SIMD branch, available as wip/simd, is ready for review/merge. It could use some review---Simon and Simon, I'd be especially grateful if you both had a quick look. Some major points: 1) I have added support for AVX 512, although this is necessarily untested. AVX and AVX2 are also both supported. 2) After the recent churn regarding patching LLVM's GHC calling convention, by default only 128-bit wide SIMD vectors are passed in registers, and then only on X86_64. There is a "hidden" flag, -fllvm-pass-vectors-in-regs, that causes GHC to generate LLVM code that assumes all vectors are passed in registers by LLVM. This can be used with a suitably patched version of LLVM, and if we get LLVM 3.4 patched, we can consider turning it on by default for LLVM 3.4+. This would mean that we couldn't mix LLVM <3.3-compiled object files with LLVM >3.4-compiled object files, but I don't see that as much of a problem. 3) utils/genprimcode has been hacked up to allow us to write vector operations once and have them instantiated at multiple vector types. I'm not thrilled with this solution, but after discussing with Simon PJ, what I've implemented seems to be the minimal reasonable solution to the problem of exploding primop boilerplate. The changes are documented in compiler/prelude/primops.txt.pp. 4) Error handling is sub-optimal. My patch checks to make sure that vector primops can be compiled efficiently based on the current set of dynamic flags. For example, if -mavx is not specified and the user tries to use a primop that adds together two 256-bit wide vectors of double-precision elements, the user will see an error message like: ghc-stage2: sorry! (unimplemented feature or known bug) (GHC version 7.7.20130916 for x86_64-unknown-linux): 256-bit wide floating point SIMD vector instructions require at least -mavx. This is because the only good place to check for this kind of error is during STG->Cmm translation (in compiler/codeGen/StgCmmPrim.hs), and we don't have much of an error handling infrastructure there in contrast to when we're working in the typechecking/renaming monad. If there is a better way/place to do this, please let me know. Thanks, Geoff

4 8

Possible bug related to stm and exceptions
by Andreas Voellmy 07 Nov '13

07 Nov '13

Hi all, I have a program that uses STM heavily and also performs lots of foreign calls. I've noticed that sometimes the program uses 100% CPU indefinitely and uses lots of memory - I see it go up to about 5GB before I kill it. I've grabbed some preliminary samples of stack traces and see lots stm related stuff (e.g. lots of stg_atomically_frame_info and stmCommitTransaction entries). I can pretty reliably get the behavior to happen now by closing a socket that my Haskell program is trying to recv from. When this causes an exception to be raised (something like "recv: resource vanished (Connection reset by peer)") , then this behavior gets triggered. I haven't pinned down the bug yet, but I'm suspecting it is STM related - somehow the exception causes some STM transaction to go wrong. Are there any known bugs that sound similar to this? BTW, this is with GHC 7.6.3 from a recent HP release on OS X. -Andi

4 6

windows agiain
by Simon Peyton-Jones 07 Nov '13

07 Nov '13

Austin I'm crashing in the testsuite as below. On Windows with the new msys. Any ideas? How do I set my "Windows terminal type"? Simon python2 ../../../driver/runtests.py -e ghc_compiler_always_flags="'-fforce-recomp -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts '" -e ghc_debugged=False -e ghc_with_native_codegen=1 -e ghc_with_vanilla=1 -e ghc_with_dynamic=1 -e ghc_with_profiling=0 -e ghc_with_threaded_rts=1 -e ghc_with_dynamic_rts=1 -e ghc_with_interpreter=0 -e ghc_unregisterised=0 -e ghc_dynamic_by_default=False -e ghc_dynamic=False -e ghc_with_smp=1 -e ghc_with_llvm=0 -e windows=True -e darwin=False -e in_tree_compiler=True -e clean_only=False --rootdir=. --config=../../../config/ghc -e 'config.confdir="../../../config"' -e 'config.compiler="C:/code/HEAD/inplace/bin/ghc-stage1.exe"' -e 'config.ghc_pkg="C:/code/HEAD/inplace/bin/ghc-pkg.exe"' -e 'config.hp2ps="C:/code/HEAD/inplace/bin/hp2ps.exe"' -e 'config.hpc="C:/code/HEAD/inplace/bin/hpc.exe"' -e 'config.gs="gs"' -e 'config.platform="i386-unknown-mingw32"' -e 'config.os="mingw32"' -e 'config.arch="i386"' -e 'config.wordsize="32"' -e 'default_testopts.cleanup=""' -e 'config.timeout=int() or config.timeout' -e 'config.timeout_prog="../../../timeout/install-inplace/bin/timeout.exe"' -e 'config.exeext=".exe"' -e 'config.top="C:/code/HEAD/testsuite"' \ --only=T8037 \ \ \ \ \ \ Traceback (most recent call last): File "../../../driver/runtests.py", line 144, in <module> raise Exception("Can't detect Windows terminal type") Exception: Can't detect Windows terminal type ../../../mk/test.mk:242: recipe for target 'test' failed make: *** [test] Error 1 should_compile $

1 0

Windows linking
by Simon Peyton-Jones 07 Nov '13

07 Nov '13

Austin Ok with the new msys2 I get the too-many-symbols thing below. This didn't happen before. Maybe I can switch off dynamic linking? How? Simon Too many symbols in DLL compiler/stage2/build/libHSghc-7.7.20131107-0-ghc7.7.20131107.dll [5263] ghczm7zi7zi20131107_Var_zdfUniquableVar_closure [5264] ghczm7zi7zi20131107_Var_zdfUniquableVar_info [5265] ghczm7zi7zi20131107_Var_zdwsetVarUnique_closure [5266] ghczm7zi7zi20131107_Var_zdwsetVarUnique_info [5267] ghczm7zi7zi20131107_Var_zdwupdateTyVarKindM_closure [5268] ghczm7zi7zi20131107_Var_zdwupdateTyVarKindM_info [5269] ghczm7zi7zi20131107_Var_zdwzdcgmapMp_closure [5270] ghczm7zi7zi20131107_Var_zdwzdcgmapMp_info [5271] setHeapSize compiler/ghc.mk:472: recipe for target 'compiler/stage2/build/libHSghc-7.7.20131107-0-ghc7.7.20131107.dll' failed make[1]: *** [compiler/stage2/build/libHSghc-7.7.20131107-0-ghc7.7.20131107.dll] Error 1 make[1]: *** Deleting file 'compiler/stage2/build/libHSghc-7.7.20131107-0-ghc7.7.20131107.dll' Makefile:64: recipe for target 'all' failed make: *** [all] Error 2 HEAD $

1 0

GHC error message on type mismatch
by Daniel Trstenjak 07 Nov '13

07 Nov '13

Hello, I don't know if I'm the only one struggeling with this GHC error message on type mismatches or it's because I'm not a full time Haskeller, or because I'm not a native english speaker. Couldn't match type `A' with `B´ Expected type: B Actual type: A My problem is with 'Expected' and 'Actual', that I'm often unsure if the compiler is "expecting" something or if I'm the expecting one and the same goes for "actual". I know that the compiler is the expecting one and that I'm given the actual thing, but it's quite often that I'm looking at this error and have to repeat this reasoning. It's strange, because normaly I can memorize such things quite good, but this one bothers me. Perhaps it would be easier for myself if 'Actual type' would be called e.g. 'Given type', I don't know, that just one of the two has a less generic meaning. Greetings, Daniel

1 0

Another CPP gotcha for the manual
by Mark Lentczner 06 Nov '13

06 Nov '13

Turns out that c pre-processing removes C style (/*…*/) comments. Of course, it will do this anywhere in the source. The following program can tell if it is compiled *-cpp* or not: module Main where (*/**) :: Double -> Double -> Double (*/**) a b = a / (2 ** b) howCompiled = if use > 1/16 then "normal" else "-cpp" where use = 1 */** 2 + 3 */** 4 main = putStrLn $ "compiled " ++ howCompiled When run: & runhaskell CppTest.hs compiled normal & runhaskell -cpp CppTest.hs compiled -cpp An example in the wild is in the package *wai-extra*, in the file *Network/Wai/Middleware/RequestLogger.hs* where the */* construct appears twice in the comments. Short of defining and implementing our own CPP-like preprocessing (something we might actually consider), I don't think there really is any fix for this, so the bug is that it should appear in the GHC documentation on CPP mode (§4.12.3), along with similar warnings about trailing back-slashes. Note that the way in which a multi-line comment is removed differs between *gcc* and *clang*. In *gcc*, the comment is removed and content of the line before the comment, and contents of the line after the comment are joined into a single line. In *clang*, the two line fragments are kept on separate lines. In both cases extra empty lines are added to keep the line count the same. The consequence of the *gcc* / *clang* difference is that neither the above code, nor wai-extra will compile with *clang*. Note: As far as I can tell this is not a *clang* bug, but a failure of specs: The C definition of comments and their removal is vague, and my guess is *gcc* choose its method based on historical use. The C++ definition makes it clear that comments are whitespace, even once removed, and so the *clang* method is correct. - Mark

7 10

Any GHC plans to adopt tf-random?
by Nicolas Frisby 06 Nov '13

06 Nov '13

Bugs #3575 #3620 have to do with splitting StdGens yielding dependent generators. The Haskell 2013 paper *Splittable Pseudorandom Number Generators Using Cryptographic Hashing* by Claessen and Pałka yielded the tf-random package on Hackage. Are there any plans to integrate these corrections into System.Random? Thanks.

2 2

Template Haskell and stage-1
by Simon Peyton-Jones 06 Nov '13

06 Nov '13

I'd like to make -XtemplateHaskell simply illegal in a stage-1 compiler. After all, it is! But for some reason we use the stage1 compiler to generate dependencies for the DPH libraries; those libraries have {-# LANGUAGE TemplateHaskell #-}, so the dependency generation fails. See below. But WHY do we generate deps for DPH with stage1? We don't *compile* the DPH libraries with stage1, because they need TH. Can't we just generate deps with stage2? For now I'll make it a warning, but could someone fix the build system? Thanks Simon mv libraries/dph/dph-lifted-copy/dist-install/build/.depend-v-dyn.c_asm.tmp libraries/dph/dph-lifted-copy/dist-install/build/.depend-v-dyn.c_asm "rm" -f libraries/dph/dph-lifted-copy/dist-install/build/.depend-v-dyn.haskell.tmp "inplace/bin/ghc-stage1" -M -static -H32m -O -Werror -Wall -H64m -O0 -package-name dph-lifted-copy-0.8.0.1 -hide-all-packages -i -ilibraries/dph/dph-lifted-copy/. -ilibraries/dph/dph-lifted-copy/dist-install/build -ilibraries/dph/dph-lifted-copy/dist-install/build/autogen -Ilibraries/dph/dph-lifted-copy/dist-install/build -Ilibraries/dph/dph-lifted-copy/dist-install/build/autogen -Ilibraries/dph/dph-lifted-copy/. -optP-include -optPlibraries/dph/dph-lifted-copy/dist-install/build/autogen/cabal_macros.h -package array-0.5.0.0 -package base-4.7.0.0 -package dph-base-0.8.0.1 -package dph-prim-par-0.8.0.1 -package ghc-7.7.20131106 -package random-1.0.1.1 -package template-haskell-2.9.0.0 -package vector-0.10.9.1 -Odph -funbox-strict-fields -fcpr-off -fno-warn-orphans -fno-warn-missing-signatures -XHaskell98 -XTypeFamilies -XGADTs -XRankNTypes -XBangPatterns -XMagicHash -XUnboxedTuples -XTypeOperators -O2 -O -dcore-lint -fno-warn-amp -fno-warn-deprecated-flags -Wwarn -no-user-package-db -rtsopts -odir libraries/dph/dph-lifted-copy/dist-install/build -hidir libraries/dph/dph-lifted-copy/dist-install/build -stubdir libraries/dph/dph-lifted-copy/dist-install/build -dep-makefile libraries/dph/dph-lifted-copy/dist-install/build/.depend-v-dyn.haskell.tmp -dep-suffix "" -dep-suffix "dyn_" -include-pkg-deps libraries/dph/dph-lifted-copy/./Data/Array/Parallel.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Lifted.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Lifted/Closure.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Lifted/Combinators.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Lifted/PArray.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Lifted/Scalar.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Lifted/TH/Repr.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Lifted/Unboxed.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/PArr.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/PArray.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/PArray/Base.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/PArray/PData.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/PArray/PDataInstances.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/PArray/PRepr.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/PArray/PReprInstances.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/PArray/Scalar.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/PArray/ScalarInstances.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/PArray/Types.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Prelude.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Prelude/Base.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Prelude/Bool.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Prelude/Double.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Prelude/Float.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Prelude/Int.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Prelude/Tuple.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Prelude/Word8.hs libraries/dph/dph-lifted-copy/./Data/Array/Parallel/Prim.hs ghc-stage1: libraries/dph/dph-lifted-copy/Data/Array/Parallel/Lifted/TH/Repr.hs:1:14-28: Template Haskell requires GHC with interpreter support Perhaps you are using a stage-1 compiler? Microsoft Research Limited (company number 03369488) is registered in England and Wales Registered office is at 21 Station Road, Cambridge, CB1 2FB

5 4

Re: [commit: packages/integer-gmp] master: Clean-up Cmm of import/export primitives (dfd65a2)
by Gabor Greif 05 Nov '13

05 Nov '13

I suppose the integer-simple library also needs the <new-primops> treatment. Many embedded platforms won't have GMP. Should we just provide 'error "unimplemented"' stubs? Cheers, Gabor On 11/5/13, git(a)git.haskell.org <git(a)git.haskell.org> wrote: > Repository : ssh://git@git.haskell.org/integer-gmp > > On branch : master > Link : > http://git.haskell.org/packages/integer-gmp.git/commitdiff/dfd65a28de3fe609… > >>--------------------------------------------------------------- > > commit dfd65a28de3fe6093d7e39fab6de960408abeb7e > Author: Herbert Valerio Riedel <hvr(a)gnu.org> > Date: Tue Nov 5 21:26:17 2013 +0100 > > Clean-up Cmm of import/export primitives > > This is a follow-up to e94799c9 fixing the Cmm implementation of the > primops based on suggestions by Duncan Coutts. > > Signed-off-by: Herbert Valerio Riedel <hvr(a)gnu.org> > > >>--------------------------------------------------------------- > > dfd65a28de3fe6093d7e39fab6de960408abeb7e > cbits/gmp-wrappers.cmm | 33 +++++++++++++++++---------------- > 1 file changed, 17 insertions(+), 16 deletions(-) > > diff --git a/cbits/gmp-wrappers.cmm b/cbits/gmp-wrappers.cmm > index 186cfad..0da3db8 100644 > --- a/cbits/gmp-wrappers.cmm > +++ b/cbits/gmp-wrappers.cmm > @@ -71,47 +71,48 @@ import "integer-gmp" integer_cbits_decodeDouble; > > integer_cmm_importIntegerzh (P_ ba, W_ of, W_ sz, W_ e) > { > - P_ p; > - W_ mp_result1; > + W_ src_ptr; > + W_ mp_result; > > again: > STK_CHK_GEN_N (SIZEOF_MP_INT); > MAYBE_GC(again); > > - p = ba + SIZEOF_StgArrWords + of; > + mp_result = Sp - SIZEOF_MP_INT; > > - mp_result1 = Sp - SIZEOF_MP_INT; > + src_ptr = BYTE_ARR_CTS(ba) + of; > > - ccall __gmpz_init(mp_result1 "ptr"); > - ccall __gmpz_import(mp_result1 "ptr", sz, W_TO_INT(e), W_TO_INT(1), > W_TO_INT(0), 0, p "ptr"); > + ccall __gmpz_init(mp_result "ptr"); > + ccall __gmpz_import(mp_result "ptr", sz, W_TO_INT(e), W_TO_INT(1), > W_TO_INT(0), 0, src_ptr "ptr"); > > - return(TO_W_(MP_INT__mp_size(mp_result1)), > - MP_INT__mp_d(mp_result1) - SIZEOF_StgArrWords); > + return(TO_W_(MP_INT__mp_size(mp_result)), > + MP_INT__mp_d(mp_result) - SIZEOF_StgArrWords); > } > > +/* :: Int# -> ByteArray# -> MutableByteArray# s -> Word# -> Int# -> State# > s -> (# State# s, Word# #) */ > integer_cmm_exportIntegerzh (W_ s1, P_ d1, P_ mba, W_ of, W_ e) > { > - P_ dst; > + W_ dst_ptr; > W_ mp_tmp; > - W_ cnt_result1; > + W_ cnt_result; > > again: > STK_CHK_GEN_N (SIZEOF_MP_INT + SIZEOF_W); > MAYBE_GC(again); > > - mp_tmp = Sp - SIZEOF_MP_INT; > + mp_tmp = Sp - SIZEOF_MP_INT; > MP_INT__mp_alloc(mp_tmp) = W_TO_INT(BYTE_ARR_WDS(d1)); > MP_INT__mp_size(mp_tmp) = (s1); > MP_INT__mp_d(mp_tmp) = BYTE_ARR_CTS(d1); > > - cnt_result1 = Sp - (SIZEOF_MP_INT + SIZEOF_W); > - W_[cnt_result1] = 0; > + cnt_result = Sp - (SIZEOF_MP_INT + SIZEOF_W); > + W_[cnt_result] = 0; > > - dst = mba + SIZEOF_StgArrWords + of; > + dst_ptr = BYTE_ARR_CTS(mba) + of; > > - ccall __gmpz_export(dst "ptr", cnt_result1 "ptr", W_TO_INT(e), > W_TO_INT(1), W_TO_INT(0), 0, mp_tmp "ptr"); > + ccall __gmpz_export(dst_ptr "ptr", cnt_result "ptr", W_TO_INT(e), > W_TO_INT(1), W_TO_INT(0), 0, mp_tmp "ptr"); > > - return (W_[cnt_result1]); > + return (W_[cnt_result]); > } > > integer_cmm_int2Integerzh (W_ val) > > _______________________________________________ > ghc-commits mailing list > ghc-commits(a)haskell.org > http://www.haskell.org/mailman/listinfo/ghc-commits >

2 2

Generics bugs
by Simon Peyton-Jones 05 Nov '13

05 Nov '13

Pedro, what's the status on #8479 #8468 #7346 ? Thanks, Simon Microsoft Research Limited (company number 03369488) is registered in England and Wales Registered office is at 21 Station Road, Cambridge, CB1 2FB

2 2