- ghc-devs - Haskell.org

simd branch ready for review
by Geoffrey Mainland 06 Feb '13

06 Feb '13

Hi Simon, I've pushed my simd branch to darcs.haskell.org. Everything has been rebased against HEAD. Simon PJ and I looked over the changes together already, but I wanted to give you (and everyone on ghc-devs) the opportunity to look things over before I merge to HEAD. Simon PJ and I came up with a few questions/notes for you, but hopefully nothing that should delay a merge. * Win32 issues Modern 32-bit x86 *NIX systems align the stack to 16-bytes, but Win32 aligns only to 4-bytes. LLVM does not assume 16-byte stack alignment. Instead, on platforms where 16-byte stack alignment is not guaranteed, it 1) always outputs a function prologue that 2) aligns the stack to a 16-byte boundary with an "and" instructions, and it also 3) disables tail calls. Because LLVM aligns the stack for a function that has SSE register spills, it also generates movaps instructions (aligned SSE moves) for the spills. This makes SSE support on Win32 difficult, and in my opinion not worth worrying about. The alternative is to 1) patch LLVM to disable the stack-alignment code so that we recover the ability to use tail calls and so that ebp scribbled over by the prologue and 2) patch the mangler to rewrite LLVM's movaps (move aligned) instructions to movups (move unaligned) instructions. I have these patches, but they are not included in the simd branch. * How hard would it be to dump ArgRep for PrimRep? It looks straightforward. Is it worth doing? * How hard would it be to track bit width in PrimRep? I recall chatting with you once about adding explicit support for, e.g., 8- and 16-bit Word/Int primops instead of relying on narrowing. Since SIMD vectors need to know the exact bit-width of their elements, I've had to create a PrimElemRep data type in compiler/types/TyCon.lhs, but I'd really like to be able to re-use PrimRep instead. * If we replaced all old-style C-- code, could we get rid of the explicit STG registers completely? Simon PJ suggested that we use real machine registers directly, so, for example, GlobalReg's constructors would have FastString fields instead of Int fields. * Could we add a CmmType field to GlobalReg's constructors? You'll see that I added a new XmmReg constructor to GlobalReg, but because I don't know the type of an XmmReg, I have to bitcast everywhere in the generated LLVM code because LLVM wants to know not just that a value is a 16-byte vector, but that it is, e.g., a 16-byte vector containing 2 64-bit doubles. Having a CmmType attached to a GlobalReg---or pairing a GlobalReg with a CmmType when assigning registers---would let me avoid all these casts. Thanks! Geoff

8 27

Re: [commit: ghc] master: Rewrote vectorisation avoidance (based on the HS paper) (b77da25)
by David Terei 06 Feb '13

06 Feb '13

Hi Manuel, I recently committed a patch to document the -fvectorise and -favoid-vect flags. Would you mind taking a look at the documentation for these flags to ensure they are valid still with these recent changes? docs/userguide/using.xml is the file in question. Cheers, David On 5 February 2013 19:16, Manuel Chakravarty <chak(a)cse.unsw.edu.au> wrote: > Repository : ssh://darcs.haskell.org//srv/darcs/ghc > > On branch : master > > http://hackage.haskell.org/trac/ghc/changeset/b77da25ef0d95e776a43779bbb484… > >>--------------------------------------------------------------- > > commit b77da25ef0d95e776a43779bbb4843eb01d33552 > Author: Manuel M T Chakravarty <chak(a)cse.unsw.edu.au> > Date: Wed Dec 5 15:28:19 2012 +1100 > > Rewrote vectorisation avoidance (based on the HS paper) > > * Vectorisation avoidance is now the default > * Types and values from unvectorised modules are permitted in scalar code > * Simplified the VECTORISE pragmas (see http://hackage.haskell.org/trac/ghc/wiki/DataParallel/VectPragma for the spec) > * Vectorisation information is now included in the annotated Core AST > > compiler/coreSyn/CoreFVs.lhs | 11 +- > compiler/coreSyn/CoreSubst.lhs | 11 +- > compiler/coreSyn/CoreSyn.lhs | 4 +- > compiler/coreSyn/PprCore.lhs | 3 +- > compiler/deSugar/Desugar.lhs | 2 +- > compiler/hsSyn/HsDecls.lhs | 10 +- > compiler/iface/LoadIface.lhs | 14 +- > compiler/iface/MkIface.lhs | 18 +- > compiler/iface/TcIface.lhs | 30 +- > compiler/main/HscTypes.lhs | 54 +- > compiler/main/TidyPgm.lhs | 10 +- > compiler/parser/Parser.y.pp | 5 +- > compiler/rename/RnSource.lhs | 10 +- > compiler/typecheck/TcBinds.lhs | 9 +- > compiler/typecheck/TcHsSyn.lhs | 2 +- > compiler/vectorise/Vectorise.hs | 394 ++++------ > compiler/vectorise/Vectorise/Convert.hs | 18 +- > compiler/vectorise/Vectorise/Env.hs | 86 +-- > compiler/vectorise/Vectorise/Exp.hs | 1026 +++++++++++++------------ > compiler/vectorise/Vectorise/Monad.hs | 26 +- > compiler/vectorise/Vectorise/Monad/Global.hs | 60 +- > compiler/vectorise/Vectorise/Monad/InstEnv.hs | 13 +- > compiler/vectorise/Vectorise/Monad/Local.hs | 22 +- > compiler/vectorise/Vectorise/Type/Classify.hs | 42 +- > compiler/vectorise/Vectorise/Type/Env.hs | 211 +++--- > compiler/vectorise/Vectorise/Type/Type.hs | 13 +- > compiler/vectorise/Vectorise/Utils.hs | 25 +- > 27 files changed, 1066 insertions(+), 1063 deletions(-) > > > Diff suppressed because of size. To see it, use: > > git show b77da25ef0d95e776a43779bbb4843eb01d33552 > > _______________________________________________ > ghc-commits mailing list > ghc-commits(a)haskell.org > http://www.haskell.org/mailman/listinfo/ghc-commits

1 0

Re: [commit: nofib] master: Note why all shootout benchmarks couldn't be added (c82455f)
by Ian Lynagh 06 Feb '13

06 Feb '13

Hi Johan, On Tue, Feb 05, 2013 at 03:19:59PM -0800, Johan Tibell wrote: > > + * The k-nucleotide benchmark wasn't included as it segfaults on my > + 64-bit OS X 10.8.2 machine. That sounds more serious than a performance regression! Or is it a bug in the benchmark? Thanks Ian

2 1

Re: [commit: ghc] master: Add the new random commit again (6a46b46)
by Gabor Greif 05 Feb '13

05 Feb '13

Hi Ian, since you have pinned a commit that is on a branch, people whose clones do not track that branch will never see the commit: $ ( cd libraries/random ; git remote show; git remote show origin ) origin * remote origin Fetch URL: https://github.com/ghc/packages-random.git Push URL: https://github.com/ghc/packages-random.git HEAD branch: master Remote branch: master tracked Local ref configured for 'git push': master pushes to master (up to date) This may be the problem what we see. Possibly the 'sync-all' script could be amended, so that your 'ghc-head' branch is added to the remote? Cheers, Gabor On 2/5/13, Ian Lynagh <igloo(a)earth.li> wrote: > Repository : ssh://darcs.haskell.org//srv/darcs/ghc > > On branch : master > > http://hackage.haskell.org/trac/ghc/changeset/6a46b4627d4f3f81d65bf6b5b5c0b… > >>--------------------------------------------------------------- > > commit 6a46b4627d4f3f81d65bf6b5b5c0bcf86607c463 > Author: Ian Lynagh <ian(a)well-typed.com> > Date: Tue Feb 5 14:08:05 2013 +0000 > > Add the new random commit again > >>--------------------------------------------------------------- > > libraries/random | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/libraries/random b/libraries/random > index 69bfde2..2117e38 160000 > --- a/libraries/random > +++ b/libraries/random > @@ -1 +1 @@ > -Subproject commit 69bfde219bab869729fdbe9c1496371f912bf41e > +Subproject commit 2117e38729adaa4f465f3c5b7a8c5c4d77702d3f > > > > _______________________________________________ > ghc-commits mailing list > ghc-commits(a)haskell.org > http://www.haskell.org/mailman/listinfo/ghc-commits >

2 2

Git confusion
by Ian Lynagh 04 Feb '13

04 Feb '13

Hi all, If I do git clone darcs.haskell.org:/srv/darcs/ghc.git cd ghc git show origin then the output is [...] Local branch configured for 'git pull': master merges with remote master [...] However, none of: git remote rm origin git remote add origin darcs.haskell.org:/srv/darcs/ghc.git git remote show origin git remote rm origin git remote add origin darcs.haskell.org:/srv/darcs/ghc.git -m master git remote show origin git remote rm origin git remote add origin darcs.haskell.org:/srv/darcs/ghc.git -t master git remote show origin git remote rm origin git remote add origin darcs.haskell.org:/srv/darcs/ghc.git -t master -m master git remote show origin have the "Local branch configured for 'git pull':" section. So having done "git remote rm origin", how do I configure the git pull branches again? Thanks Ian

3 3

links to diffs on ghc-commit emails seem to be broken
by Carter Schonwald 04 Feb '13

04 Feb '13

Hey all, links seem to not work, though they are included in the emails eg from the email [commit: ghc] master: Move AsmCodeGen.makeFarBranches to PPC.Instr (#709) (aa1d7d3) http://hackage.haskell.org/trac/ghc/changeset/aa1d7d35ac27625c9aa67fe71c186… *should* these be broken or is something needing to be fixed up? thanks -Carter

2 1

Re: [commit: ghc] master: Follow random submodule change (ca5d15a)
by Manuel M T Chakravarty 04 Feb '13

04 Feb '13

Ian, That still doesn't work: > Cloning into 'libraries/random'... > fatal: reference is not a tree: 2117e38729adaa4f465f3c5b7a8c5c4d77702d3f > Unable to checkout '2117e38729adaa4f465f3c5b7a8c5c4d77702d3f' in submodule path 'libraries/random' Also: http://darcs.haskell.org/ghcBuilder/builders/tn23/826/3.html Manuel Ian Lynagh <igloo(a)earth.li>: > Repository : ssh://darcs.haskell.org//srv/darcs/ghc > > On branch : master > > http://hackage.haskell.org/trac/ghc/changeset/ca5d15a61c45d6ea763872df46efe… > >> --------------------------------------------------------------- > > commit ca5d15a61c45d6ea763872df46efe5c7b33f8186 > Author: Ian Lynagh <igloo(a)earth.li> > Date: Sun Feb 3 16:29:17 2013 +0000 > > Follow random submodule change > >> --------------------------------------------------------------- > > libraries/random | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/libraries/random b/libraries/random > index 69bfde2..2117e38 160000 > --- a/libraries/random > +++ b/libraries/random > @@ -1 +1 @@ > -Subproject commit 69bfde219bab869729fdbe9c1496371f912bf41e > +Subproject commit 2117e38729adaa4f465f3c5b7a8c5c4d77702d3f > > > > _______________________________________________ > ghc-commits mailing list > ghc-commits(a)haskell.org > http://www.haskell.org/mailman/listinfo/ghc-commits

1 0

Validate failures on OS X/amd64
by Austin Seipp 31 Jan '13

31 Jan '13

Today, while running validate I encountered a handful of errors: VERALL SUMMARY for test run started at Thu Jan 31 15:58:15 CST 2013 3591 total tests, which gave rise to 14298 test cases, of which 4 caused framework failures 11044 were skipped 3156 expected passes 28 had missing libraries 56 expected failures 1 unexpected passes 13 unexpected failures Unexpected passes: ../../libraries/directory/tests T4113 (normal) Unexpected failures: ../../libraries/base/tests qsemn001 [bad exit code] (normal) codeGen/should_run cgrun071 [bad exit code] (normal) concurrent/should_run T5611 [bad stdout] (normal) perf/compiler T4801 [stat too good] (normal) perf/haddock haddock.base [stat not good enough] (normal) perf/should_run T5113 [stat not good enough] (normal) simplCore/should_compile T3717 [stderr mismatch] (optasm) simplCore/should_compile T4908 [stderr mismatch] (optasm) simplCore/should_compile T7360 [stderr mismatch] (optasm) simplCore/should_compile spec-inline [stderr mismatch] (optasm) typecheck/should_compile holes [stderr mismatch] (normal) typecheck/should_compile holes2 [stderr mismatch] (normal) typecheck/should_compile holes3 [stderr mismatch] (normal) Most of these are benign. All the 'simplCore' tests are failures that seem to be a result of the new demand analyzer, with tiny things like this: --- ./simplCore/should_compile/spec-inline.stderr 2013-01-18 12:55:59.000000000 -0600 +++ ./simplCore/should_compile/spec-inline.comp.stderr 2013-01-31 16:12:33.000000000 -0600 @@ -116,7 +116,7 @@ Roman.foo1 :: Data.Maybe.Maybe GHC.Types.Int [GblId, Caf=NoCafRefs, - Str=DmdType, + Str=DmdType m2, Unf=Unf{Src=<vanilla>, TopLvl=True, Arity=0, Value=True, ConLike=True, WorkFree=True, Expandable=True, Guidance=IF_ARGS [] 10 20}] and: --- ./simplCore/should_compile/T3717.stderr 2013-01-18 12:55:59.000000000 -0600 +++ ./simplCore/should_compile/T3717.comp.stderr 2013-01-31 16:12:33.000000000 -0600 @@ -22,13 +22,13 @@ ConLike=True, WorkFree=True, Expandable=True, Guidance=ALWAYS_IF(unsat_ok=True,boring_ok=False) Tmpl= \ (w [Occ=Once!] :: GHC.Types.Int) -> - case w of _ { GHC.Types.I# ww [Occ=Once] -> - case T3717.$wfoo ww of ww1 { __DEFAULT -> GHC.Types.I# ww1 } + case w of _ { GHC.Types.I# ww1 [Occ=Once] -> + case T3717.$wfoo ww1 of ww2 { __DEFAULT -> GHC.Types.I# ww2 } So these look like just minor output wibbles and otherwise fine changes. The 'holes' tests are failing because of an error message tweak. They are all of the form: holes3.hs:15:15: - Found hole `_' with type b0 + Found hole `_' with type: b0 so these are an easy fix too. Did someone forget to push testsuite patches? I can at least fix the TypeHoles tests since they're so trivial, and the demand analyzer changes look benign too, but I just wanted to make sure these are OK and I'm not stepping on toes if I were to do that. I'm investigating the others at the moment, these were just the easy ones I saw. -- Regards, Austin

2 1

Code review, new scheduler:
by Alexander Kjeldaas 31 Jan '13

31 Jan '13

Hi Edward, thanks for your work on the new scheduler! I have a done a super light-weight review. I think documenting the code a little more would help readability. http://hackage.haskell.org/trac/ghc/attachment/ticket/7606/0002-Stride-sche… TSO.h: Could you document ss_*? These are all important variables in the scheduler, and not documented. For example, the code in Threads.c for setting these is can act as some documentation, but something needs to be documented here. (Please don't point to a paper as primary documentation.) // 64-bit to prevent overflows; only ever accessed by the task which owns TSO. 170 StgWord64 ss_pass; 171 // These are bounded above by STRIDE1, which is less than max 32-bit word. 172 // You must take out the sched_lock to write to these; reads are OK 173 StgWord ss_tickets, ss_stride, ss_remain; 174 Schedule.c: 727 StgTSO *t; Rename to 'tso' to be descriptive. 744 // go through all of the TSOs in the run queue and decide where 745 // they should go 746 // XXX We can create the new heap more efficiently O(n) by just 747 // blitting them in and then re-heapifying 748 if (!emptyRunQueue(cap)) { 749 StgWord64 k; Ditto for 'k'. 1201 scheduleHandleThreadBlocked( Capability *cap, StgTSO *t ) Rename 't' to 'tso'. Schedule.h: 125 // oh no magic constant 126 #define STRIDE1 (1 << 20) Document STRIDE1 147 EXTERN_INLINE void 148 annulTSO(StgTSO *tso) { 149 // hack to make some invariants with regards to block_info and _link work 150 // this is called whereever we would have stepped all over the 151 // fields in the linked list implementation 152 tso->_link = END_TSO_QUEUE; 153 tso->block_info.closure = (StgClosure*)END_TSO_QUEUE; It would be great to have a pointer to the invariants, or the invariant(s) documented. 213 tso->ss_pass += tso->ss_stride; 214 StgWord64 r; 215 if (tso->ss_pass <= cap->ss_pass) { 216 // Thread is behind, it will get scheduled in front with no 217 // intervention (note that cap->ss_pass is probably nonsense, 218 // since it doesn't include *this* thread.) 219 r = tso->ss_pass; 220 } else if (tso->ss_pass - tso->ss_pass <= cap->ss_pass) { This expression looks weird/magic, tso->ss_pass - tso->ss_pass is 0. 221 // Thread is in good standing, schedule it in front 222 // (next iteration, they will not be in good standing if 223 // the global pass doesn't advance by much; that is, this 224 // thread managed to cut in front of other threads which 225 // are running behind.) 226 r = cap->ss_pass; 227 } else { 228 // Thread is not in good standing, schedule it later. 229 // Eventually, global pass will advance enough that the 230 // thread will be in good standing again, and can cut 231 // to the front. 232 r = tso->ss_pass; Threads.c: 361 if (migrating) { 362 joinRunQueue(cap,tso); 363 } else { 364 appendToRunQueue(cap,tso); 365 } Space after ',' Select.c: and rts/win32/AsyncIO.c 309 tso->ss_remain = 0; 310 joinRunQueue(&MainCapability,tso); Should this be an abstraction in itself? rts/Capability.h: 59 PQueue *run_pqueue; 60 61 // [SSS] Stride scheduling extensions. The Task with this 62 // Capability has exclusive access to this variable. 63 nat ss_pass; Document ss_pass. For example how it relates to capPassUpdate and pushOnRunQueue (the weird/magic I commented on above) and invariants. Alexander

2 1

Re: Code review, new scheduler:
by Alexander Kjeldaas 31 Jan '13

31 Jan '13

On Thu, Jan 31, 2013 at 11:06 AM, Edward Z. Yang <ezyang(a)mit.edu> wrote: > Excerpts from Alexander Kjeldaas's message of Thu Jan 31 01:57:17 -0800 > 2013: > > Found it, it's still there: > > > http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/kernel/timer.c?v… > > > > Note that these are *timers*. Timers are often set up and then canceled. > > As you see, the buckets are not sorted, timers are just put at the end > of > > a list, and the lists are even hard-coded so they are more general than > > log2 buckets. > > > > The point of this is to partially sort the timers as they get closer to > > expiration. Thus you don't pay the full sorting cost if the timer is > > modified before it expires. > > Ah, I see. GHC threads are very rarely canceled, so I don't think we would > see much benefit here. > > You are probably right. I can think of priority inversion with rapidly changing priorities as a possible use-case then. Anyways, keep up the good work! Alexander > Edward >

1 0