- ghc-devs - Haskell.org

Git confusion
by Ian Lynagh 04 Feb '13

04 Feb '13

Hi all, If I do git clone darcs.haskell.org:/srv/darcs/ghc.git cd ghc git show origin then the output is [...] Local branch configured for 'git pull': master merges with remote master [...] However, none of: git remote rm origin git remote add origin darcs.haskell.org:/srv/darcs/ghc.git git remote show origin git remote rm origin git remote add origin darcs.haskell.org:/srv/darcs/ghc.git -m master git remote show origin git remote rm origin git remote add origin darcs.haskell.org:/srv/darcs/ghc.git -t master git remote show origin git remote rm origin git remote add origin darcs.haskell.org:/srv/darcs/ghc.git -t master -m master git remote show origin have the "Local branch configured for 'git pull':" section. So having done "git remote rm origin", how do I configure the git pull branches again? Thanks Ian

3 3

links to diffs on ghc-commit emails seem to be broken
by Carter Schonwald 04 Feb '13

04 Feb '13

Hey all, links seem to not work, though they are included in the emails eg from the email [commit: ghc] master: Move AsmCodeGen.makeFarBranches to PPC.Instr (#709) (aa1d7d3) http://hackage.haskell.org/trac/ghc/changeset/aa1d7d35ac27625c9aa67fe71c186… *should* these be broken or is something needing to be fixed up? thanks -Carter

2 1

Re: [commit: ghc] master: Follow random submodule change (ca5d15a)
by Manuel M T Chakravarty 04 Feb '13

04 Feb '13

Ian, That still doesn't work: > Cloning into 'libraries/random'... > fatal: reference is not a tree: 2117e38729adaa4f465f3c5b7a8c5c4d77702d3f > Unable to checkout '2117e38729adaa4f465f3c5b7a8c5c4d77702d3f' in submodule path 'libraries/random' Also: http://darcs.haskell.org/ghcBuilder/builders/tn23/826/3.html Manuel Ian Lynagh <igloo(a)earth.li>: > Repository : ssh://darcs.haskell.org//srv/darcs/ghc > > On branch : master > > http://hackage.haskell.org/trac/ghc/changeset/ca5d15a61c45d6ea763872df46efe… > >> --------------------------------------------------------------- > > commit ca5d15a61c45d6ea763872df46efe5c7b33f8186 > Author: Ian Lynagh <igloo(a)earth.li> > Date: Sun Feb 3 16:29:17 2013 +0000 > > Follow random submodule change > >> --------------------------------------------------------------- > > libraries/random | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/libraries/random b/libraries/random > index 69bfde2..2117e38 160000 > --- a/libraries/random > +++ b/libraries/random > @@ -1 +1 @@ > -Subproject commit 69bfde219bab869729fdbe9c1496371f912bf41e > +Subproject commit 2117e38729adaa4f465f3c5b7a8c5c4d77702d3f > > > > _______________________________________________ > ghc-commits mailing list > ghc-commits(a)haskell.org > http://www.haskell.org/mailman/listinfo/ghc-commits

1 0

Validate failures on OS X/amd64
by Austin Seipp 31 Jan '13

31 Jan '13

Today, while running validate I encountered a handful of errors: VERALL SUMMARY for test run started at Thu Jan 31 15:58:15 CST 2013 3591 total tests, which gave rise to 14298 test cases, of which 4 caused framework failures 11044 were skipped 3156 expected passes 28 had missing libraries 56 expected failures 1 unexpected passes 13 unexpected failures Unexpected passes: ../../libraries/directory/tests T4113 (normal) Unexpected failures: ../../libraries/base/tests qsemn001 [bad exit code] (normal) codeGen/should_run cgrun071 [bad exit code] (normal) concurrent/should_run T5611 [bad stdout] (normal) perf/compiler T4801 [stat too good] (normal) perf/haddock haddock.base [stat not good enough] (normal) perf/should_run T5113 [stat not good enough] (normal) simplCore/should_compile T3717 [stderr mismatch] (optasm) simplCore/should_compile T4908 [stderr mismatch] (optasm) simplCore/should_compile T7360 [stderr mismatch] (optasm) simplCore/should_compile spec-inline [stderr mismatch] (optasm) typecheck/should_compile holes [stderr mismatch] (normal) typecheck/should_compile holes2 [stderr mismatch] (normal) typecheck/should_compile holes3 [stderr mismatch] (normal) Most of these are benign. All the 'simplCore' tests are failures that seem to be a result of the new demand analyzer, with tiny things like this: --- ./simplCore/should_compile/spec-inline.stderr 2013-01-18 12:55:59.000000000 -0600 +++ ./simplCore/should_compile/spec-inline.comp.stderr 2013-01-31 16:12:33.000000000 -0600 @@ -116,7 +116,7 @@ Roman.foo1 :: Data.Maybe.Maybe GHC.Types.Int [GblId, Caf=NoCafRefs, - Str=DmdType, + Str=DmdType m2, Unf=Unf{Src=<vanilla>, TopLvl=True, Arity=0, Value=True, ConLike=True, WorkFree=True, Expandable=True, Guidance=IF_ARGS [] 10 20}] and: --- ./simplCore/should_compile/T3717.stderr 2013-01-18 12:55:59.000000000 -0600 +++ ./simplCore/should_compile/T3717.comp.stderr 2013-01-31 16:12:33.000000000 -0600 @@ -22,13 +22,13 @@ ConLike=True, WorkFree=True, Expandable=True, Guidance=ALWAYS_IF(unsat_ok=True,boring_ok=False) Tmpl= \ (w [Occ=Once!] :: GHC.Types.Int) -> - case w of _ { GHC.Types.I# ww [Occ=Once] -> - case T3717.$wfoo ww of ww1 { __DEFAULT -> GHC.Types.I# ww1 } + case w of _ { GHC.Types.I# ww1 [Occ=Once] -> + case T3717.$wfoo ww1 of ww2 { __DEFAULT -> GHC.Types.I# ww2 } So these look like just minor output wibbles and otherwise fine changes. The 'holes' tests are failing because of an error message tweak. They are all of the form: holes3.hs:15:15: - Found hole `_' with type b0 + Found hole `_' with type: b0 so these are an easy fix too. Did someone forget to push testsuite patches? I can at least fix the TypeHoles tests since they're so trivial, and the demand analyzer changes look benign too, but I just wanted to make sure these are OK and I'm not stepping on toes if I were to do that. I'm investigating the others at the moment, these were just the easy ones I saw. -- Regards, Austin

2 1

Code review, new scheduler:
by Alexander Kjeldaas 31 Jan '13

31 Jan '13

Hi Edward, thanks for your work on the new scheduler! I have a done a super light-weight review. I think documenting the code a little more would help readability. http://hackage.haskell.org/trac/ghc/attachment/ticket/7606/0002-Stride-sche… TSO.h: Could you document ss_*? These are all important variables in the scheduler, and not documented. For example, the code in Threads.c for setting these is can act as some documentation, but something needs to be documented here. (Please don't point to a paper as primary documentation.) // 64-bit to prevent overflows; only ever accessed by the task which owns TSO. 170 StgWord64 ss_pass; 171 // These are bounded above by STRIDE1, which is less than max 32-bit word. 172 // You must take out the sched_lock to write to these; reads are OK 173 StgWord ss_tickets, ss_stride, ss_remain; 174 Schedule.c: 727 StgTSO *t; Rename to 'tso' to be descriptive. 744 // go through all of the TSOs in the run queue and decide where 745 // they should go 746 // XXX We can create the new heap more efficiently O(n) by just 747 // blitting them in and then re-heapifying 748 if (!emptyRunQueue(cap)) { 749 StgWord64 k; Ditto for 'k'. 1201 scheduleHandleThreadBlocked( Capability *cap, StgTSO *t ) Rename 't' to 'tso'. Schedule.h: 125 // oh no magic constant 126 #define STRIDE1 (1 << 20) Document STRIDE1 147 EXTERN_INLINE void 148 annulTSO(StgTSO *tso) { 149 // hack to make some invariants with regards to block_info and _link work 150 // this is called whereever we would have stepped all over the 151 // fields in the linked list implementation 152 tso->_link = END_TSO_QUEUE; 153 tso->block_info.closure = (StgClosure*)END_TSO_QUEUE; It would be great to have a pointer to the invariants, or the invariant(s) documented. 213 tso->ss_pass += tso->ss_stride; 214 StgWord64 r; 215 if (tso->ss_pass <= cap->ss_pass) { 216 // Thread is behind, it will get scheduled in front with no 217 // intervention (note that cap->ss_pass is probably nonsense, 218 // since it doesn't include *this* thread.) 219 r = tso->ss_pass; 220 } else if (tso->ss_pass - tso->ss_pass <= cap->ss_pass) { This expression looks weird/magic, tso->ss_pass - tso->ss_pass is 0. 221 // Thread is in good standing, schedule it in front 222 // (next iteration, they will not be in good standing if 223 // the global pass doesn't advance by much; that is, this 224 // thread managed to cut in front of other threads which 225 // are running behind.) 226 r = cap->ss_pass; 227 } else { 228 // Thread is not in good standing, schedule it later. 229 // Eventually, global pass will advance enough that the 230 // thread will be in good standing again, and can cut 231 // to the front. 232 r = tso->ss_pass; Threads.c: 361 if (migrating) { 362 joinRunQueue(cap,tso); 363 } else { 364 appendToRunQueue(cap,tso); 365 } Space after ',' Select.c: and rts/win32/AsyncIO.c 309 tso->ss_remain = 0; 310 joinRunQueue(&MainCapability,tso); Should this be an abstraction in itself? rts/Capability.h: 59 PQueue *run_pqueue; 60 61 // [SSS] Stride scheduling extensions. The Task with this 62 // Capability has exclusive access to this variable. 63 nat ss_pass; Document ss_pass. For example how it relates to capPassUpdate and pushOnRunQueue (the weird/magic I commented on above) and invariants. Alexander

2 1

Re: Code review, new scheduler:
by Alexander Kjeldaas 31 Jan '13

31 Jan '13

On Thu, Jan 31, 2013 at 11:06 AM, Edward Z. Yang <ezyang(a)mit.edu> wrote: > Excerpts from Alexander Kjeldaas's message of Thu Jan 31 01:57:17 -0800 > 2013: > > Found it, it's still there: > > > http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/kernel/timer.c?v… > > > > Note that these are *timers*. Timers are often set up and then canceled. > > As you see, the buckets are not sorted, timers are just put at the end > of > > a list, and the lists are even hard-coded so they are more general than > > log2 buckets. > > > > The point of this is to partially sort the timers as they get closer to > > expiration. Thus you don't pay the full sorting cost if the timer is > > modified before it expires. > > Ah, I see. GHC threads are very rarely canceled, so I don't think we would > see much benefit here. > > You are probably right. I can think of priority inversion with rapidly changing priorities as a possible use-case then. Anyways, keep up the good work! Alexander > Edward >

1 0

Re: Code review, new scheduler:
by Alexander Kjeldaas 31 Jan '13

31 Jan '13

Found it, it's still there: http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/kernel/timer.c?v… Note that these are *timers*. Timers are often set up and then canceled. As you see, the buckets are not sorted, timers are just put at the end of a list, and the lists are even hard-coded so they are more general than log2 buckets. The point of this is to partially sort the timers as they get closer to expiration. Thus you don't pay the full sorting cost if the timer is modified before it expires. Alexander On Thu, Jan 31, 2013 at 10:50 AM, Edward Z. Yang <ezyang(a)mit.edu> wrote: > Excerpts from Alexander Kjeldaas's message of Thu Jan 31 01:43:51 -0800 > 2013: > > What operations do you mostly do in the priority queue? > > Insert, removeEmpty, and the occasional delete. > > > I'd like to point you to the timer implementation that we did in the > linux > > kernel ages ago. It is basically a set of log2 slots where each slot > > contains timers for time [2^i..2^(i+1)>. The dlinked lists make it O(1) > to > > remove, change, or add a timer. The timer interrupt basically "eats" > > linked lists, and when empty "splats" the first free slot out over the > > upper slots (which is O(n) worst case). The structure is simple and > > optimized for doing a lot of mutation of the timers. It beats a heap > hands > > down in the kernel at least. > > Interesting! That sounds simple enough to merit an implementation. However, > I'm given to understand that the current kernel scheduler uses a red-black > tree? (I also have an implementation of one of those, though it's based > off of > jemalloc's implementation.) > > Edward >

1 0

Re: Code review, new scheduler:
by Alexander Kjeldaas 31 Jan '13

31 Jan '13

Good to know. What operations do you mostly do in the priority queue? I'd like to point you to the timer implementation that we did in the linux kernel ages ago. It is basically a set of log2 slots where each slot contains timers for time [2^i..2^(i+1)>. The dlinked lists make it O(1) to remove, change, or add a timer. The timer interrupt basically "eats" linked lists, and when empty "splats" the first free slot out over the upper slots (which is O(n) worst case). The structure is simple and optimized for doing a lot of mutation of the timers. It beats a heap hands down in the kernel at least. Alexander On Thu, Jan 31, 2013 at 10:34 AM, Edward Z. Yang <ezyang(a)mit.edu> wrote: > Hey Alex, > > Thanks for CR. At the current point in time, the overall system design > is very much in flux so a lot of this code may end up not existing in the > final revision. I will be sure to up the number of comments in the final > version though! > > Edward > > Excerpts from Alexander Kjeldaas's message of Thu Jan 31 01:30:39 -0800 > 2013: > > Hi Edward, thanks for your work on the new scheduler! > > > > I have a done a super light-weight review. I think documenting the code > a > > little more would help readability. > > > > > http://hackage.haskell.org/trac/ghc/attachment/ticket/7606/0002-Stride-sche… > > > > TSO.h: > > > > Could you document ss_*? These are all important variables in the > > scheduler, and not documented. For example, the code in Threads.c for > > setting these is can act as some documentation, but something needs to be > > documented here. > > > > (Please don't point to a paper as primary documentation.) > > > > // 64-bit to prevent overflows; only ever accessed by the task which > > owns TSO. > > 170 StgWord64 ss_pass; > > 171 // These are bounded above by STRIDE1, which is less than max > > 32-bit word. > > 172 // You must take out the sched_lock to write to these; reads are > > OK > > 173 StgWord ss_tickets, ss_stride, ss_remain; > > 174 > > > > Schedule.c: > > > > 727 StgTSO *t; > > > > Rename to 'tso' to be descriptive. > > > > 744 // go through all of the TSOs in the run queue and decide > > where > > 745 // they should go > > 746 // XXX We can create the new heap more efficiently O(n) by > > just > > 747 // blitting them in and then re-heapifying > > 748 if (!emptyRunQueue(cap)) { > > 749 StgWord64 k; > > > > Ditto for 'k'. > > > > 1201 scheduleHandleThreadBlocked( Capability *cap, StgTSO *t ) > > > > Rename 't' to 'tso'. > > > > Schedule.h: > > > > 125 // oh no magic constant > > 126 #define STRIDE1 (1 << 20) > > > > Document STRIDE1 > > > > 147 EXTERN_INLINE void > > 148 annulTSO(StgTSO *tso) { > > 149 // hack to make some invariants with regards to block_info and > > _link work > > 150 // this is called whereever we would have stepped all over the > > 151 // fields in the linked list implementation > > 152 tso->_link = END_TSO_QUEUE; > > 153 tso->block_info.closure = (StgClosure*)END_TSO_QUEUE; > > > > It would be great to have a pointer to the invariants, or the > invariant(s) > > documented. > > > > 213 tso->ss_pass += tso->ss_stride; > > 214 StgWord64 r; > > 215 if (tso->ss_pass <= cap->ss_pass) { > > 216 // Thread is behind, it will get scheduled in front with no > > 217 // intervention (note that cap->ss_pass is probably > nonsense, > > 218 // since it doesn't include *this* thread.) > > 219 r = tso->ss_pass; > > 220 } else if (tso->ss_pass - tso->ss_pass <= cap->ss_pass) { > > > > This expression looks weird/magic, tso->ss_pass - tso->ss_pass is 0. > > > > 221 // Thread is in good standing, schedule it in front > > 222 // (next iteration, they will not be in good standing if > > 223 // the global pass doesn't advance by much; that is, this > > 224 // thread managed to cut in front of other threads which > > 225 // are running behind.) > > 226 r = cap->ss_pass; > > 227 } else { > > 228 // Thread is not in good standing, schedule it later. > > 229 // Eventually, global pass will advance enough that the > > 230 // thread will be in good standing again, and can cut > > 231 // to the front. > > 232 r = tso->ss_pass; > > > > > > Threads.c: > > > > 361 if (migrating) { > > 362 joinRunQueue(cap,tso); > > 363 } else { > > 364 appendToRunQueue(cap,tso); > > 365 } > > > > Space after ',' > > > > Select.c: and rts/win32/AsyncIO.c > > > > 309 tso->ss_remain = 0; > > 310 joinRunQueue(&MainCapability,tso); > > > > Should this be an abstraction in itself? > > > > > > rts/Capability.h: > > > > 59 PQueue *run_pqueue; > > 60 > > 61 // [SSS] Stride scheduling extensions. The Task with this > > 62 // Capability has exclusive access to this variable. > > 63 nat ss_pass; > > > > Document ss_pass. For example how it relates to capPassUpdate and > > pushOnRunQueue (the weird/magic I commented on above) and invariants. > > > > > > Alexander >

1 0

RE: This week
by Simon Peyton-Jones 30 Jan '13

30 Jan '13

| The discussion about replacing abbot seems to be achieving consensus. | There's no urgency, but if it seems to taper out again then I'll try | prodding the committee to put the plan into action. What *is* the consensus? We should get on with this. It bit several times recently, and we don't want to wait until it becomes really serious! (And it looks bad.) Simon

7 10

Today's validate failures
by Simon Marlow 30 Jan '13

30 Jan '13

I have several validate failures today: Unexpected failures: cabal 1750 [bad stderr] (normal) cabal shadow [bad stderr] (normal) perf/haddock haddock.base [stat not good enough] (normal) rename/should_fail rnfail055 [stderr mismatch] (normal) simplCore/should_compile T4201 [bad stdout] (normal) The cabal ones are mysterious, but I definitely didn't see them yesterday, and I'm certain they're not caused by anything I've done. The only patch in my tree is one that puts back the vector/primitive submodules that I accidentally deleted. Actual stderr output differs from expected: --- ./rename/should_fail/rnfail055.stderr 2013-01-07 08:47:29.856696930 +0000 +++ ./rename/should_fail/rnfail055.comp.stderr 2013-01-18 09:59:24.531420493 +0000 @@ -52,6 +52,19 @@ RnFail055.hs-boot:17:12: T3' is exported by the hs-boot file, but not exported by the module +RnFail055.hs-boot:19:6: + Type constructor `T4' has conflicting definitions in the module and its hs-boot file + Main module: data T4 a + No C type associated + RecFlag Recursive + = T4 :: forall a. (forall b. a -> b) -> T4 a Stricts: _ + FamilyInstance: none + Boot file: data T4 b + No C type associated + RecFlag NonRecursive + = T4 :: forall b. (forall a. b -> a) -> T4 b Stricts: _ + FamilyInstance: none + RnFail055.hs-boot:21:6: Type constructor `T5' has conflicting definitions in the module and its hs-boot file Main module: data T5 a *** unexpected failure for rnfail055(normal) Actual stdout output differs from expected: --- ./simplCore/should_compile/T4201.stdout 2013-01-07 08:47:29.856696930 +0000 +++ ./simplCore/should_compile/T4201.run.stdout 2013-01-18 09:57:59.015418374 +0000 @@ -1,3 +1,3 @@ - Unfolding: (Eta.bof - `cast` - (Sym (Eta.NTCo:Foo[0]) -> Refl Eta.T)) -} + {- Arity: 1, HasNoCafRefs, Strictness: <S,U>m, + Unfolding: InlineRule (0, True, True) + Eta.bof `cast` (Sym (Eta.NTCo:Foo[0]) -> Refl Eta.T) -} *** unexpected failure for T4201(normal) Actual stderr output differs from expected: --- ./rename/should_fail/rnfail055.stderr 2013-01-07 08:47:29.856696930 +0000 +++ ./rename/should_fail/rnfail055.comp.stderr 2013-01-18 09:59:24.531420493 +0000 @@ -52,6 +52,19 @@ RnFail055.hs-boot:17:12: T3' is exported by the hs-boot file, but not exported by the module +RnFail055.hs-boot:19:6: + Type constructor `T4' has conflicting definitions in the module and its hs-boot file + Main module: data T4 a + No C type associated + RecFlag Recursive + = T4 :: forall a. (forall b. a -> b) -> T4 a Stricts: _ + FamilyInstance: none + Boot file: data T4 b + No C type associated + RecFlag NonRecursive + = T4 :: forall b. (forall a. b -> a) -> T4 b Stricts: _ + FamilyInstance: none + RnFail055.hs-boot:21:6: Type constructor `T5' has conflicting definitions in the module and its hs-boot file Main module: data T5 a *** unexpected failure for rnfail055(normal)

4 10