January 2013 - ghc-devs

Re: Code review, new scheduler:
by Alexander Kjeldaas 31 Jan '13

31 Jan '13

Good to know. What operations do you mostly do in the priority queue? I'd like to point you to the timer implementation that we did in the linux kernel ages ago. It is basically a set of log2 slots where each slot contains timers for time [2^i..2^(i+1)>. The dlinked lists make it O(1) to remove, change, or add a timer. The timer interrupt basically "eats" linked lists, and when empty "splats" the first free slot out over the upper slots (which is O(n) worst case). The structure is simple and optimized for doing a lot of mutation of the timers. It beats a heap hands down in the kernel at least. Alexander On Thu, Jan 31, 2013 at 10:34 AM, Edward Z. Yang <ezyang(a)mit.edu> wrote: > Hey Alex, > > Thanks for CR. At the current point in time, the overall system design > is very much in flux so a lot of this code may end up not existing in the > final revision. I will be sure to up the number of comments in the final > version though! > > Edward > > Excerpts from Alexander Kjeldaas's message of Thu Jan 31 01:30:39 -0800 > 2013: > > Hi Edward, thanks for your work on the new scheduler! > > > > I have a done a super light-weight review. I think documenting the code > a > > little more would help readability. > > > > > http://hackage.haskell.org/trac/ghc/attachment/ticket/7606/0002-Stride-sche… > > > > TSO.h: > > > > Could you document ss_*? These are all important variables in the > > scheduler, and not documented. For example, the code in Threads.c for > > setting these is can act as some documentation, but something needs to be > > documented here. > > > > (Please don't point to a paper as primary documentation.) > > > > // 64-bit to prevent overflows; only ever accessed by the task which > > owns TSO. > > 170 StgWord64 ss_pass; > > 171 // These are bounded above by STRIDE1, which is less than max > > 32-bit word. > > 172 // You must take out the sched_lock to write to these; reads are > > OK > > 173 StgWord ss_tickets, ss_stride, ss_remain; > > 174 > > > > Schedule.c: > > > > 727 StgTSO *t; > > > > Rename to 'tso' to be descriptive. > > > > 744 // go through all of the TSOs in the run queue and decide > > where > > 745 // they should go > > 746 // XXX We can create the new heap more efficiently O(n) by > > just > > 747 // blitting them in and then re-heapifying > > 748 if (!emptyRunQueue(cap)) { > > 749 StgWord64 k; > > > > Ditto for 'k'. > > > > 1201 scheduleHandleThreadBlocked( Capability *cap, StgTSO *t ) > > > > Rename 't' to 'tso'. > > > > Schedule.h: > > > > 125 // oh no magic constant > > 126 #define STRIDE1 (1 << 20) > > > > Document STRIDE1 > > > > 147 EXTERN_INLINE void > > 148 annulTSO(StgTSO *tso) { > > 149 // hack to make some invariants with regards to block_info and > > _link work > > 150 // this is called whereever we would have stepped all over the > > 151 // fields in the linked list implementation > > 152 tso->_link = END_TSO_QUEUE; > > 153 tso->block_info.closure = (StgClosure*)END_TSO_QUEUE; > > > > It would be great to have a pointer to the invariants, or the > invariant(s) > > documented. > > > > 213 tso->ss_pass += tso->ss_stride; > > 214 StgWord64 r; > > 215 if (tso->ss_pass <= cap->ss_pass) { > > 216 // Thread is behind, it will get scheduled in front with no > > 217 // intervention (note that cap->ss_pass is probably > nonsense, > > 218 // since it doesn't include *this* thread.) > > 219 r = tso->ss_pass; > > 220 } else if (tso->ss_pass - tso->ss_pass <= cap->ss_pass) { > > > > This expression looks weird/magic, tso->ss_pass - tso->ss_pass is 0. > > > > 221 // Thread is in good standing, schedule it in front > > 222 // (next iteration, they will not be in good standing if > > 223 // the global pass doesn't advance by much; that is, this > > 224 // thread managed to cut in front of other threads which > > 225 // are running behind.) > > 226 r = cap->ss_pass; > > 227 } else { > > 228 // Thread is not in good standing, schedule it later. > > 229 // Eventually, global pass will advance enough that the > > 230 // thread will be in good standing again, and can cut > > 231 // to the front. > > 232 r = tso->ss_pass; > > > > > > Threads.c: > > > > 361 if (migrating) { > > 362 joinRunQueue(cap,tso); > > 363 } else { > > 364 appendToRunQueue(cap,tso); > > 365 } > > > > Space after ',' > > > > Select.c: and rts/win32/AsyncIO.c > > > > 309 tso->ss_remain = 0; > > 310 joinRunQueue(&MainCapability,tso); > > > > Should this be an abstraction in itself? > > > > > > rts/Capability.h: > > > > 59 PQueue *run_pqueue; > > 60 > > 61 // [SSS] Stride scheduling extensions. The Task with this > > 62 // Capability has exclusive access to this variable. > > 63 nat ss_pass; > > > > Document ss_pass. For example how it relates to capPassUpdate and > > pushOnRunQueue (the weird/magic I commented on above) and invariants. > > > > > > Alexander >

1 0

RE: This week
by Simon Peyton-Jones 30 Jan '13

30 Jan '13

| The discussion about replacing abbot seems to be achieving consensus. | There's no urgency, but if it seems to taper out again then I'll try | prodding the committee to put the plan into action. What *is* the consensus? We should get on with this. It bit several times recently, and we don't want to wait until it becomes really serious! (And it looks bad.) Simon

7 10

Today's validate failures
by Simon Marlow 30 Jan '13

30 Jan '13

I have several validate failures today: Unexpected failures: cabal 1750 [bad stderr] (normal) cabal shadow [bad stderr] (normal) perf/haddock haddock.base [stat not good enough] (normal) rename/should_fail rnfail055 [stderr mismatch] (normal) simplCore/should_compile T4201 [bad stdout] (normal) The cabal ones are mysterious, but I definitely didn't see them yesterday, and I'm certain they're not caused by anything I've done. The only patch in my tree is one that puts back the vector/primitive submodules that I accidentally deleted. Actual stderr output differs from expected: --- ./rename/should_fail/rnfail055.stderr 2013-01-07 08:47:29.856696930 +0000 +++ ./rename/should_fail/rnfail055.comp.stderr 2013-01-18 09:59:24.531420493 +0000 @@ -52,6 +52,19 @@ RnFail055.hs-boot:17:12: T3' is exported by the hs-boot file, but not exported by the module +RnFail055.hs-boot:19:6: + Type constructor `T4' has conflicting definitions in the module and its hs-boot file + Main module: data T4 a + No C type associated + RecFlag Recursive + = T4 :: forall a. (forall b. a -> b) -> T4 a Stricts: _ + FamilyInstance: none + Boot file: data T4 b + No C type associated + RecFlag NonRecursive + = T4 :: forall b. (forall a. b -> a) -> T4 b Stricts: _ + FamilyInstance: none + RnFail055.hs-boot:21:6: Type constructor `T5' has conflicting definitions in the module and its hs-boot file Main module: data T5 a *** unexpected failure for rnfail055(normal) Actual stdout output differs from expected: --- ./simplCore/should_compile/T4201.stdout 2013-01-07 08:47:29.856696930 +0000 +++ ./simplCore/should_compile/T4201.run.stdout 2013-01-18 09:57:59.015418374 +0000 @@ -1,3 +1,3 @@ - Unfolding: (Eta.bof - `cast` - (Sym (Eta.NTCo:Foo[0]) -> Refl Eta.T)) -} + {- Arity: 1, HasNoCafRefs, Strictness: <S,U>m, + Unfolding: InlineRule (0, True, True) + Eta.bof `cast` (Sym (Eta.NTCo:Foo[0]) -> Refl Eta.T) -} *** unexpected failure for T4201(normal) Actual stderr output differs from expected: --- ./rename/should_fail/rnfail055.stderr 2013-01-07 08:47:29.856696930 +0000 +++ ./rename/should_fail/rnfail055.comp.stderr 2013-01-18 09:59:24.531420493 +0000 @@ -52,6 +52,19 @@ RnFail055.hs-boot:17:12: T3' is exported by the hs-boot file, but not exported by the module +RnFail055.hs-boot:19:6: + Type constructor `T4' has conflicting definitions in the module and its hs-boot file + Main module: data T4 a + No C type associated + RecFlag Recursive + = T4 :: forall a. (forall b. a -> b) -> T4 a Stricts: _ + FamilyInstance: none + Boot file: data T4 b + No C type associated + RecFlag NonRecursive + = T4 :: forall b. (forall a. b -> a) -> T4 b Stricts: _ + FamilyInstance: none + RnFail055.hs-boot:21:6: Type constructor `T5' has conflicting definitions in the module and its hs-boot file Main module: data T5 a *** unexpected failure for rnfail055(normal)

4 10

[PATCH] wrap functions in base for android
by Nathan Hüsken 29 Jan '13

29 Jan '13

Hey, When compiling the cross compiler for android, it fails because of undefined symbols to mkfifo, tcgetattr and tcsetattr. This seems to be because the android ndk defines them as inline. This patch wraps them (when compiling for android) in __mkfifo __tcgetattr and __tcsetattr.

2 2

Stage1Only=YES does not seem to work on current master
by Sergei Trofimovich 29 Jan '13

29 Jan '13

simple build.mk: SRC_CC_OPTS+=-O2 -pipe BUILD_DOCBOOK_PDF = NO BUILD_DOCBOOK_PS = NO BUILD_DOCBOOK_HTML = NO HADDOCK_DOCS = NO SRC_HC_OPTS+=-w Stage1Only=YES GhcUnregisterised=YES SplitObjs=NO Leads to failures of missing stage2: "inplace/lib/unlit" utils/count_lines/count_lines.lprl utils/count_lines/dist/count_lines.prl "inplace/bin/ghc-stage2" -optc-O2 -optc-pipe -optc-Ilibraries/primitive/cbits -optc-I'/var/tmp/portage/cross-ia64-unknown-linux-gnu/ghc-9999/work/ghc-9999/libraries/base/include' -optc-I'/var/tmp/portage/cross-ia64-unknown-linux-gnu/ghc-9999/work/ghc-9999/rts/dist/build' -optc-I'/var/tmp/portage/cross-ia64-unknown-linux-gnu/ghc-9999/work/ghc-9999/includes' -optc-I'/var/tmp/portage/cross-ia64-unknown-linux-gnu/ghc-9999/work/ghc-9999/includes/dist-derivedconstants/header' -static -H32m -O -w -package-name primitive-0.4.0.1 -hide-all-packages -i -ilibraries/primitive/. -ilibraries/primitive/dist-install/build -ilibraries/primitive/dist-install/build/autogen -Ilibraries/primitive/dist-install/build -Ilibraries/primitive/dist-install/build/autogen -Ilibraries/primitive/cbits -optP-include -optPlibraries/primitive/dist-install/build/autogen/cabal_macros.h -package base-4.7.0.0 -package ghc-prim-0.3.1.0 -O2 -XHaskell98 -XCPP -O2 -no-user-package-db -rtsopts -c libraries/primitive/cbits/primitive-memops.c -o libraries/primitive/dist-install/build/cbits/primitive-memops.o "inplace/bin/ghc-stage2" -optc-O2 -optc-pipe -optc-Ilibraries/primitive/cbits -optc-I'/var/tmp/portage/cross-ia64-unknown-linux-gnu/ghc-9999/work/ghc-9999/libraries/base/include' -optc-I'/var/tmp/portage/cross-ia64-unknown-linux-gnu/ghc-9999/work/ghc-9999/rts/dist/build' -optc-I'/var/tmp/portage/cross-ia64-unknown-linux-gnu/ghc-9999/work/ghc-9999/includes' -optc-I'/var/tmp/portage/cross-ia64-unknown-linux-gnu/ghc-9999/work/ghc-9999/includes/dist-derivedconstants/header' -static -prof -H32m -O -w -package-name primitive-0.4.0.1 -hide-all-packages -i -ilibraries/primitive/. -ilibraries/primitive/dist-install/build -ilibraries/primitive/dist-install/build/autogen -Ilibraries/primitive/dist-install/build -Ilibraries/primitive/dist-install/build/autogen -Ilibraries/primitive/cbits -optP-include -optPlibraries/primitive/dist-install/build/autogen/cabal_macros.h -package base-4.7.0.0 -package ghc-prim-0.3.1.0 -O2 -XHaskell98 -XCPP -O2 -no-user-package-db -rtsopts -c libraries/primitive/cbits/primitive-memops.c -o libraries/primitive/dist-install/build/cbits/primitive-memops.p_o /bin/sh: inplace/bin/ghc-stage2: No such file or directory make[1]: *** [libraries/primitive/dist-install/build/cbits/primitive-memops.o] Error 127 make[1]: *** Waiting for unfinished jobs.... /bin/sh: inplace/bin/ghc-stage2: No such file or directory -- Sergei

2 2

[PATCH] add better support for Raspberry Pi board
by Karel Gardas 29 Jan '13

29 Jan '13

This patch adds support for Raspberry Pi board. The board is based on Broadcom BCM2835, which includes ARM1176JZF-S CPU core. The core is classical ARM11 so implementing ARMv6 ISA, but also provides VFPv2 FPU unit. As this looks like fairly popular hardware, in this patch we assume detected ARMv6 hardware to support VFPv2. Also as Raspberry Pi is very commonly used with hard-float ABI Linux distribution, the patch relaxes a condition of "hard-float support for ARMv7 only" to any general ARM core. --- aclocal.m4 | 2 +- compiler/main/DriverPipeline.hs | 9 ++++++--- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/aclocal.m4 b/aclocal.m4 index edbe389..b23568f 100644 --- a/aclocal.m4 +++ b/aclocal.m4 @@ -369,7 +369,7 @@ AC_DEFUN([GET_ARM_ISA], [AC_DEFINE(arm_HOST_ARCH_PRE_ARMv7, 1, [ARM pre v7]) changequote(, )dnl ARM_ISA=ARMv6 - ARM_ISA_EXT="[]" + ARM_ISA_EXT="[VFPv2]" changequote([, ])dnl ], [changequote(, )dnl diff --git a/compiler/main/DriverPipeline.hs b/compiler/main/DriverPipeline.hs index c24bb51..81d0bc0 100644 --- a/compiler/main/DriverPipeline.hs +++ b/compiler/main/DriverPipeline.hs @@ -1454,14 +1454,17 @@ runPhase LlvmLlc input_fn dflags else if (elem VFPv3D16 ext) then ["-mattr=+v7,+vfp3,+d16"] else [] + ArchARM ARMv6 ext _ -> if (elem VFPv2 ext) + then ["-mattr=+v6,+vfp2"] + else ["-mattr=+v6"] _ -> [] -- On Ubuntu/Debian with ARM hard float ABI, LLVM's llc still -- compiles into soft-float ABI. We need to explicitly set abi -- to hard abiOpts = case platformArch (targetPlatform dflags) of - ArchARM ARMv7 _ HARD -> ["-float-abi=hard"] - ArchARM ARMv7 _ _ -> [] - _ -> [] + ArchARM _ _ HARD -> ["-float-abi=hard"] + ArchARM _ _ _ -> [] + _ -> [] sseOpts | isSse4_2Enabled dflags = ["-mattr=+sse42"] | isSse2Enabled dflags = ["-mattr=+sse2"] -- 1.7.3.2

2 5

rnfail055
by Simon Marlow 29 Jan '13

29 Jan '13

This test has been failing for a couple of days or so, anyone know why? Actual stderr output differs from expected: --- ./rename/should_fail/rnfail055.stderr 2013-01-07 08:47:29.856696930 +0000 +++ ./rename/should_fail/rnfail055.comp.stderr 2013-01-29 09:51:26.450096724 +0000 @@ -24,12 +24,12 @@ Type constructor `T1' has conflicting definitions in the module and its hs-boot file Main module: data T1 a b No C type associated - RecFlag Recursive + RecFlag Recursive, Promotable = T1 :: forall a b. [b] -> [a] -> T1 a b Stricts: _ _ FamilyInstance: none Boot file: data T1 a b No C type associated - RecFlag NonRecursive + RecFlag NonRecursive, Promotable = T1 :: forall a b. [a] -> [b] -> T1 a b Stricts: _ _ FamilyInstance: none @@ -37,12 +37,12 @@ Type constructor `T2' has conflicting definitions in the module and its hs-boot file Main module: data Eq b => T2 a b No C type associated - RecFlag Recursive + RecFlag Recursive, Promotable = T2 :: forall a b. a -> T2 a b Stricts: _ FamilyInstance: none Boot file: data Eq a => T2 a b No C type associated - RecFlag NonRecursive + RecFlag NonRecursive, Promotable = T2 :: forall a b. a -> T2 a b Stricts: _ FamilyInstance: none @@ -56,12 +56,12 @@ Type constructor `T5' has conflicting definitions in the module and its hs-boot file Main module: data T5 a No C type associated - RecFlag Recursive + RecFlag Recursive, Promotable = T5 :: forall a. a -> T5 a Stricts: _ Fields: field5 FamilyInstance: none Boot file: data T5 a No C type associated - RecFlag NonRecursive + RecFlag NonRecursive, Promotable = T5 :: forall a. a -> T5 a Stricts: _ FamilyInstance: none @@ -69,12 +69,12 @@ Type constructor `T6' has conflicting definitions in the module and its hs-boot file Main module: data T6 No C type associated - RecFlag Recursive + RecFlag Recursive, Not promotable = T6 :: Int -> T6 Stricts: _ FamilyInstance: none Boot file: data T6 No C type associated - RecFlag NonRecursive + RecFlag NonRecursive, Not promotable = T6 :: Int -> T6 HasWrapper Stricts: ! FamilyInstance: none @@ -82,12 +82,12 @@ Type constructor `T7' has conflicting definitions in the module and its hs-boot file Main module: data T7 a No C type associated - RecFlag Recursive + RecFlag Recursive, Promotable = T7 :: forall a a1. a1 -> T7 a Stricts: _ FamilyInstance: none Boot file: data T7 a No C type associated - RecFlag NonRecursive + RecFlag NonRecursive, Promotable = T7 :: forall a b. a -> T7 a Stricts: _ FamilyInstance: none *** unexpected failure for rnfail055(normal)

2 1

Re: Size of crosscompiled exectuable
by Simon Marlow 29 Jan '13

29 Jan '13

On 28/01/13 20:05, Nathan Hüsken wrote: > With > > SplitObjs = YES > > I get > > on the commandline: Warning: ignoring -fsplit-objs > "rm" -f libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.1.0.a > libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.1.0.a.contents > /usr/bin/find libraries/ghc-prim/dist-install/build/GHC/Classes_o_split > libraries/ghc-prim/dist-install/build/GHC/CString_o_split > libraries/ghc-prim/dist-install/build/GHC/Debug_o_split > libraries/ghc-prim/dist-install/build/GHC/Magic_o_split > libraries/ghc-prim/dist-install/build/GHC/PrimopWrappers_o_split > libraries/ghc-prim/dist-install/build/GHC/IntWord64_o_split > libraries/ghc-prim/dist-install/build/GHC/Tuple_o_split > libraries/ghc-prim/dist-install/build/GHC/Types_o_split -name '*.o' > -print >> > libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.1.0.a.contents > /usr/bin/find: > `libraries/ghc-prim/dist-install/build/GHC/Classes_o_split': No such > file or directory > /usr/bin/find: > `libraries/ghc-prim/dist-install/build/GHC/CString_o_split': No such > file or directory > /usr/bin/find: > `libraries/ghc-prim/dist-install/build/GHC/Debug_o_split': No such file > or directory > /usr/bin/find: > `libraries/ghc-prim/dist-install/build/GHC/Magic_o_split': No such file > or directory > /usr/bin/find: > `libraries/ghc-prim/dist-install/build/GHC/PrimopWrappers_o_split': No > such file or directory > /usr/bin/find: > `libraries/ghc-prim/dist-install/build/GHC/IntWord64_o_split': No such > file or directory > /usr/bin/find: > `libraries/ghc-prim/dist-install/build/GHC/Tuple_o_split': No such file > or directory > /usr/bin/find: > `libraries/ghc-prim/dist-install/build/GHC/Types_o_split': No such file > or directory > make[1]: *** > [libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.1.0.a] Error 1 > make: *** [all] Error 2 > > Anyone able to build a cross compiler with split-objs? It looks like we don't have split-objs support for ARM yet. It probably isn't too hard, but someone needs to look at the Perl script in driver/split/ghc-split.lprl. Cheers, Simon

1 0

[PATCH] #7595: Static flags code needs cleanup
by Jan Stolarek 29 Jan '13

29 Jan '13

I submitted a patch for ticket #7595 almost two weeks ago. It would be nice to have it reviewed and either accepted or rejected :) I know it's only a small refactoring of code, not a bug, and it may not be high on the priority list, but stil. http://hackage.haskell.org/trac/ghc/ticket/7595 Janek

2 2

HEADS UP (was Re: [commit: ghc] master: Tidy up cross-compiling (109a1e5))
by Simon Marlow 27 Jan '13

27 Jan '13

Cross-compilers please take note: On 17/01/13 12:57, Simon Marlow wrote: > Before > ------ > > To build a cross-compiler: > ./configure --target=<..> > > To compile a foreign GHC: > ./configure --host=<..> --target=<..> > > Now > --- > > To build a cross-compiler: > ./configure --target=<..> > And set "Stage1Only=YES" in mk/build.mk > > To compile a foreign GHC: > ./configure --target=<..> Cheers, Simon

5 9