January 2013 - ghc-devs

Trac source browser not working
by Simon Marlow 09 Mar '13

09 Mar '13

I tried to follow a link to a commit on Trac today, and got an error: internal error (got unexpected object kind 'commit') The link I followed: http://hackage.haskell.org/trac/ghc/changeset/109a1e53287f50103e8a5b5922759… This was working before, has anything changed recently that would break it? I tried browsing to a couple of other commits and got the same error, so it doesn't seem specific to that commit. Cheers, Simon

3 2

[PATCH] fixes for android in libraries/unix
by Nathan Hüsken 17 Feb '13

17 Feb '13

Hey, These patches are for libraries/unix. They may need some discussing, because I am not sure if this is the right thing. 1. On android the passwd struct does not define pw_gecos. The first patch therefor sets it to the empty string. 2. The android ndk does not define _POSIX_VDISABLE, but it defines _PC_VDISABLE. The second patch defines _POSIX_VDISABLE to _PC_VIDISABLE, which is (as I undestand it) what the POSIX programmer guide suggests. 3. The android ndk does not define telldir and seekdir. The third patch removes them when compiling for android.

3 5

[PATCH] (haskeline) Include termios.h on android
by Nathan Hüsken 17 Feb '13

17 Feb '13

Hey, This patch is for libraries/haskeline. On android, winsize is defined in termios.h and therefor termios.h must be included.

2 3

[PATCH] GHC_CONVERT_VENDOR: allow 'w64' vendor, like x86_64-w64-mingw32
by Sergei Trofimovich 12 Feb '13

12 Feb '13

From: Sergei Trofimovich <slyfox(a)gentoo.org> Gentoo calls 'w64' a mingw flavour with mingw-w64 runtime. Signed-off-by: Sergei Trofimovich <slyfox(a)gentoo.org> --- aclocal.m4 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aclocal.m4 b/aclocal.m4 index 9f9593a..30d492c 100644 --- a/aclocal.m4 +++ b/aclocal.m4 @@ -1812,7 +1812,7 @@ case "$1" in # converts vendor from gnu to ghc naming, and assigns the result to $target_var AC_DEFUN([GHC_CONVERT_VENDOR],[ case "$1" in - pc|gentoo) # like i686-pc-linux-gnu and i686-gentoo-freebsd8 + pc|gentoo|w64) # like i686-pc-linux-gnu, i686-gentoo-freebsd8, x86_64-w64-mingw32 $2="unknown" ;; softfloat) # like armv5tel-softfloat-linux-gnueabi -- 1.8.1.1

2 1

Compiling DynFlags is jolly slow
by Ian Lynagh 09 Feb '13

09 Feb '13

On Thu, Jan 31, 2013 at 12:23:02PM -0000, GHC wrote: > > I can't compile `DynFlags` on my 256MB Raspberry Pi. Could we do > something about having less auto-generated code in this module please? > > -- > Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/7258#comment:13> The ticket's really about performance problems in the compiler rather than DynFlags in particular, so moving to the list. Even the non-generated code in DynFlags is quite large, and ungainly to edit. What would you think about splitting it up, perhaps moving DumpFlag, GeneralFlag and WarningFlag into separate modules? Ideally each module would also export a [Flag (CmdLineP DynFlags)], which would then be ++ed together in DynFlags, but I'm not sure if that's possible without making the modules mutually recursive. On the other hand, perhaps there's a way to split it up which would remove the need for (some of) the existing import loops; I'm not sure how feasible that is OTTOMH. Thanks Ian

2 2

simd branch ready for review
by Geoffrey Mainland 06 Feb '13

06 Feb '13

Hi Simon, I've pushed my simd branch to darcs.haskell.org. Everything has been rebased against HEAD. Simon PJ and I looked over the changes together already, but I wanted to give you (and everyone on ghc-devs) the opportunity to look things over before I merge to HEAD. Simon PJ and I came up with a few questions/notes for you, but hopefully nothing that should delay a merge. * Win32 issues Modern 32-bit x86 *NIX systems align the stack to 16-bytes, but Win32 aligns only to 4-bytes. LLVM does not assume 16-byte stack alignment. Instead, on platforms where 16-byte stack alignment is not guaranteed, it 1) always outputs a function prologue that 2) aligns the stack to a 16-byte boundary with an "and" instructions, and it also 3) disables tail calls. Because LLVM aligns the stack for a function that has SSE register spills, it also generates movaps instructions (aligned SSE moves) for the spills. This makes SSE support on Win32 difficult, and in my opinion not worth worrying about. The alternative is to 1) patch LLVM to disable the stack-alignment code so that we recover the ability to use tail calls and so that ebp scribbled over by the prologue and 2) patch the mangler to rewrite LLVM's movaps (move aligned) instructions to movups (move unaligned) instructions. I have these patches, but they are not included in the simd branch. * How hard would it be to dump ArgRep for PrimRep? It looks straightforward. Is it worth doing? * How hard would it be to track bit width in PrimRep? I recall chatting with you once about adding explicit support for, e.g., 8- and 16-bit Word/Int primops instead of relying on narrowing. Since SIMD vectors need to know the exact bit-width of their elements, I've had to create a PrimElemRep data type in compiler/types/TyCon.lhs, but I'd really like to be able to re-use PrimRep instead. * If we replaced all old-style C-- code, could we get rid of the explicit STG registers completely? Simon PJ suggested that we use real machine registers directly, so, for example, GlobalReg's constructors would have FastString fields instead of Int fields. * Could we add a CmmType field to GlobalReg's constructors? You'll see that I added a new XmmReg constructor to GlobalReg, but because I don't know the type of an XmmReg, I have to bitcast everywhere in the generated LLVM code because LLVM wants to know not just that a value is a 16-byte vector, but that it is, e.g., a 16-byte vector containing 2 64-bit doubles. Having a CmmType attached to a GlobalReg---or pairing a GlobalReg with a CmmType when assigning registers---would let me avoid all these casts. Thanks! Geoff

8 27

Validate failures on OS X/amd64
by Austin Seipp 31 Jan '13

31 Jan '13

Today, while running validate I encountered a handful of errors: VERALL SUMMARY for test run started at Thu Jan 31 15:58:15 CST 2013 3591 total tests, which gave rise to 14298 test cases, of which 4 caused framework failures 11044 were skipped 3156 expected passes 28 had missing libraries 56 expected failures 1 unexpected passes 13 unexpected failures Unexpected passes: ../../libraries/directory/tests T4113 (normal) Unexpected failures: ../../libraries/base/tests qsemn001 [bad exit code] (normal) codeGen/should_run cgrun071 [bad exit code] (normal) concurrent/should_run T5611 [bad stdout] (normal) perf/compiler T4801 [stat too good] (normal) perf/haddock haddock.base [stat not good enough] (normal) perf/should_run T5113 [stat not good enough] (normal) simplCore/should_compile T3717 [stderr mismatch] (optasm) simplCore/should_compile T4908 [stderr mismatch] (optasm) simplCore/should_compile T7360 [stderr mismatch] (optasm) simplCore/should_compile spec-inline [stderr mismatch] (optasm) typecheck/should_compile holes [stderr mismatch] (normal) typecheck/should_compile holes2 [stderr mismatch] (normal) typecheck/should_compile holes3 [stderr mismatch] (normal) Most of these are benign. All the 'simplCore' tests are failures that seem to be a result of the new demand analyzer, with tiny things like this: --- ./simplCore/should_compile/spec-inline.stderr 2013-01-18 12:55:59.000000000 -0600 +++ ./simplCore/should_compile/spec-inline.comp.stderr 2013-01-31 16:12:33.000000000 -0600 @@ -116,7 +116,7 @@ Roman.foo1 :: Data.Maybe.Maybe GHC.Types.Int [GblId, Caf=NoCafRefs, - Str=DmdType, + Str=DmdType m2, Unf=Unf{Src=<vanilla>, TopLvl=True, Arity=0, Value=True, ConLike=True, WorkFree=True, Expandable=True, Guidance=IF_ARGS [] 10 20}] and: --- ./simplCore/should_compile/T3717.stderr 2013-01-18 12:55:59.000000000 -0600 +++ ./simplCore/should_compile/T3717.comp.stderr 2013-01-31 16:12:33.000000000 -0600 @@ -22,13 +22,13 @@ ConLike=True, WorkFree=True, Expandable=True, Guidance=ALWAYS_IF(unsat_ok=True,boring_ok=False) Tmpl= \ (w [Occ=Once!] :: GHC.Types.Int) -> - case w of _ { GHC.Types.I# ww [Occ=Once] -> - case T3717.$wfoo ww of ww1 { __DEFAULT -> GHC.Types.I# ww1 } + case w of _ { GHC.Types.I# ww1 [Occ=Once] -> + case T3717.$wfoo ww1 of ww2 { __DEFAULT -> GHC.Types.I# ww2 } So these look like just minor output wibbles and otherwise fine changes. The 'holes' tests are failing because of an error message tweak. They are all of the form: holes3.hs:15:15: - Found hole `_' with type b0 + Found hole `_' with type: b0 so these are an easy fix too. Did someone forget to push testsuite patches? I can at least fix the TypeHoles tests since they're so trivial, and the demand analyzer changes look benign too, but I just wanted to make sure these are OK and I'm not stepping on toes if I were to do that. I'm investigating the others at the moment, these were just the easy ones I saw. -- Regards, Austin

2 1

Code review, new scheduler:
by Alexander Kjeldaas 31 Jan '13

31 Jan '13

Hi Edward, thanks for your work on the new scheduler! I have a done a super light-weight review. I think documenting the code a little more would help readability. http://hackage.haskell.org/trac/ghc/attachment/ticket/7606/0002-Stride-sche… TSO.h: Could you document ss_*? These are all important variables in the scheduler, and not documented. For example, the code in Threads.c for setting these is can act as some documentation, but something needs to be documented here. (Please don't point to a paper as primary documentation.) // 64-bit to prevent overflows; only ever accessed by the task which owns TSO. 170 StgWord64 ss_pass; 171 // These are bounded above by STRIDE1, which is less than max 32-bit word. 172 // You must take out the sched_lock to write to these; reads are OK 173 StgWord ss_tickets, ss_stride, ss_remain; 174 Schedule.c: 727 StgTSO *t; Rename to 'tso' to be descriptive. 744 // go through all of the TSOs in the run queue and decide where 745 // they should go 746 // XXX We can create the new heap more efficiently O(n) by just 747 // blitting them in and then re-heapifying 748 if (!emptyRunQueue(cap)) { 749 StgWord64 k; Ditto for 'k'. 1201 scheduleHandleThreadBlocked( Capability *cap, StgTSO *t ) Rename 't' to 'tso'. Schedule.h: 125 // oh no magic constant 126 #define STRIDE1 (1 << 20) Document STRIDE1 147 EXTERN_INLINE void 148 annulTSO(StgTSO *tso) { 149 // hack to make some invariants with regards to block_info and _link work 150 // this is called whereever we would have stepped all over the 151 // fields in the linked list implementation 152 tso->_link = END_TSO_QUEUE; 153 tso->block_info.closure = (StgClosure*)END_TSO_QUEUE; It would be great to have a pointer to the invariants, or the invariant(s) documented. 213 tso->ss_pass += tso->ss_stride; 214 StgWord64 r; 215 if (tso->ss_pass <= cap->ss_pass) { 216 // Thread is behind, it will get scheduled in front with no 217 // intervention (note that cap->ss_pass is probably nonsense, 218 // since it doesn't include *this* thread.) 219 r = tso->ss_pass; 220 } else if (tso->ss_pass - tso->ss_pass <= cap->ss_pass) { This expression looks weird/magic, tso->ss_pass - tso->ss_pass is 0. 221 // Thread is in good standing, schedule it in front 222 // (next iteration, they will not be in good standing if 223 // the global pass doesn't advance by much; that is, this 224 // thread managed to cut in front of other threads which 225 // are running behind.) 226 r = cap->ss_pass; 227 } else { 228 // Thread is not in good standing, schedule it later. 229 // Eventually, global pass will advance enough that the 230 // thread will be in good standing again, and can cut 231 // to the front. 232 r = tso->ss_pass; Threads.c: 361 if (migrating) { 362 joinRunQueue(cap,tso); 363 } else { 364 appendToRunQueue(cap,tso); 365 } Space after ',' Select.c: and rts/win32/AsyncIO.c 309 tso->ss_remain = 0; 310 joinRunQueue(&MainCapability,tso); Should this be an abstraction in itself? rts/Capability.h: 59 PQueue *run_pqueue; 60 61 // [SSS] Stride scheduling extensions. The Task with this 62 // Capability has exclusive access to this variable. 63 nat ss_pass; Document ss_pass. For example how it relates to capPassUpdate and pushOnRunQueue (the weird/magic I commented on above) and invariants. Alexander

2 1

Re: Code review, new scheduler:
by Alexander Kjeldaas 31 Jan '13

31 Jan '13

On Thu, Jan 31, 2013 at 11:06 AM, Edward Z. Yang <ezyang(a)mit.edu> wrote: > Excerpts from Alexander Kjeldaas's message of Thu Jan 31 01:57:17 -0800 > 2013: > > Found it, it's still there: > > > http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/kernel/timer.c?v… > > > > Note that these are *timers*. Timers are often set up and then canceled. > > As you see, the buckets are not sorted, timers are just put at the end > of > > a list, and the lists are even hard-coded so they are more general than > > log2 buckets. > > > > The point of this is to partially sort the timers as they get closer to > > expiration. Thus you don't pay the full sorting cost if the timer is > > modified before it expires. > > Ah, I see. GHC threads are very rarely canceled, so I don't think we would > see much benefit here. > > You are probably right. I can think of priority inversion with rapidly changing priorities as a possible use-case then. Anyways, keep up the good work! Alexander > Edward >

1 0

Re: Code review, new scheduler:
by Alexander Kjeldaas 31 Jan '13

31 Jan '13

Found it, it's still there: http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/kernel/timer.c?v… Note that these are *timers*. Timers are often set up and then canceled. As you see, the buckets are not sorted, timers are just put at the end of a list, and the lists are even hard-coded so they are more general than log2 buckets. The point of this is to partially sort the timers as they get closer to expiration. Thus you don't pay the full sorting cost if the timer is modified before it expires. Alexander On Thu, Jan 31, 2013 at 10:50 AM, Edward Z. Yang <ezyang(a)mit.edu> wrote: > Excerpts from Alexander Kjeldaas's message of Thu Jan 31 01:43:51 -0800 > 2013: > > What operations do you mostly do in the priority queue? > > Insert, removeEmpty, and the occasional delete. > > > I'd like to point you to the timer implementation that we did in the > linux > > kernel ages ago. It is basically a set of log2 slots where each slot > > contains timers for time [2^i..2^(i+1)>. The dlinked lists make it O(1) > to > > remove, change, or add a timer. The timer interrupt basically "eats" > > linked lists, and when empty "splats" the first free slot out over the > > upper slots (which is O(n) worst case). The structure is simple and > > optimized for doing a lot of mutation of the timers. It beats a heap > hands > > down in the kernel at least. > > Interesting! That sounds simple enough to merit an implementation. However, > I'm given to understand that the current kernel scheduler uses a red-black > tree? (I also have an implementation of one of those, though it's based > off of > jemalloc's implementation.) > > Edward >

1 0