April 2013 - ghc-tickets

[GHC] #7853: UTF encodings do not detect overlong forms
by GHC 23 Apr '13

23 Apr '13

#7853: UTF encodings do not detect overlong forms ----------------------------------------+----------------------------------- Reporter: batterseapower | Owner: Type: bug | Status: new Priority: normal | Component: libraries/base Version: 7.6.3 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Incorrect result at runtime | Blockedby: Blocking: | Related: ----------------------------------------+----------------------------------- Overlong UTF-{8,16} sequences can have security implications (http://www.cl.cam.ac.uk/~mgk25/unicode.html) Decoders for these encodings should detect them and flag them as invalid characters. GHC's implementations of these decoders do not do so! This problem has additional implications for GHC since as we are not rejecting overlong sequences, trying to roundtrip 0xC0 0xB1 through UTF-8//ROUNDTRIP results in 0x31 rather than the expected sequence. The roundtripping fails because the overlong sequence is not flagged up by the UTF-8 encoder and so the surrogate escape mechanism never gets a chance to work. -- Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/7853> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 2

Re: [GHC] #3977: Support double-byte encodings (Chinese/Japanese/Korean) on Windows
by GHC 23 Apr '13

23 Apr '13

#3977: Support double-byte encodings (Chinese/Japanese/Korean) on Windows ---------------------------------+------------------------------------------ Reporter: shelarcy | Owner: batterseapower Type: feature request | Status: new Priority: low | Milestone: 7.6.2 Component: libraries/base | Version: 6.13 Keywords: | Os: Windows Architecture: Unknown/Multiple | Failure: Incorrect result at runtime Difficulty: | Testcase: Blockedby: | Blocking: Related: #5754 | ---------------------------------+------------------------------------------ Changes (by batterseapower): * owner: => batterseapower Comment: Fixed on the dbcs branch of the base library. -- Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/3977#comment:16> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 0

Re: [GHC] #2648: Report out of date interface files robustly
by GHC 23 Apr '13

23 Apr '13

#2648: Report out of date interface files robustly ---------------------------------+------------------------------------------ Reporter: simonpj | Owner: Type: feature request | Status: new Priority: lowest | Milestone: 7.6.2 Component: GHCi | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: GHCi crash Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Comment(by liyang): I'm not entirely sure if what I'm seeing is due to out-of-date interface files, but see the details below: {{{ ghc: panic! (the 'impossible' happened) (GHC version 7.6.3 for x86_64-unknown-linux): tcIfaceGlobal (local): not found: <<details unavailable>> }}} -- Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/2648#comment:11> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 0

[GHC] #7662: Improve GC of mutable objects
by GHC 23 Apr '13

23 Apr '13

#7662: Improve GC of mutable objects -----------------------------+---------------------------------------------- Reporter: ezyang | Owner: Type: feature request | Status: new Priority: normal | Component: Runtime System Version: 7.7 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: None/Unknown | Blockedby: Blocking: | Related: -----------------------------+---------------------------------------------- Haskell is a purely functional language at its core, but it can be also used for workloads involving mutation; it is for these workloads that our GC can be pretty poorly tuned (historically, we’ve fixed these issues case-by-case when they were causing problems for users, e.g. #650). Fortunately, the question of generational garbage collection in the face of lots of mutability is a well-studied one (c.f. the JVM) so this bug asks whether or not we can systematically appropriate any of the machinery developed here for GHC. Some ideas include: * Utilizing card marking at the page level for write barriers, instead of our current mutable lists, * Separating mutable and immutable (and lazy) data on the heaps, to make the previous scheme more likely to work well, * Organizing our generations in different pages, so that checking the generation of any object is as simple as a pointer comparison, * Figure out if we can do better than permanently keeping all mutable arrays permanently on the mutable list (which is really killer when you have lots of tiny mutable arrays), * More mutable closure types to remove the indirection that would normally result from an IORef (actually, you can UNPACK these, and I’m not 100% what happens in that case; it depends on how MutVar#s are represented) There’s probably more things but this is just a start to get things rolling. Test cases and benchmarks also appreciated! -- Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/7662> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 5

Re: [GHC] #2648: Report out of date interface files robustly
by GHC 23 Apr '13

23 Apr '13

#2648: Report out of date interface files robustly ---------------------------------+------------------------------------------ Reporter: simonpj | Owner: Type: feature request | Status: new Priority: lowest | Milestone: 7.6.2 Component: GHCi | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: GHCi crash Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Changes (by liyang): * failure: None/Unknown => GHCi crash * component: Compiler => GHCi Comment: For what it's worth, I don't think I have ever seen this when compiling the project. -- Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/2648#comment:10> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 0

Re: [GHC] #2648: Report out of date interface files robustly
by GHC 23 Apr '13

23 Apr '13

#2648: Report out of date interface files robustly ---------------------------------+------------------------------------------ Reporter: simonpj | Owner: Type: feature request | Status: new Priority: lowest | Milestone: 7.6.2 Component: Compiler | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Changes (by liyang): * cc: hackage.haskell.org@… (added) * failure: => None/Unknown * version: 6.8.3 => 7.6.3 Comment: This is biting me on a regular basis when I {{{:reload}}} a fairly large project in GHCi. It didn't seem to crop up so often (at all?) back when I was still using 7.4.2… -- Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/2648#comment:9> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 0

[GHC] #7606: Stride scheduling for Haskell threads with priorities
by GHC 23 Apr '13

23 Apr '13

#7606: Stride scheduling for Haskell threads with priorities -----------------------------+---------------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: normal | Component: Runtime System Version: 7.7 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: None/Unknown | Blockedby: Blocking: | Related: -----------------------------+---------------------------------------------- Currently, GHC uses a round-robin scheduler for Haskell threads, with some heuristics for when threads should be bumped to the front of the queue. This patch set replaces this scheduler with 'stride scheduling', as described by Waldspurger and Weihl '95, which is an efficient, deterministic method for scheduling processes with differing priorities. Priorities are assigned by giving 'tickets' to threads; a thread with twice as many tickets as another will run twice as often. I’d like to replace the round-robin scheduler completely with this scheduler. Here are nofib benchmarks comparing the old scheduler to the new: {{{ -------------------------------------------------------------------------------- Program Size Allocs Runtime Elapsed TotalMem -------------------------------------------------------------------------------- Min -0.0% -52.2% -18.8% -18.6% -83.1% Max +1.0% +4.2% +4.9% +5.1% +7.7% Geometric Mean +0.1% -2.8% -0.9% -0.9% -2.9% }}} Here are some technical details: * Without any changes to ticket values, scheduling behavior should be functionally identical to round-robin. (By default, new threads, including the IO thread, get allocated the max nubmer of tickets possible.) This is not quite identical, since our heap does not have FIFO property (see below.) * The current patch-set uses a very simple (e.g. undergrad level) resizable-array backed heap to implement the priority queue; we can play some tricks to reduce the memory footprint of the priority queue (e.g. using the container_of macro to eliminate the extra keys store); and a more fancy data structure would make it easier for us to surgically remove entries or reweight them, but I wanted to keep overhead low. If anyone has a pet implementation of priority queues in C feel free to swap it in. Right now, this only affects the uses of promoteInRunQueue() in Messages.c; I still need to check if #3838 has regressed. * We get three new primops: `setTickets#`, `getTickets#` and `modifyTickets#`. We don't support creating threads with specific numbers of tickets (mostly because that would have added an annoyingly large set of extra primops); instead, you're expected to spawn a thread which gets max-ticket allocation, and then weight it down. * `_link` is no longer used for linking TSOs in the run queue. I have tried my best to stamp out any code which operated on this assumption, but I may have missed some. * Modifying a TSO's tickets takes out the scheduler lock; the hope is that this operation is quick and rare enough that a global lock here is not too bad. * We are unfortunately stuck with some magic constants w.r.t. ticket values: 1 << 20 is the maximum number of tickets our implementation is hard-coded to support. * Sleeping and blocked tasks do not get any 'bonus' for being blocked. * In an ideal world, when a thread hits a black hole, it should temporarily give its tickets to the thread evaluating the black hole, so it will unblock more quickly. Unfortunately, implementing this is pretty complicated (the blackhole evaluating thread could die; or it could get stuck on a blackhole itself and need to gift its tickets; it shouldn't be able to give away the tickets it was gifted.) So this implementation leaves that out. Similar semantics for MVars are probably possible, but will require userland assistance too. I haven't prepared a patch to base yet with a user-level API, but here is a proposed draft: {{{ type Tickets = Int -- | Maximum number of tickets we support a thread having. (Currently 2 >> 20) -- Note that this doesn't bound the *global* maximum tickets. maxTickets :: Tickets -- | Changes the number of tickets allocated to a thread. The ticket count must -- not be less than or equal to zero, or greater than maxTickets. (Corresponds -- to setTickets# primop) setTickets :: ThreadId -> Tickets -> IO () -- | Returns the number of tickets currently allocated to a thread. (Corresponds to -- getTickets# primop) getTickets :: ThreadId -> IO Tickets -- | Atomically performs a linear transformation on the number of tickets a thread; -- e.g. scaling the number of tickets by a rational number, adding another fixed -- set of tickets, and then returning the number of 'leftover' tickets from the operation; e.g. -- if the net amount of tickets is reduced, then the returned result is positive; -- if the net amount of tickets is increased, the returned result is negative. -- In the absence of concurrent threads, the following property holds forall -- t, m and x: -- -- do r <- getTickets t -- d <- scaleTickets t m x -- r' <- getTickets t -- return (r == r' + d) -- -- If the scaling would reduce the number of tickets below zero or increase the -- number of tickets beyond maxTickets, this function will instead fail with -- an exception. This function will be subject to some rounding error; powers of two -- are, however, likely to be exact. (Corresponds to modifyTickets# primop; note -- that the sentinel value for failure is maxTickets + 1, since it is impossible for -- a thread's ticket value to change by that much.) modifyTickets :: ThreadId -> Ratio Int -> Tickets -> IO Tickets -- | Forks a new thread, transferring some percentage of tickets from the current -- thread to it (so the net number of tickets stays constant.) Fails if the rational -- is greater than 1 or less than or equal to zero, or if there are not enough tickets -- in the current thread. forkIOWith :: IO a -> Ratio Int -> IO ThreadId }}} -- Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/7606> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 40

[GHC] #7857: GHC says there are unsafe overlapping instances, but only one instance applies
by GHC 23 Apr '13

23 Apr '13

#7857: GHC says there are unsafe overlapping instances, but only one instance applies -----------------------------+---------------------------------------------- Reporter: dmwit | Owner: Type: bug | Status: new Priority: normal | Component: Compiler Version: 7.6.1 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: None/Unknown | Blockedby: Blocking: | Related: -----------------------------+---------------------------------------------- The following code rightfully does not compile: {{{ import Text.Printf f :: a -> b f = undefined g i = f $ printf "" i }}} However, the error message is a very surprising one: {{{ test.hs:6:1: Unsafe overlapping instances for PrintfType (t -> a0) arising from the ambiguity check for `g' The matching instance is: instance [safe] (PrintfArg a, PrintfType r) => PrintfType (a -> r) -- Defined in `Text.Printf' It is compiled in a Safe module and as such can only overlap instances from the same module, however it overlaps the following instances from different modules: When checking that `g' has the inferred type `forall a b t. PrintfType (t -> a) => t -> b' Probable cause: the inferred type is ambiguous }}} What I find surprising is the claim that there are overlapping instances, followed by a list of instances that clearly does not have any overlaps. For comparison, changing the definition of g to be {{{ g = f . printf "" }}} gives a much more reasonable error: {{{ test.hs:6:9: No instance for (PrintfArg a0) arising from a use of `printf' The type variable `a0' is ambiguous Possible fix: add a type signature that fixes these type variable(s) Note: there are several potential instances: instance [safe] PrintfArg Char -- Defined in `Text.Printf' instance [safe] PrintfArg Double -- Defined in `Text.Printf' instance [safe] PrintfArg Float -- Defined in `Text.Printf' ...plus 12 others In the second argument of `(.)', namely `printf ""' In the expression: f . printf "" In an equation for `g': g = f . printf "" }}} -- Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/7857> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 2

Re: [GHC] #5682: Properly parse kind operators (from promoted type operators)
by GHC 22 Apr '13

22 Apr '13

#5682: Properly parse kind operators (from promoted type operators) -------------------------------------+-------------------------------------- Reporter: lunaris | Owner: dreixel Type: bug | Status: new Priority: low | Milestone: 7.6.2 Component: Compiler | Version: 7.3 Keywords: PolyKinds, ghc-kinds | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: GHC rejects valid program Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | -------------------------------------+-------------------------------------- Changes (by elliottt): * cc: trevor@… (added) -- Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/5682#comment:7> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 0

[GHC] #7818: Cost of STACK object should be attributed to CCCS at time of thread creation
by GHC 22 Apr '13

22 Apr '13

#7818: Cost of STACK object should be attributed to CCCS at time of thread creation -----------------------------+---------------------------------------------- Reporter: ezyang | Owner: ezyang Type: task | Status: new Priority: normal | Component: Profiling Version: 7.7 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: None/Unknown | Blockedby: Blocking: | Related: -----------------------------+---------------------------------------------- Right now, we always charge STACK objects to the SYSTEM cost centre. I think it makes more sense for us to charge them to the CCCS responsible for creating the thread: it’s more specific and we don’t really lose anything in the trade (it’s easy to ask for a per-closure-type profile). Here’s a strawman patch (obviously this doesn’t do anything when not compiling the profiling way.) Are there any objections? {{{ diff --git a/rts/Threads.c b/rts/Threads.c index b617616..76ac705 100644 --- a/rts/Threads.c +++ b/rts/Threads.c @@ -84,7 +84,7 @@ createThread(Capability *cap, W_ size) stack_size = round_to_mblocks(size - sizeofW(StgTSO)); stack = (StgStack *)allocate(cap, stack_size); TICK_ALLOC_STACK(stack_size); - SET_HDR(stack, &stg_STACK_info, CCS_SYSTEM); + SET_HDR(stack, &stg_STACK_info, cap->r.rCCCS); stack->stack_size = stack_size - sizeofW(StgStack); stack->sp = stack->stack + stack->stack_size; stack->dirty = 1; @@ -575,7 +575,11 @@ threadStackOverflow (Capability *cap, StgTSO *tso) chunk_size * sizeof(W_)); new_stack = (StgStack*) allocate(cap, chunk_size); - SET_HDR(new_stack, &stg_STACK_info, CCS_SYSTEM); +#ifdef PROFILING + SET_HDR(new_stack, &stg_STACK_info, old_stack->header.prof.ccs); +#else + SET_HDR(new_stack, &stg_STACK_info, CCCS_SYSTEM); +#endif TICK_ALLOC_STACK(chunk_size); new_stack->dirty = 0; // begin clean, we'll mark it dirty below }}} -- Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/7818> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 4