[GHC] #9923: Offer copy-on-GC sliced arrays

#9923: Offer copy-on-GC sliced arrays -------------------------------------+------------------------------------- Reporter: dfeuer | Owner: simonmar Type: feature request | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 7.11 Keywords: | Operating System: Architecture: Unknown/Multiple | Unknown/Multiple Difficulty: Unknown | Type of failure: Blocked By: | None/Unknown Related Tickets: | Test Case: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- It's sometimes useful to split an array and send each part on to a different computation. Slicing works very well for this sort of thing, as long as both computations run to completion or become garbage at about the same time. It can work badly, however, if one computation is completed or abandoned and the other is retained—it may only hold a tiny slice, but that's enough to keep the entire array alive. The alternative, currently, is to copy the array to form two new arrays. This gets rid of the retention problem, but introduces a performance problem. One way to solve these problems might be to offer sliced arrays supported by the RTS. On collection, the garbage collector would copy each slice separately, turning it into an independent (and independently collectible) array. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9923 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9923: Offer copy-on-GC sliced arrays -------------------------------------+------------------------------------- Reporter: dfeuer | Owner: simonmar Type: feature request | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by svenpanne): Hmmm, I doubt that this would work: The proposal means that the garbage collector must allocate memory during collections, which is quite the opposite what it is supposed to do. Put another way: If somebody (SimonM?) thinks that this is possible, I would be very interested in the details. Even if this somehow works, there are several pathological cases like having lots of 999999-element slices of a 1000000-element array. Short-circuiting projection functions is OK (IIRC nhc did that first) because one doesn't have to allocate, but slicing is not really projecting. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9923#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9923: Offer copy-on-GC sliced arrays -------------------------------------+------------------------------------- Reporter: dfeuer | Owner: simonmar Type: feature request | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by dfeuer): Replying to [comment:1 svenpanne]:
Hmmm, I doubt that this would work: The proposal means that the garbage
collector must allocate memory during collections, which is quite the opposite what it is supposed to do. Put another way: If somebody (SimonM?) thinks that this is possible, I would be very interested in the details. Even if this somehow works, there are several pathological cases like having lots of 999999-element slices of a 1000000-element array. > > Short-circuiting projection functions is OK (IIRC nhc did that first) because one doesn't have to allocate, but slicing is not really projecting.
Yes, it would have to allocate while collecting, but I think this happens anyway. As for pathological cases, "Doctor! It hurts when I do ''this''! Don't do that." The suggested mechanism would not replace the usual sort of array slicing in general; it would be an alternative mechanism for situations where it was useful. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9923#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9923: Offer copy-on-GC sliced arrays -------------------------------------+------------------------------------- Reporter: dfeuer | Owner: simonmar Type: feature request | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by simonmar): The problem occurs if you have both a slice and a reference to the original array. The GC can't know until the end of GC whether it should just copy the slice or keep the whole array, and neither can it copy both the slice and the array (GC would require more memory than the original heap). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9923#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

The problem occurs if you have both a slice and a reference to the original array. The GC can't know until the end of GC whether it should just copy the slice or keep the whole array, and neither can it copy both
#9923: Offer copy-on-GC sliced arrays -------------------------------------+------------------------------------- Reporter: dfeuer | Owner: simonmar Type: feature request | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by dfeuer): Replying to [comment:3 simonmar]: the slice and the array (GC would require more memory than the original heap). Why is that a problem, per se? The application I was thinking about when I came up with this idea was a function for converting an `Array` to a `Seq`. Currently, there are two ways to do this: 1. You can lazily build a sequence "view" of the array. This is very fast, and only allocates subtrees of the finger tree as they are needed. But the entire underlying array is live until the last leaf of the tree is forced. 2. You can build the sequence monadically, copying each element out of the array (using a nice little function from `Data.Primitive.Array`). This is sure to release the array, at the expense, of course, of actually building the whole sequence (a good bit larger than the array it represents) before using any of it. The copy-on-GC concept would, I believe, offer a middle ground between these options, with performance close to (1) with the leak safety of (2). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9923#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9923: Offer copy-on-GC sliced arrays -------------------------------------+------------------------------------- Reporter: dfeuer | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by thomie): * owner: simonmar => * failure: None/Unknown => Runtime performance bug -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9923#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC