#9655: Do not UNPACK strict fields that are very wide -------------------------------------+------------------------------------- Reporter: simonpj | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Description changed by simonpj: Old description:
John Lato [http://www.haskell.org/pipermail/ghc- devs/2014-September/006473.html writes} (a propos of another discussion): This is possibly unrelated, but the setup seems almost identical to a very similar problem we had in some code, i.e. very long compile times (6+ minutes for 1 module) and excessive memory usage when compiling generic serialization instances for some data structures.
In our case, I also thought that INLINE functions were the cause of the problem, but it turns out they were not. We had a nested data structure, e.g. {{{
data Foo { fooBar :: !Bar, ... } }}} with `Bar` a very wide product type (~150 fields).
Even when we explicitly NOINLINE'd the function that serialized Bar, GHC still created a very large helper function of the form: {{{
serialize_foo :: Int# -> Int# -> ... }}} where the arguments were the unboxed fields of the Bar structure, along with the other fields within Foo. It appears that even though the serialization function was NOINLINE'd, it simply created a Builder, and while combining the Builder's ghc saw the full structure. Our serializer uses blaze, but perhaps Binary's builder is similar enough the same thing could happen.
Anyway, in our case the fix was to simply remove the bang pattern from the 'fooBar' record field.
So the question is: '''should GHC auto-unpack a strict argument of a data constructor, if the argument type is a very wide product type?'''. I think "no". The unpacking can save allocation (by allocating one object instead of two) but it can also increase it (at a pattern matching site). So it should probably only happen automatically for sufficiently narrow types.
How narrow? We need some testing, but my guess is three or four words. Maybe a flag to set the size? (Maybe not worth the pain.)
Incidentally, the choice can already be manually controlled with `{-# UNPACK #-}` and `{-# NOUNPACK #-}` pragmas.
New description: John Lato [http://www.haskell.org/pipermail/ghc- devs/2014-September/006473.html writes] (a propos of another discussion): "This is possibly unrelated, but the setup seems almost identical to a very similar problem we had in some code, i.e. very long compile times (6+ minutes for 1 module) and excessive memory usage when compiling generic serialization instances for some data structures. In our case, I also thought that INLINE functions were the cause of the problem, but it turns out they were not. We had a nested data structure, e.g. {{{
data Foo { fooBar :: !Bar, ... } }}} with `Bar` a very wide product type (~150 fields).
Even when we explicitly NOINLINE'd the function that serialized Bar, GHC still created a very large helper function of the form: {{{
serialize_foo :: Int# -> Int# -> ... }}} where the arguments were the unboxed fields of the Bar structure, along with the other fields within Foo. It appears that even though the serialization function was NOINLINE'd, it simply created a Builder, and while combining the Builder's ghc saw the full structure. Our serializer uses blaze, but perhaps Binary's builder is similar enough the same thing could happen.
Anyway, in our case the fix was to simply remove the bang pattern from the 'fooBar' record field." So the question is: '''should GHC auto-unpack a strict argument of a data constructor, if the argument type is a very wide product type?'''. I think "no". The unpacking can save allocation (by allocating one object instead of two) but it can also increase it (at a pattern matching site). So it should probably only happen automatically for sufficiently narrow types. How narrow? We need some testing, but my guess is three or four words. Maybe a flag to set the size? (Maybe not worth the pain.) Incidentally, the choice can already be manually controlled with `{-# UNPACK #-}` and `{-# NOUNPACK #-}` pragmas. -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9655#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler