Unpacking across modules
Hello, What is the interaction between the UNPACK pragma and modules? I've attached a small test case where a datatype is unpacked when it and its associated functions are defined in the same file in which they are used, but it is not unpacked if the definitions are given in another module. There are two versions of the data type, a monomorphic one, defined by:
data Vec3 = Vec3 Double Double Double
and a polymorphic, recursive one, defined by
data C a b = C a b
With the following typedef to make them equivalent
type Vec3 = C Double (C Double (C Double () ))
In the actual file, they're both appropriately annotated with bangs and UNPACK pragmas. The mono datatype is successfully unpacked in both cases, when it is defined in the same file and in a different file. The poly datatype is only unpacked when it is defined in the same file. To see the behavior, compile the attached files with -DMONO_SAME, -DMONO_OTHER, -DPOLY_SAME and -DPOLY_OTHER. In all cases except POLY_OTHER, the program runs in constant space, and in time that is quite competitive with C. (Nice work guys!) However, for POLY_OTHER you will likely have to kill it. The only other flag I used is -O2. Please let me know there's something I'm missing here. Is there a reason why this data type cannot be unboxed across modules? Or is this a bug? Thanks, Scott PS: Also, when the INLINE pragma is used with the Storable instance for the polymorphic data type, it causes heap explosion, even in the same file. Any thoughts here?
Scott Dillard wrote:
What is the interaction between the UNPACK pragma and modules? I've attached a small test case where a datatype is unpacked when it and its associated functions are defined in the same file in which they are used, but it is not unpacked if the definitions are given in another module. There are two versions of the data type, a monomorphic one, defined by:
data Vec3 = Vec3 Double Double Double
and a polymorphic, recursive one, defined by
data C a b = C a b
With the following typedef to make them equivalent
type Vec3 = C Double (C Double (C Double () ))
Firstly, adding an UNPACK pragma to a polymorphic component has no effect (try removing them, you'll get the same result). The UNPACK pragma affects the representation of the constructor; a given constructor has only one representation, not one per instantiation, so a polymorphic field is always represented by a pointer. There's a good reason for this: if a constructor had multiple representations, it would require compiling multiple versions of code that pattern matched on it, and things get quite complicated. It's not impossible - I believe .NET does this, but GHC doesn't. What you're seeing here is the magic of automatic specialisation. When the Vec type and instances are in the same module as the use, GHC can see what type the instances are being used at, and can create specialised instances, which it does. Then the strictness analyser kicks in and everything is unboxed. When the use of the instance is in a separate module, GHC cannot specialise, because it doesn't know what type to specialise at. If you add appropriate {-# SPECIALISE instance #-} pragmas in the POLY_OTHER case, you should be able to get good code.
PS: Also, when the INLINE pragma is used with the Storable instance for the polymorphic data type, it causes heap explosion, even in the same file. Any thoughts here?
I suspect this is defeating the specialisation somehow, but I'm not sure. Cheers, Simon
On Nov 13, 2007 3:45 AM, Simon Marlow
What you're seeing here is the magic of automatic specialisation. When the Vec type and instances are in the same module as the use, GHC can see what type the instances are being used at, and can create specialised instances, which it does. Then the strictness analyser kicks in and everything is unboxed. When the use of the instance is in a separate module, GHC cannot specialise, because it doesn't know what type to specialise at. If you add appropriate {-# SPECIALISE instance #-} pragmas in the POLY_OTHER case, you should be able to get good code.
Yes, SPECIALISE did the trick. Thanks. So, If I have a sizable library using these kinds of data types, do I need to specialize only the exported functions? Or do I also need to specialize the functions internal to the library? As long as everything has inline pragmas, it should only be necessary to specialize the exported functions, correct?
PS: Also, when the INLINE pragma is used with the Storable instance for the polymorphic data type, it causes heap explosion, even in the same file. Any thoughts here?
I suspect this is defeating the specialisation somehow, but I'm not sure.
It turns out it was the default definitions of peekElemOff and pokeElemOff. I only defined peek and poke for my Storable instance, but I used the ElemOff variants. When I provided definitions for those functions as well, things sped right up. Thanks for your help, Scott
participants (2)
-
Scott Dillard -
Simon Marlow