Turning ForceSpecConstr/NoSpecConstr into pragmas?

Hello all, Early last week I was reminded of something, which was that vector/dph depend on the stage2 compiler - this is because both packages use annotations to specify ForceSpecConstr and NoSpecConstr on several key datatypes. For most of our platforms (now including ARM,) this should generally be OK, because we have stage2 and the linker available to support it. But in particular, it makes vector and dph unusable for cross compilers. This might be somewhat problematic for e.g. iOS or an RPi, where we only have a stage1 cross compiler - but it's reasonable to assume we may want to use vector there! And more and more libraries depend on vector these days. I believe these are the only instances in which vector/dph needs stage2. So I ask: is it reasonable to change this to a pragma built into the compiler? That is, ------------------------------------------------------------ data SPEC = SPEC | SPEC2 {-# ANN type SPEC ForceSpecConstr #-} data PArray a = PArray Int# (PData a) {-# ANN type PArray NoSpecConstr #-} ------------------------------------------------------------- becomes something like: ------------------------------------------------------------- data SPEC = SPEC | SPEC2 {-# SPECIALIZE Constructor SPEC #-} data PArray a = PArray Int# (PData a) {-# NOSPECIALIZE Constructor PArray #-} ------------------------------------------------------------- I'm not particularly interested in a bikeshedding discussion about the exact syntax for the pragma (although this somewhat falls in line with 'INLINE ConLike' as a special case,) - I just want to know if this sounds reasonable. Looking at SpecConstr in the compiler, there seems to be quite a lot of note summarising that we need a better design - in particular, notes about nuking NoSpecConstr (as it appeared before ForceSpecConstr,) and turning ForceSpecConstr into a library type of some sort. I don't propose changing any of this really, just removing the dependency on the annotations. But if someone thinks a library type would be better suited for this - I'm totally fine with that too and am all-ears for a suggestion. And of course, both of these can continue to be supported for a while, although the patches to vector, at least, would be trivial to switch it over. Ben, Manuel, Simon - you three are the experts here I believe. Thoughts? Perhaps I'm missing something key here? -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

I have always wondered why this stuff was an annotation instead of a pragma in the first place. Making it a pragma sounds like a great idea to me, but I have no idea whether or not there's a good reason to make ForceSpecConstr an annotation instead. Geoff On 10/09/2013 10:13 AM, Austin Seipp wrote:
Hello all,
Early last week I was reminded of something, which was that vector/dph depend on the stage2 compiler - this is because both packages use annotations to specify ForceSpecConstr and NoSpecConstr on several key datatypes.
For most of our platforms (now including ARM,) this should generally be OK, because we have stage2 and the linker available to support it.
But in particular, it makes vector and dph unusable for cross compilers. This might be somewhat problematic for e.g. iOS or an RPi, where we only have a stage1 cross compiler - but it's reasonable to assume we may want to use vector there! And more and more libraries depend on vector these days.
I believe these are the only instances in which vector/dph needs stage2. So I ask: is it reasonable to change this to a pragma built into the compiler? That is,
------------------------------------------------------------ data SPEC = SPEC | SPEC2 {-# ANN type SPEC ForceSpecConstr #-}
data PArray a = PArray Int# (PData a) {-# ANN type PArray NoSpecConstr #-} -------------------------------------------------------------
becomes something like:
------------------------------------------------------------- data SPEC = SPEC | SPEC2 {-# SPECIALIZE Constructor SPEC #-}
data PArray a = PArray Int# (PData a) {-# NOSPECIALIZE Constructor PArray #-} -------------------------------------------------------------
I'm not particularly interested in a bikeshedding discussion about the exact syntax for the pragma (although this somewhat falls in line with 'INLINE ConLike' as a special case,) - I just want to know if this sounds reasonable.
Looking at SpecConstr in the compiler, there seems to be quite a lot of note summarising that we need a better design - in particular, notes about nuking NoSpecConstr (as it appeared before ForceSpecConstr,) and turning ForceSpecConstr into a library type of some sort. I don't propose changing any of this really, just removing the dependency on the annotations.
But if someone thinks a library type would be better suited for this - I'm totally fine with that too and am all-ears for a suggestion.
And of course, both of these can continue to be supported for a while, although the patches to vector, at least, would be trivial to switch it over.
Ben, Manuel, Simon - you three are the experts here I believe. Thoughts? Perhaps I'm missing something key here?

+1 for pragma, though the SPEC type is always the same anyway... is it exported from somewhere? It seems silly that I have to define and annotate my own version when I want to use it. I realize that doesn't solve your stage 1 problem... more of just a general question.

Hm, no, it's not exported anywhere currently. It could be exported
from somewhere and marked with such a pragma, although then presumably
we'd also need to persist that pragma to the interface file so client
code sees it.
This actually circles back to making some kind of library types to
enforce this, e.g. making SPEC a builtin type specifically for this
case (it would also remove the annoying gotcha that SPEC must be a
sum-type where SPEC2 is unused, so the compiler doesn't outright
eliminate it.) Although I don't know about the NoSpecConstr case -
since that tends to be enforced on several different types in e.g.
DPH. Presumably though, as the comments say - maybe we should just
nuke NoSpecConstr, which would make that a non-issue. Then SPEC can be
a special built-in type.
On Wed, Oct 9, 2013 at 9:30 AM, Andrew Farmer
+1 for pragma, though the SPEC type is always the same anyway... is it exported from somewhere? It seems silly that I have to define and annotate my own version when I want to use it. I realize that doesn't solve your stage 1 problem... more of just a general question.
-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

On Wed, Oct 9, 2013 at 7:13 AM, Austin Seipp
Early last week I was reminded of something, which was that vector/dph depend on the stage2 compiler - this is because both packages use annotations to specify ForceSpecConstr and NoSpecConstr on several key datatypes.
For most of our platforms (now including ARM,) this should generally be OK, because we have stage2 and the linker available to support it.
But in particular, it makes vector and dph unusable for cross compilers. This might be somewhat problematic for e.g. iOS or an RPi, where we only have a stage1 cross compiler - but it's reasonable to assume we may want to use vector there! And more and more libraries depend on vector these days.
I raised the same issue with Simon PJ a while ago. We ran into this problem when trying use GHC as a cross-compiler at Google. I think we should 1. Convert it to a pragma. 2. Look into a design that was suggested at ICFP, namely to put a pragma on the data type we want to get optimized away instead. -- Johan

Wouldn't #2 mean we'd have to annotate (via pragma) every type we want to
eliminate, rather than a single type?
I think the current design serves the need: "I want you to try really hard
on this function." ... which seems useful.
#2 seems like an orthogonal issue: "I want to ensure this type is optimized
away" ... which also seems useful, but I'm not sure it replaces the first
use. (Though I missed the suggestion at ICFP... is this the ephemeral types
one?)
On Wed, Oct 9, 2013 at 9:55 AM, Johan Tibell
On Wed, Oct 9, 2013 at 7:13 AM, Austin Seipp
wrote: Early last week I was reminded of something, which was that vector/dph depend on the stage2 compiler - this is because both packages use annotations to specify ForceSpecConstr and NoSpecConstr on several key datatypes.
For most of our platforms (now including ARM,) this should generally be OK, because we have stage2 and the linker available to support it.
But in particular, it makes vector and dph unusable for cross compilers. This might be somewhat problematic for e.g. iOS or an RPi, where we only have a stage1 cross compiler - but it's reasonable to assume we may want to use vector there! And more and more libraries depend on vector these days.
I raised the same issue with Simon PJ a while ago. We ran into this problem when trying use GHC as a cross-compiler at Google. I think we should
1. Convert it to a pragma. 2. Look into a design that was suggested at ICFP, namely to put a pragma on the data type we want to get optimized away instead.
-- Johan _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

This insight is this: in stream fusion what we're trying to SpecConstr
away is the intermediate stream state constructor, so why not just say
that, instead of annotating every function. By annotation the Stream
constructor we're more explicit in what we want to happen. In fact, we
could even have GHC omit a warning if it couldn't eliminate the
constructor!
On Wed, Oct 9, 2013 at 8:03 AM, Andrew Farmer
Wouldn't #2 mean we'd have to annotate (via pragma) every type we want to eliminate, rather than a single type?
I think the current design serves the need: "I want you to try really hard on this function." ... which seems useful.
#2 seems like an orthogonal issue: "I want to ensure this type is optimized away" ... which also seems useful, but I'm not sure it replaces the first use. (Though I missed the suggestion at ICFP... is this the ephemeral types one?)
On Wed, Oct 9, 2013 at 9:55 AM, Johan Tibell
wrote: On Wed, Oct 9, 2013 at 7:13 AM, Austin Seipp
wrote: Early last week I was reminded of something, which was that vector/dph depend on the stage2 compiler - this is because both packages use annotations to specify ForceSpecConstr and NoSpecConstr on several key datatypes.
For most of our platforms (now including ARM,) this should generally be OK, because we have stage2 and the linker available to support it.
But in particular, it makes vector and dph unusable for cross compilers. This might be somewhat problematic for e.g. iOS or an RPi, where we only have a stage1 cross compiler - but it's reasonable to assume we may want to use vector there! And more and more libraries depend on vector these days.
I raised the same issue with Simon PJ a while ago. We ran into this problem when trying use GHC as a cross-compiler at Google. I think we should
1. Convert it to a pragma. 2. Look into a design that was suggested at ICFP, namely to put a pragma on the data type we want to get optimized away instead.
-- Johan _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

This feature was implemented as an annotation by Roman in part because Simon was keen to see the then new annotation feature used, in part because we were unsure whether the design would last, and it part as it seemed easier than hacking it into GHC.
Personally, I would have always preferred it to be a proper pragma, mainly for the reason that causes grief now (i.e., because it requires stage2). So, as far as I'm concerned, please make it a pragma.
Manuel
Austin Seipp
Hello all,
Early last week I was reminded of something, which was that vector/dph depend on the stage2 compiler - this is because both packages use annotations to specify ForceSpecConstr and NoSpecConstr on several key datatypes.
For most of our platforms (now including ARM,) this should generally be OK, because we have stage2 and the linker available to support it.
But in particular, it makes vector and dph unusable for cross compilers. This might be somewhat problematic for e.g. iOS or an RPi, where we only have a stage1 cross compiler - but it's reasonable to assume we may want to use vector there! And more and more libraries depend on vector these days.
I believe these are the only instances in which vector/dph needs stage2. So I ask: is it reasonable to change this to a pragma built into the compiler? That is,
------------------------------------------------------------ data SPEC = SPEC | SPEC2 {-# ANN type SPEC ForceSpecConstr #-}
data PArray a = PArray Int# (PData a) {-# ANN type PArray NoSpecConstr #-} -------------------------------------------------------------
becomes something like:
------------------------------------------------------------- data SPEC = SPEC | SPEC2 {-# SPECIALIZE Constructor SPEC #-}
data PArray a = PArray Int# (PData a) {-# NOSPECIALIZE Constructor PArray #-} -------------------------------------------------------------
I'm not particularly interested in a bikeshedding discussion about the exact syntax for the pragma (although this somewhat falls in line with 'INLINE ConLike' as a special case,) - I just want to know if this sounds reasonable.
Looking at SpecConstr in the compiler, there seems to be quite a lot of note summarising that we need a better design - in particular, notes about nuking NoSpecConstr (as it appeared before ForceSpecConstr,) and turning ForceSpecConstr into a library type of some sort. I don't propose changing any of this really, just removing the dependency on the annotations.
But if someone thinks a library type would be better suited for this - I'm totally fine with that too and am all-ears for a suggestion.
And of course, both of these can continue to be supported for a while, although the patches to vector, at least, would be trivial to switch it over.
Ben, Manuel, Simon - you three are the experts here I believe. Thoughts? Perhaps I'm missing something key here?
-- Regards,
Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/ _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

It's true that I suggested making it an annotation. The DPH libraries use a lot of Template Haskell and so have to be compiled with a stage2 compiler anyway.
The thing about ForceSpecConstr is that it is an unprincipled hack that I hate with a passion. It clearly is not the Right Thing. I just don't yet know a better way to do it. Johan suggests a more principled approach, about eliminating uses of the stream constructor. I know that Roman considered that but could not make it work. I'm afraid I can't remember why.
Because it is such a hack I'm reluctant to bake it more deeply into the compiler, and to sink further effort into doing so. Also I'm not sure what problem we are trying to solve here. If it's compiling DPH libraries with stage1, that won't work because they use TH.
All that said, I don't seriously object to someone making it a pragma if you want. Just make clear that it's a horrible hack.
Simon
| -----Original Message-----
| From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Manuel
| M T Chakravarty
| Sent: 10 October 2013 04:03
| To: Austin Seipp
| Cc: Roman Leshchinskiy; ghc-devs@haskell.org
| Subject: Re: Turning ForceSpecConstr/NoSpecConstr into pragmas?
|
| This feature was implemented as an annotation by Roman in part because
| Simon was keen to see the then new annotation feature used, in part
| because we were unsure whether the design would last, and it part as it
| seemed easier than hacking it into GHC.
|
| Personally, I would have always preferred it to be a proper pragma,
| mainly for the reason that causes grief now (i.e., because it requires
| stage2). So, as far as I'm concerned, please make it a pragma.
|
| Manuel
|
| Austin Seipp

It's also used by vector, which is widely deployed and, I think, doesn't use TH otherwise.
Manuel
Simon Peyton-Jones
It's true that I suggested making it an annotation. The DPH libraries use a lot of Template Haskell and so have to be compiled with a stage2 compiler anyway.
The thing about ForceSpecConstr is that it is an unprincipled hack that I hate with a passion. It clearly is not the Right Thing. I just don't yet know a better way to do it. Johan suggests a more principled approach, about eliminating uses of the stream constructor. I know that Roman considered that but could not make it work. I'm afraid I can't remember why.
Because it is such a hack I'm reluctant to bake it more deeply into the compiler, and to sink further effort into doing so. Also I'm not sure what problem we are trying to solve here. If it's compiling DPH libraries with stage1, that won't work because they use TH.
All that said, I don't seriously object to someone making it a pragma if you want. Just make clear that it's a horrible hack.
Simon
| -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Manuel | M T Chakravarty | Sent: 10 October 2013 04:03 | To: Austin Seipp | Cc: Roman Leshchinskiy; ghc-devs@haskell.org | Subject: Re: Turning ForceSpecConstr/NoSpecConstr into pragmas? | | This feature was implemented as an annotation by Roman in part because | Simon was keen to see the then new annotation feature used, in part | because we were unsure whether the design would last, and it part as it | seemed easier than hacking it into GHC. | | Personally, I would have always preferred it to be a proper pragma, | mainly for the reason that causes grief now (i.e., because it requires | stage2). So, as far as I'm concerned, please make it a pragma. | | Manuel | | Austin Seipp
: | > Hello all, | > | > Early last week I was reminded of something, which was that vector/dph | > depend on the stage2 compiler - this is because both packages use | > annotations to specify ForceSpecConstr and NoSpecConstr on several key | > datatypes. | > | > For most of our platforms (now including ARM,) this should generally | > be OK, because we have stage2 and the linker available to support it. | > | > But in particular, it makes vector and dph unusable for cross | > compilers. This might be somewhat problematic for e.g. iOS or an RPi, | > where we only have a stage1 cross compiler - but it's reasonable to | > assume we may want to use vector there! And more and more libraries | > depend on vector these days. | > | > I believe these are the only instances in which vector/dph needs | > stage2. So I ask: is it reasonable to change this to a pragma built | > into the compiler? That is, | > | > ------------------------------------------------------------ | > data SPEC = SPEC | SPEC2 | > {-# ANN type SPEC ForceSpecConstr #-} | > | > data PArray a | > = PArray Int# (PData a) | > {-# ANN type PArray NoSpecConstr #-} | > ------------------------------------------------------------- | > | > becomes something like: | > | > ------------------------------------------------------------- | > data SPEC = SPEC | SPEC2 | > {-# SPECIALIZE Constructor SPEC #-} | > | > data PArray a | > = PArray Int# (PData a) | > {-# NOSPECIALIZE Constructor PArray #-} | > ------------------------------------------------------------- | > | > I'm not particularly interested in a bikeshedding discussion about the | > exact syntax for the pragma (although this somewhat falls in line with | > 'INLINE ConLike' as a special case,) - I just want to know if this | > sounds reasonable. | > | > Looking at SpecConstr in the compiler, there seems to be quite a lot | > of note summarising that we need a better design - in particular, | > notes about nuking NoSpecConstr (as it appeared before | > ForceSpecConstr,) and turning ForceSpecConstr into a library type of | > some sort. I don't propose changing any of this really, just removing | > the dependency on the annotations. | > | > But if someone thinks a library type would be better suited for this - | > I'm totally fine with that too and am all-ears for a suggestion. | > | > And of course, both of these can continue to be supported for a while, | > although the patches to vector, at least, would be trivial to switch | > it over. | > | > Ben, Manuel, Simon - you three are the experts here I believe. | > Thoughts? Perhaps I'm missing something key here? | > | > -- | > Regards, | > | > Austin Seipp, Haskell Consultant | > Well-Typed LLP, http://www.well-typed.com/ | > _______________________________________________ | > ghc-devs mailing list | > ghc-devs@haskell.org | > http://www.haskell.org/mailman/listinfo/ghc-devs | | _______________________________________________ | ghc-devs mailing list | ghc-devs@haskell.org | http://www.haskell.org/mailman/listinfo/ghc-devs _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

On Thu, Oct 10, 2013 at 2:46 AM, Manuel M T Chakravarty < chak@cse.unsw.edu.au> wrote:
It's also used by vector, which is widely deployed and, I think, doesn't use TH otherwise.
Not bring able to compile vector with a cross compiler is the problem we had.
Manuel
It's true that I suggested making it an annotation. The DPH libraries use a lot of Template Haskell and so have to be compiled with a stage2 compiler anyway.
The thing about ForceSpecConstr is that it is an unprincipled hack that I hate with a passion. It clearly is not the Right Thing. I just don't yet know a better way to do it. Johan suggests a more principled approach, about eliminating uses of the stream constructor. I know that Roman considered that but could not make it work. I'm afraid I can't remember why.
Because it is such a hack I'm reluctant to bake it more deeply into the compiler, and to sink further effort into doing so. Also I'm not sure what
Simon Peyton-Jones
: problem we are trying to solve here. If it's compiling DPH libraries with stage1, that won't work because they use TH. All that said, I don't seriously object to someone making it a pragma if
you want. Just make clear that it's a horrible hack.
Simon
| -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of
| M T Chakravarty | Sent: 10 October 2013 04:03 | To: Austin Seipp | Cc: Roman Leshchinskiy; ghc-devs@haskell.org | Subject: Re: Turning ForceSpecConstr/NoSpecConstr into pragmas? | | This feature was implemented as an annotation by Roman in part because | Simon was keen to see the then new annotation feature used, in part | because we were unsure whether the design would last, and it part as it | seemed easier than hacking it into GHC. | | Personally, I would have always preferred it to be a proper pragma, | mainly for the reason that causes grief now (i.e., because it requires | stage2). So, as far as I'm concerned, please make it a pragma. | | Manuel | | Austin Seipp
: | > Hello all, | > | > Early last week I was reminded of something, which was that vector/dph | > depend on the stage2 compiler - this is because both packages use | > annotations to specify ForceSpecConstr and NoSpecConstr on several key | > datatypes. | > | > For most of our platforms (now including ARM,) this should generally | > be OK, because we have stage2 and the linker available to support it. | > | > But in particular, it makes vector and dph unusable for cross | > compilers. This might be somewhat problematic for e.g. iOS or an RPi, | > where we only have a stage1 cross compiler - but it's reasonable to | > assume we may want to use vector there! And more and more libraries | > depend on vector these days. | > | > I believe these are the only instances in which vector/dph needs | > stage2. So I ask: is it reasonable to change this to a pragma built | > into the compiler? That is, | > | > ------------------------------------------------------------ | > data SPEC = SPEC | SPEC2 | > {-# ANN type SPEC ForceSpecConstr #-} | > | > data PArray a | > = PArray Int# (PData a) | > {-# ANN type PArray NoSpecConstr #-} | > ------------------------------------------------------------- | > | > becomes something like: | > | > ------------------------------------------------------------- | > data SPEC = SPEC | SPEC2 | > {-# SPECIALIZE Constructor SPEC #-} | > | > data PArray a | > = PArray Int# (PData a) | > {-# NOSPECIALIZE Constructor PArray #-} | > ------------------------------------------------------------- | > | > I'm not particularly interested in a bikeshedding discussion about | > exact syntax for the pragma (although this somewhat falls in line with | > 'INLINE ConLike' as a special case,) - I just want to know if this | > sounds reasonable. | > | > Looking at SpecConstr in the compiler, there seems to be quite a lot | > of note summarising that we need a better design - in particular, | > notes about nuking NoSpecConstr (as it appeared before | > ForceSpecConstr,) and turning ForceSpecConstr into a library type of | > some sort. I don't propose changing any of this really, just removing | > the dependency on the annotations. | > | > But if someone thinks a library type would be better suited for this
Manuel the -
| > I'm totally fine with that too and am all-ears for a suggestion. | > | > And of course, both of these can continue to be supported for a while, | > although the patches to vector, at least, would be trivial to switch | > it over. | > | > Ben, Manuel, Simon - you three are the experts here I believe. | > Thoughts? Perhaps I'm missing something key here? | > | > -- | > Regards, | > | > Austin Seipp, Haskell Consultant | > Well-Typed LLP, http://www.well-typed.com/ | > _______________________________________________ | > ghc-devs mailing list | > ghc-devs@haskell.org | > http://www.haskell.org/mailman/listinfo/ghc-devs | | _______________________________________________ | ghc-devs mailing list | ghc-devs@haskell.org | http://www.haskell.org/mailman/listinfo/ghc-devs _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

(Blagh, sorry for the double-email to those involved. I forgot to use
my email for this list. Verbatim response below.)
-------
Hi all,
It seems there seems to be some consensus about making some changes to
ForceSpecConstr. Indeed, cross compilation of vector is, I think, a
Big Deal, and as people begin deploying cross compilers more, this
will probably become more of an issue - according to
http://packdeps.haskellers.com/reverse/vector it has almost 400
reverse dependencies! And this includes some packages - like aeson,
io-streams, mwc-random, statistics - that are quite a big deal to a
lot of people.
I understand that ForceSpecConstr is a bit of a hack (and I, of
course, don't know how to fix it either :) And note that I was
intentionally conservative in my original proposal to not introduce a
lot more stuff or really shift things around - preferably, this would
be easily doable in the 7.8 timeframe with minimal changes to vector
itself.
So all that said, let's take DPH and NoSpecConstr out of the picture for now.
After thinking about it more, rather than threading a pragma
throughout the compiler, I'm coming around to the point Andrew brought
up at first - make SPEC something of a built-in library type (perhaps
exported from ghc-prim,) which the SpecConstr pass deals with
specifically. This is close to what we do now, we just look at the
ForceSpecConstr data type instead in the SpecConstr pass. So, this
means
A) we don't add any new syntax, just a new wired-in for SPEC, which
might require a little fiddling.
B) Vector can be upgraded with minimal changes (and #ifdef'd
appropriately for older compilers, since they don't really support
cross compilation that well.)
Does this sound acceptable to those interested? If there aren't any
complaints I can file a ticket pertaining to this.
On Thu, Oct 10, 2013 at 9:16 AM, Johan Tibell
On Thu, Oct 10, 2013 at 2:46 AM, Manuel M T Chakravarty
wrote: It's also used by vector, which is widely deployed and, I think, doesn't use TH otherwise.
Not bring able to compile vector with a cross compiler is the problem we had.
Manuel
Simon Peyton-Jones
: It's true that I suggested making it an annotation. The DPH libraries use a lot of Template Haskell and so have to be compiled with a stage2 compiler anyway.
The thing about ForceSpecConstr is that it is an unprincipled hack that I hate with a passion. It clearly is not the Right Thing. I just don't yet know a better way to do it. Johan suggests a more principled approach, about eliminating uses of the stream constructor. I know that Roman considered that but could not make it work. I'm afraid I can't remember why.
Because it is such a hack I'm reluctant to bake it more deeply into the compiler, and to sink further effort into doing so. Also I'm not sure what problem we are trying to solve here. If it's compiling DPH libraries with stage1, that won't work because they use TH.
All that said, I don't seriously object to someone making it a pragma if you want. Just make clear that it's a horrible hack.
Simon
| -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Manuel | M T Chakravarty | Sent: 10 October 2013 04:03 | To: Austin Seipp | Cc: Roman Leshchinskiy; ghc-devs@haskell.org | Subject: Re: Turning ForceSpecConstr/NoSpecConstr into pragmas? | | This feature was implemented as an annotation by Roman in part because | Simon was keen to see the then new annotation feature used, in part | because we were unsure whether the design would last, and it part as it | seemed easier than hacking it into GHC. | | Personally, I would have always preferred it to be a proper pragma, | mainly for the reason that causes grief now (i.e., because it requires | stage2). So, as far as I'm concerned, please make it a pragma. | | Manuel | | Austin Seipp
: | > Hello all, | > | > Early last week I was reminded of something, which was that vector/dph | > depend on the stage2 compiler - this is because both packages use | > annotations to specify ForceSpecConstr and NoSpecConstr on several key | > datatypes. | > | > For most of our platforms (now including ARM,) this should generally | > be OK, because we have stage2 and the linker available to support it. | > | > But in particular, it makes vector and dph unusable for cross | > compilers. This might be somewhat problematic for e.g. iOS or an RPi, | > where we only have a stage1 cross compiler - but it's reasonable to | > assume we may want to use vector there! And more and more libraries | > depend on vector these days. | > | > I believe these are the only instances in which vector/dph needs | > stage2. So I ask: is it reasonable to change this to a pragma built | > into the compiler? That is, | > | > ------------------------------------------------------------ | > data SPEC = SPEC | SPEC2 | > {-# ANN type SPEC ForceSpecConstr #-} | > | > data PArray a | > = PArray Int# (PData a) | > {-# ANN type PArray NoSpecConstr #-} | > ------------------------------------------------------------- | > | > becomes something like: | > | > ------------------------------------------------------------- | > data SPEC = SPEC | SPEC2 | > {-# SPECIALIZE Constructor SPEC #-} | > | > data PArray a | > = PArray Int# (PData a) | > {-# NOSPECIALIZE Constructor PArray #-} | > ------------------------------------------------------------- | > | > I'm not particularly interested in a bikeshedding discussion about the | > exact syntax for the pragma (although this somewhat falls in line with | > 'INLINE ConLike' as a special case,) - I just want to know if this | > sounds reasonable. | > | > Looking at SpecConstr in the compiler, there seems to be quite a lot | > of note summarising that we need a better design - in particular, | > notes about nuking NoSpecConstr (as it appeared before | > ForceSpecConstr,) and turning ForceSpecConstr into a library type of | > some sort. I don't propose changing any of this really, just removing | > the dependency on the annotations. | > | > But if someone thinks a library type would be better suited for this - | > I'm totally fine with that too and am all-ears for a suggestion. | > | > And of course, both of these can continue to be supported for a while, | > although the patches to vector, at least, would be trivial to switch | > it over. | > | > Ben, Manuel, Simon - you three are the experts here I believe. | > Thoughts? Perhaps I'm missing something key here? | > | > -- | > Regards, | > | > Austin Seipp, Haskell Consultant | > Well-Typed LLP, http://www.well-typed.com/ | > _______________________________________________ | > ghc-devs mailing list | > ghc-devs@haskell.org | > http://www.haskell.org/mailman/listinfo/ghc-devs | | _______________________________________________ | ghc-devs mailing list | ghc-devs@haskell.org | http://www.haskell.org/mailman/listinfo/ghc-devs _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

[First attempt sent from wrong account.]
The real solution, in the general scheme of things, would be to make TH work with cross-compilation. I think, the ghc-iOS folks do have a concrete plan for that (i.e., a cross-compiler needs to generate two binaries, one for the target arch and one for the host arch).
Manuel
Manuel M T Chakravarty
It's also used by vector, which is widely deployed and, I think, doesn't use TH otherwise.
Manuel
Simon Peyton-Jones
: It's true that I suggested making it an annotation. The DPH libraries use a lot of Template Haskell and so have to be compiled with a stage2 compiler anyway.
The thing about ForceSpecConstr is that it is an unprincipled hack that I hate with a passion. It clearly is not the Right Thing. I just don't yet know a better way to do it. Johan suggests a more principled approach, about eliminating uses of the stream constructor. I know that Roman considered that but could not make it work. I'm afraid I can't remember why.
Because it is such a hack I'm reluctant to bake it more deeply into the compiler, and to sink further effort into doing so. Also I'm not sure what problem we are trying to solve here. If it's compiling DPH libraries with stage1, that won't work because they use TH.
All that said, I don't seriously object to someone making it a pragma if you want. Just make clear that it's a horrible hack.
Simon
| -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Manuel | M T Chakravarty | Sent: 10 October 2013 04:03 | To: Austin Seipp | Cc: Roman Leshchinskiy; ghc-devs@haskell.org | Subject: Re: Turning ForceSpecConstr/NoSpecConstr into pragmas? | | This feature was implemented as an annotation by Roman in part because | Simon was keen to see the then new annotation feature used, in part | because we were unsure whether the design would last, and it part as it | seemed easier than hacking it into GHC. | | Personally, I would have always preferred it to be a proper pragma, | mainly for the reason that causes grief now (i.e., because it requires | stage2). So, as far as I'm concerned, please make it a pragma. | | Manuel | | Austin Seipp
: | > Hello all, | > | > Early last week I was reminded of something, which was that vector/dph | > depend on the stage2 compiler - this is because both packages use | > annotations to specify ForceSpecConstr and NoSpecConstr on several key | > datatypes. | > | > For most of our platforms (now including ARM,) this should generally | > be OK, because we have stage2 and the linker available to support it. | > | > But in particular, it makes vector and dph unusable for cross | > compilers. This might be somewhat problematic for e.g. iOS or an RPi, | > where we only have a stage1 cross compiler - but it's reasonable to | > assume we may want to use vector there! And more and more libraries | > depend on vector these days. | > | > I believe these are the only instances in which vector/dph needs | > stage2. So I ask: is it reasonable to change this to a pragma built | > into the compiler? That is, | > | > ------------------------------------------------------------ | > data SPEC = SPEC | SPEC2 | > {-# ANN type SPEC ForceSpecConstr #-} | > | > data PArray a | > = PArray Int# (PData a) | > {-# ANN type PArray NoSpecConstr #-} | > ------------------------------------------------------------- | > | > becomes something like: | > | > ------------------------------------------------------------- | > data SPEC = SPEC | SPEC2 | > {-# SPECIALIZE Constructor SPEC #-} | > | > data PArray a | > = PArray Int# (PData a) | > {-# NOSPECIALIZE Constructor PArray #-} | > ------------------------------------------------------------- | > | > I'm not particularly interested in a bikeshedding discussion about the | > exact syntax for the pragma (although this somewhat falls in line with | > 'INLINE ConLike' as a special case,) - I just want to know if this | > sounds reasonable. | > | > Looking at SpecConstr in the compiler, there seems to be quite a lot | > of note summarising that we need a better design - in particular, | > notes about nuking NoSpecConstr (as it appeared before | > ForceSpecConstr,) and turning ForceSpecConstr into a library type of | > some sort. I don't propose changing any of this really, just removing | > the dependency on the annotations. | > | > But if someone thinks a library type would be better suited for this - | > I'm totally fine with that too and am all-ears for a suggestion. | > | > And of course, both of these can continue to be supported for a while, | > although the patches to vector, at least, would be trivial to switch | > it over. | > | > Ben, Manuel, Simon - you three are the experts here I believe. | > Thoughts? Perhaps I'm missing something key here? | > | > -- | > Regards, | > | > Austin Seipp, Haskell Consultant | > Well-Typed LLP, http://www.well-typed.com/ | > _______________________________________________ | > ghc-devs mailing list | > ghc-devs@haskell.org | > http://www.haskell.org/mailman/listinfo/ghc-devs | | _______________________________________________ | ghc-devs mailing list | ghc-devs@haskell.org | http://www.haskell.org/mailman/listinfo/ghc-devs _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Yes, I agree, but it's quite a bit of work to do this. I believe the
plan of attack laid out at one point was: compile both for the host,
and the target. Then, the cross-stage2 compiler when needing e.g. to
load template haskell code, can load copies of the *host* libraries
instead into the GHC process, and use them to emit code for the final
binary, for the target platform.
Unfortunately the naive approach to this would fall on its face in
some cases I think - consider cross compiling x86_64 -> ARMv7 --
`sizeOf (undefined :: Ptr a)` will be different across the two, so
loading the host libraries would imply a pointer size of 8 bytes,
while the target actually has a pointer size of 4 bytes. If you used
Template Haskell to generate code on this example (for example
emitting the ptr size into the resulting binary as a literal integer)
then it'll actually be subtly wrong.
On Thu, Oct 10, 2013 at 6:25 PM, Manuel M T Chakravarty
[First attempt sent from wrong account.]
The real solution, in the general scheme of things, would be to make TH work with cross-compilation. I think, the ghc-iOS folks do have a concrete plan for that (i.e., a cross-compiler needs to generate two binaries, one for the target arch and one for the host arch).
Manuel
Manuel M T Chakravarty
: It's also used by vector, which is widely deployed and, I think, doesn't use TH otherwise.
Manuel
Simon Peyton-Jones
: It's true that I suggested making it an annotation. The DPH libraries use a lot of Template Haskell and so have to be compiled with a stage2 compiler anyway.
The thing about ForceSpecConstr is that it is an unprincipled hack that I hate with a passion. It clearly is not the Right Thing. I just don't yet know a better way to do it. Johan suggests a more principled approach, about eliminating uses of the stream constructor. I know that Roman considered that but could not make it work. I'm afraid I can't remember why.
Because it is such a hack I'm reluctant to bake it more deeply into the compiler, and to sink further effort into doing so. Also I'm not sure what problem we are trying to solve here. If it's compiling DPH libraries with stage1, that won't work because they use TH.
All that said, I don't seriously object to someone making it a pragma if you want. Just make clear that it's a horrible hack.
Simon
| -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Manuel | M T Chakravarty | Sent: 10 October 2013 04:03 | To: Austin Seipp | Cc: Roman Leshchinskiy; ghc-devs@haskell.org | Subject: Re: Turning ForceSpecConstr/NoSpecConstr into pragmas? | | This feature was implemented as an annotation by Roman in part because | Simon was keen to see the then new annotation feature used, in part | because we were unsure whether the design would last, and it part as it | seemed easier than hacking it into GHC. | | Personally, I would have always preferred it to be a proper pragma, | mainly for the reason that causes grief now (i.e., because it requires | stage2). So, as far as I'm concerned, please make it a pragma. | | Manuel | | Austin Seipp
: | > Hello all, | > | > Early last week I was reminded of something, which was that vector/dph | > depend on the stage2 compiler - this is because both packages use | > annotations to specify ForceSpecConstr and NoSpecConstr on several key | > datatypes. | > | > For most of our platforms (now including ARM,) this should generally | > be OK, because we have stage2 and the linker available to support it. | > | > But in particular, it makes vector and dph unusable for cross | > compilers. This might be somewhat problematic for e.g. iOS or an RPi, | > where we only have a stage1 cross compiler - but it's reasonable to | > assume we may want to use vector there! And more and more libraries | > depend on vector these days. | > | > I believe these are the only instances in which vector/dph needs | > stage2. So I ask: is it reasonable to change this to a pragma built | > into the compiler? That is, | > | > ------------------------------------------------------------ | > data SPEC = SPEC | SPEC2 | > {-# ANN type SPEC ForceSpecConstr #-} | > | > data PArray a | > = PArray Int# (PData a) | > {-# ANN type PArray NoSpecConstr #-} | > ------------------------------------------------------------- | > | > becomes something like: | > | > ------------------------------------------------------------- | > data SPEC = SPEC | SPEC2 | > {-# SPECIALIZE Constructor SPEC #-} | > | > data PArray a | > = PArray Int# (PData a) | > {-# NOSPECIALIZE Constructor PArray #-} | > ------------------------------------------------------------- | > | > I'm not particularly interested in a bikeshedding discussion about the | > exact syntax for the pragma (although this somewhat falls in line with | > 'INLINE ConLike' as a special case,) - I just want to know if this | > sounds reasonable. | > | > Looking at SpecConstr in the compiler, there seems to be quite a lot | > of note summarising that we need a better design - in particular, | > notes about nuking NoSpecConstr (as it appeared before | > ForceSpecConstr,) and turning ForceSpecConstr into a library type of | > some sort. I don't propose changing any of this really, just removing | > the dependency on the annotations. | > | > But if someone thinks a library type would be better suited for this - | > I'm totally fine with that too and am all-ears for a suggestion. | > | > And of course, both of these can continue to be supported for a while, | > although the patches to vector, at least, would be trivial to switch | > it over. | > | > Ben, Manuel, Simon - you three are the experts here I believe. | > Thoughts? Perhaps I'm missing something key here? | > | > -- | > Regards, | > | > Austin Seipp, Haskell Consultant | > Well-Typed LLP, http://www.well-typed.com/ | > _______________________________________________ | > ghc-devs mailing list | > ghc-devs@haskell.org | > http://www.haskell.org/mailman/listinfo/ghc-devs | | _______________________________________________ | ghc-devs mailing list | ghc-devs@haskell.org | http://www.haskell.org/mailman/listinfo/ghc-devs _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

Hi all,
I have now pushed this change - there is a new type exposed from
GHC.Types, in ghc-prim which is (unsurprisingly) named 'SPEC'. The
SpecConstr pass will now explicitly look for both this type occurring
as an argument, OR for a type which has ForceSpecConstr as an
annotation.
This means it should now be possible to cross-compile vector with the
stage1 compiler, once I submit a patch upstream.
I will also add some more documentation shortly.
On Thu, Oct 10, 2013 at 8:01 PM, Austin Seipp
Yes, I agree, but it's quite a bit of work to do this. I believe the plan of attack laid out at one point was: compile both for the host, and the target. Then, the cross-stage2 compiler when needing e.g. to load template haskell code, can load copies of the *host* libraries instead into the GHC process, and use them to emit code for the final binary, for the target platform.
Unfortunately the naive approach to this would fall on its face in some cases I think - consider cross compiling x86_64 -> ARMv7 -- `sizeOf (undefined :: Ptr a)` will be different across the two, so loading the host libraries would imply a pointer size of 8 bytes, while the target actually has a pointer size of 4 bytes. If you used Template Haskell to generate code on this example (for example emitting the ptr size into the resulting binary as a literal integer) then it'll actually be subtly wrong.
On Thu, Oct 10, 2013 at 6:25 PM, Manuel M T Chakravarty
wrote: [First attempt sent from wrong account.]
The real solution, in the general scheme of things, would be to make TH work with cross-compilation. I think, the ghc-iOS folks do have a concrete plan for that (i.e., a cross-compiler needs to generate two binaries, one for the target arch and one for the host arch).
Manuel
Manuel M T Chakravarty
: It's also used by vector, which is widely deployed and, I think, doesn't use TH otherwise.
Manuel
Simon Peyton-Jones
: It's true that I suggested making it an annotation. The DPH libraries use a lot of Template Haskell and so have to be compiled with a stage2 compiler anyway.
The thing about ForceSpecConstr is that it is an unprincipled hack that I hate with a passion. It clearly is not the Right Thing. I just don't yet know a better way to do it. Johan suggests a more principled approach, about eliminating uses of the stream constructor. I know that Roman considered that but could not make it work. I'm afraid I can't remember why.
Because it is such a hack I'm reluctant to bake it more deeply into the compiler, and to sink further effort into doing so. Also I'm not sure what problem we are trying to solve here. If it's compiling DPH libraries with stage1, that won't work because they use TH.
All that said, I don't seriously object to someone making it a pragma if you want. Just make clear that it's a horrible hack.
Simon
| -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Manuel | M T Chakravarty | Sent: 10 October 2013 04:03 | To: Austin Seipp | Cc: Roman Leshchinskiy; ghc-devs@haskell.org | Subject: Re: Turning ForceSpecConstr/NoSpecConstr into pragmas? | | This feature was implemented as an annotation by Roman in part because | Simon was keen to see the then new annotation feature used, in part | because we were unsure whether the design would last, and it part as it | seemed easier than hacking it into GHC. | | Personally, I would have always preferred it to be a proper pragma, | mainly for the reason that causes grief now (i.e., because it requires | stage2). So, as far as I'm concerned, please make it a pragma. | | Manuel | | Austin Seipp
: | > Hello all, | > | > Early last week I was reminded of something, which was that vector/dph | > depend on the stage2 compiler - this is because both packages use | > annotations to specify ForceSpecConstr and NoSpecConstr on several key | > datatypes. | > | > For most of our platforms (now including ARM,) this should generally | > be OK, because we have stage2 and the linker available to support it. | > | > But in particular, it makes vector and dph unusable for cross | > compilers. This might be somewhat problematic for e.g. iOS or an RPi, | > where we only have a stage1 cross compiler - but it's reasonable to | > assume we may want to use vector there! And more and more libraries | > depend on vector these days. | > | > I believe these are the only instances in which vector/dph needs | > stage2. So I ask: is it reasonable to change this to a pragma built | > into the compiler? That is, | > | > ------------------------------------------------------------ | > data SPEC = SPEC | SPEC2 | > {-# ANN type SPEC ForceSpecConstr #-} | > | > data PArray a | > = PArray Int# (PData a) | > {-# ANN type PArray NoSpecConstr #-} | > ------------------------------------------------------------- | > | > becomes something like: | > | > ------------------------------------------------------------- | > data SPEC = SPEC | SPEC2 | > {-# SPECIALIZE Constructor SPEC #-} | > | > data PArray a | > = PArray Int# (PData a) | > {-# NOSPECIALIZE Constructor PArray #-} | > ------------------------------------------------------------- | > | > I'm not particularly interested in a bikeshedding discussion about the | > exact syntax for the pragma (although this somewhat falls in line with | > 'INLINE ConLike' as a special case,) - I just want to know if this | > sounds reasonable. | > | > Looking at SpecConstr in the compiler, there seems to be quite a lot | > of note summarising that we need a better design - in particular, | > notes about nuking NoSpecConstr (as it appeared before | > ForceSpecConstr,) and turning ForceSpecConstr into a library type of | > some sort. I don't propose changing any of this really, just removing | > the dependency on the annotations. | > | > But if someone thinks a library type would be better suited for this - | > I'm totally fine with that too and am all-ears for a suggestion. | > | > And of course, both of these can continue to be supported for a while, | > although the patches to vector, at least, would be trivial to switch | > it over. | > | > Ben, Manuel, Simon - you three are the experts here I believe. | > Thoughts? Perhaps I'm missing something key here? | > | > -- | > Regards, | > | > Austin Seipp, Haskell Consultant | > Well-Typed LLP, http://www.well-typed.com/ | > _______________________________________________ | > ghc-devs mailing list | > ghc-devs@haskell.org | > http://www.haskell.org/mailman/listinfo/ghc-devs | | _______________________________________________ | ghc-devs mailing list | ghc-devs@haskell.org | http://www.haskell.org/mailman/listinfo/ghc-devs _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
-- Regards,
Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/
-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/
participants (6)
-
Andrew Farmer
-
Austin Seipp
-
Geoffrey Mainland
-
Johan Tibell
-
Manuel M T Chakravarty
-
Simon Peyton-Jones