7.8.1, template haskell, and dynamic libraries

The docs for 7.8.1 say "Template Haskell must now load dynamic object files, not static ones". Does this mean that, if I'm using Template Haskell, every package which the templates depend on have to be built with --enable-shared? -- View this message in context: http://haskell.1045720.n5.nabble.com/7-8-1-template-haskell-and-dynamic-libr... Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com.

Yes. (And thence ghc itself is then invoked with dynamic or dynamic-too)
On Saturday, February 8, 2014, harry
The docs for 7.8.1 say "Template Haskell must now load dynamic object files, not static ones". Does this mean that, if I'm using Template Haskell, every package which the templates depend on have to be built with --enable-shared?
-- View this message in context: http://haskell.1045720.n5.nabble.com/7-8-1-template-haskell-and-dynamic-libr... Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com. _______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org javascript:; http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Carter Schonwald wrote
Yes. (And thence ghc itself is then invoked with dynamic or dynamic-too)
The docs for 7.8.1 say "Template Haskell must now load dynamic object files, not static ones". Does this mean that, if I'm using Template Haskell, every package which the templates depend on have to be built with --enable-shared?
Perhaps there should be a clear warning about this? If I've understood correctly, anything using Template Haskell will break when upgrading to 7.8, until all the dependencies are reinstalled with --enable-shared. -- View this message in context: http://haskell.1045720.n5.nabble.com/7-8-1-template-haskell-and-dynamic-libr... Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com.

On Sun, Feb 9, 2014 at 10:30 AM, harry
Carter Schonwald wrote
Yes. (And thence ghc itself is then invoked with dynamic or dynamic-too)
The docs for 7.8.1 say "Template Haskell must now load dynamic object files, not static ones". Does this mean that, if I'm using Template Haskell, every package which the templates depend on have to be built with --enable-shared?
Perhaps there should be a clear warning about this? If I've understood correctly, anything using Template Haskell will break when upgrading to 7.8, until all the dependencies are reinstalled with --enable-shared.
When upgrading to a new version of GHC, you'll have to reinstall all of the packages anyway. You can't simply use GHC 7.6 compiled packages with your new GHC 7.8.

Michael Snoyman wrote
When upgrading to a new version of GHC, you'll have to reinstall all of the packages anyway. You can't simply use GHC 7.6 compiled packages with your new GHC 7.8.
This is probably the point at which it would be useful to know that all this recompilation will have to be done with dynamic libraries, to avoid having to do it all a second time when the user tries to use templates. -- View this message in context: http://haskell.1045720.n5.nabble.com/7-8-1-template-haskell-and-dynamic-libr... Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com.

Is --enable-shared off by default?
On Feb 9, 2014 9:04 AM, "harry"
Michael Snoyman wrote
When upgrading to a new version of GHC, you'll have to reinstall all of the packages anyway. You can't simply use GHC 7.6 compiled packages with your new GHC 7.8.
This is probably the point at which it would be useful to know that all this recompilation will have to be done with dynamic libraries, to avoid having to do it all a second time when the user tries to use templates.
-- View this message in context: http://haskell.1045720.n5.nabble.com/7-8-1-template-haskell-and-dynamic-libr... Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com. _______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

On Sun, Feb 9, 2014 at 9:28 AM, Greg Horn
Is --enable-shared off by default?
It's supposed to be on by default in 7.8. That said, not sure how many people have played with ~/.cabal/config.... -- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

Indeed. The problem is that many folks might have cabal config files that
explicitly disable shared. (For the compile times!). They might need
clear information about wiping that field.
On Sunday, February 9, 2014, Brandon Allbery
On Sun, Feb 9, 2014 at 9:28 AM, Greg Horn
javascript:_e(%7B%7D,'cvml','gregmainland@gmail.com'); wrote:
Is --enable-shared off by default?
It's supposed to be on by default in 7.8. That said, not sure how many people have played with ~/.cabal/config....
-- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com javascript:_e(%7B%7D,'cvml','allbery.b@gmail.com'); ballbery@sinenomine.netjavascript:_e(%7B%7D,'cvml','ballbery@sinenomine.net'); unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

Yes, in general I think the doc needs a section: Incompatible changes. The
hope is that you can take the release and just work as usual but when (for
good reasons as in this release) it is not true is is important to have
such a section. Another case that needs to be there is how to compile so
you can load compile object files into ghci as what you did in 7.6.3 won't
work in this release.
On Sun, Feb 9, 2014 at 1:11 PM, Carter Schonwald wrote: Indeed. The problem is that many folks might have cabal config files that
explicitly disable shared. (For the compile times!). They might need
clear information about wiping that field. On Sunday, February 9, 2014, Brandon Allbery On Sun, Feb 9, 2014 at 9:28 AM, Greg Horn Is --enable-shared off by default? It's supposed to be on by default in 7.8. That said, not sure how many
people have played with ~/.cabal/config.... --
brandon s allbery kf8nh sine nomine
associates
allbery.b@gmail.com
ballbery@sinenomine.net
unix, openafs, kerberos, infrastructure, xmonad
http://sinenomine.net _______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

It is correct that Template Haskell now requires dynamic objects. However, GHC can produce static and dynamic objects at the same time, so you don't have to recompile a package twice (it's a big optimization, basically). Furthermore, if TemplateHaskell is enabled as a requirement for a package, and GHC is built dynamically, Cabal will do The Right Thing by building the shared objects for the dependencies as well. It will save time by doing so using -dynamic-too, if possible. This is because it queries GHC before compilation to figure this out (you can run 'ghc --info' with the 7.8.1 RC to see "GHC Dynamic" and "Supports dynamic-too" fields.) Finally, if you simply run 'ghc Foo.hs' on a file that requires TemplateHaskell, it will also switch on -dynamic-too for the needed objects in this simple case. There is one caveat, if I remember correctly: if a package uses TemplateHaskell, it must declare it as such in the Cabal file. This is because Cabal does not parse the source to detect if TemplateHaskell is needed in the dependency graph of the compiled modules. Only GHC can do this reliably. If you don't specify TemplateHaskell as an extension, Cabal might not do the right thing. This is noted in the release notes:
Note that Cabal will correctly handle -dynamic-too for you automatically, especially when -XTemplateHaskell is needed - but you *must* tell Cabal you are using the TemplateHaskell extension.
However, based on the other suggestions in the thread and confusion
around this, a big "Incompatible changes" section with this listed as
the first thing with clear detail would be a good idea. I'll do so.
If something else is going on, please file a bug.
On Sun, Feb 9, 2014 at 1:37 PM, George Colpitts
Yes, in general I think the doc needs a section: Incompatible changes. The hope is that you can take the release and just work as usual but when (for good reasons as in this release) it is not true is is important to have such a section. Another case that needs to be there is how to compile so you can load compile object files into ghci as what you did in 7.6.3 won't work in this release.
On Sun, Feb 9, 2014 at 1:11 PM, Carter Schonwald
wrote: Indeed. The problem is that many folks might have cabal config files that explicitly disable shared. (For the compile times!). They might need clear information about wiping that field.
On Sunday, February 9, 2014, Brandon Allbery
wrote: On Sun, Feb 9, 2014 at 9:28 AM, Greg Horn
wrote: Is --enable-shared off by default?
It's supposed to be on by default in 7.8. That said, not sure how many people have played with ~/.cabal/config....
-- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

Hi, Am Sonntag, den 09.02.2014, 14:37 -0600 schrieb Austin Seipp:
There is one caveat, if I remember correctly: if a package uses TemplateHaskell, it must declare it as such in the Cabal file. This is because Cabal does not parse the source to detect if TemplateHaskell is needed in the dependency graph of the compiled modules. Only GHC can do this reliably. If you don't specify TemplateHaskell as an extension, Cabal might not do the right thing. This is noted in the release notes:
Note that Cabal will correctly handle -dynamic-too for you automatically, especially when -XTemplateHaskell is needed - but you *must* tell Cabal you are using the TemplateHaskell extension.
we need -dynamic-too also for everything that a user ever might want to load in GHCi, right? So doesn’t that already imply that Cabal should and will build libHS*so files always anyways? Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org

Actually, just to keep it even simpler, so nobody else is confused
further: Cabal will *also* properly turn on dynamic builds for regular
packages if GHC is dynamic, TemplateHaskell or not. So any library you
compile will still work in GHCi as expected.
So here's the breakdown:
1) Cabal 1.18 will handle dynamical GHCi correctly, including
compiling things dynamically, however it must.
2) Per #1, libraries are compiled dynamically. This means libraries
work in GHCi, just like they did.
3) -Executables- are statically linked by default, still. (But
because of #1 and #2, it's very easy to turn on dynamic exes as well,
without needing to recompile a lot.)
4) TemplateHaskell works as expected due to #1 and #2. But there is
one caveat for executables, noted separately below.
The caveat with TemplateHaskell is for executables: This is because if
you end up with an executable that needs TH and profiling, Cabal must
be aware of this. Why? Because GHCi cannot load *profiled* objects,
only normal ones. So we must compile twice: once without profiling,
and once with profiling. The second compilation will use the 'normal'
objects, even though the final executable will be profiled. Cabal
doesn't know to do this if it doesn't know TemplateHaskell is a
requirement.
Does this clear things up? My last message might give the impression
some things aren't compiled dynamically, because I merely ambiguously
referred to 'packages'.
On Sun, Feb 9, 2014 at 2:37 PM, Austin Seipp
It is correct that Template Haskell now requires dynamic objects. However, GHC can produce static and dynamic objects at the same time, so you don't have to recompile a package twice (it's a big optimization, basically).
Furthermore, if TemplateHaskell is enabled as a requirement for a package, and GHC is built dynamically, Cabal will do The Right Thing by building the shared objects for the dependencies as well. It will save time by doing so using -dynamic-too, if possible. This is because it queries GHC before compilation to figure this out (you can run 'ghc --info' with the 7.8.1 RC to see "GHC Dynamic" and "Supports dynamic-too" fields.)
Finally, if you simply run 'ghc Foo.hs' on a file that requires TemplateHaskell, it will also switch on -dynamic-too for the needed objects in this simple case.
There is one caveat, if I remember correctly: if a package uses TemplateHaskell, it must declare it as such in the Cabal file. This is because Cabal does not parse the source to detect if TemplateHaskell is needed in the dependency graph of the compiled modules. Only GHC can do this reliably. If you don't specify TemplateHaskell as an extension, Cabal might not do the right thing. This is noted in the release notes:
Note that Cabal will correctly handle -dynamic-too for you automatically, especially when -XTemplateHaskell is needed - but you *must* tell Cabal you are using the TemplateHaskell extension.
However, based on the other suggestions in the thread and confusion around this, a big "Incompatible changes" section with this listed as the first thing with clear detail would be a good idea. I'll do so.
If something else is going on, please file a bug.
On Sun, Feb 9, 2014 at 1:37 PM, George Colpitts
wrote: Yes, in general I think the doc needs a section: Incompatible changes. The hope is that you can take the release and just work as usual but when (for good reasons as in this release) it is not true is is important to have such a section. Another case that needs to be there is how to compile so you can load compile object files into ghci as what you did in 7.6.3 won't work in this release.
On Sun, Feb 9, 2014 at 1:11 PM, Carter Schonwald
wrote: Indeed. The problem is that many folks might have cabal config files that explicitly disable shared. (For the compile times!). They might need clear information about wiping that field.
On Sunday, February 9, 2014, Brandon Allbery
wrote: On Sun, Feb 9, 2014 at 9:28 AM, Greg Horn
wrote: Is --enable-shared off by default?
It's supposed to be on by default in 7.8. That said, not sure how many people have played with ~/.cabal/config....
-- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
-- Regards,
Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/
-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

I think you can summarise all that with "tl;dr the right thing will happen, you don't have to remember to give any new flags to GHC or Cabal". Cheers, Simon On 09/02/2014 21:14, Austin Seipp wrote:
Actually, just to keep it even simpler, so nobody else is confused further: Cabal will *also* properly turn on dynamic builds for regular packages if GHC is dynamic, TemplateHaskell or not. So any library you compile will still work in GHCi as expected.
So here's the breakdown:
1) Cabal 1.18 will handle dynamical GHCi correctly, including compiling things dynamically, however it must. 2) Per #1, libraries are compiled dynamically. This means libraries work in GHCi, just like they did. 3) -Executables- are statically linked by default, still. (But because of #1 and #2, it's very easy to turn on dynamic exes as well, without needing to recompile a lot.) 4) TemplateHaskell works as expected due to #1 and #2. But there is one caveat for executables, noted separately below.
The caveat with TemplateHaskell is for executables: This is because if you end up with an executable that needs TH and profiling, Cabal must be aware of this. Why? Because GHCi cannot load *profiled* objects, only normal ones. So we must compile twice: once without profiling, and once with profiling. The second compilation will use the 'normal' objects, even though the final executable will be profiled. Cabal doesn't know to do this if it doesn't know TemplateHaskell is a requirement.
Does this clear things up? My last message might give the impression some things aren't compiled dynamically, because I merely ambiguously referred to 'packages'.
On Sun, Feb 9, 2014 at 2:37 PM, Austin Seipp
wrote: It is correct that Template Haskell now requires dynamic objects. However, GHC can produce static and dynamic objects at the same time, so you don't have to recompile a package twice (it's a big optimization, basically).
Furthermore, if TemplateHaskell is enabled as a requirement for a package, and GHC is built dynamically, Cabal will do The Right Thing by building the shared objects for the dependencies as well. It will save time by doing so using -dynamic-too, if possible. This is because it queries GHC before compilation to figure this out (you can run 'ghc --info' with the 7.8.1 RC to see "GHC Dynamic" and "Supports dynamic-too" fields.)
Finally, if you simply run 'ghc Foo.hs' on a file that requires TemplateHaskell, it will also switch on -dynamic-too for the needed objects in this simple case.
There is one caveat, if I remember correctly: if a package uses TemplateHaskell, it must declare it as such in the Cabal file. This is because Cabal does not parse the source to detect if TemplateHaskell is needed in the dependency graph of the compiled modules. Only GHC can do this reliably. If you don't specify TemplateHaskell as an extension, Cabal might not do the right thing. This is noted in the release notes:
Note that Cabal will correctly handle -dynamic-too for you automatically, especially when -XTemplateHaskell is needed - but you *must* tell Cabal you are using the TemplateHaskell extension.
However, based on the other suggestions in the thread and confusion around this, a big "Incompatible changes" section with this listed as the first thing with clear detail would be a good idea. I'll do so.
If something else is going on, please file a bug.
On Sun, Feb 9, 2014 at 1:37 PM, George Colpitts
wrote: Yes, in general I think the doc needs a section: Incompatible changes. The hope is that you can take the release and just work as usual but when (for good reasons as in this release) it is not true is is important to have such a section. Another case that needs to be there is how to compile so you can load compile object files into ghci as what you did in 7.6.3 won't work in this release.
On Sun, Feb 9, 2014 at 1:11 PM, Carter Schonwald
wrote: Indeed. The problem is that many folks might have cabal config files that explicitly disable shared. (For the compile times!). They might need clear information about wiping that field.
On Sunday, February 9, 2014, Brandon Allbery
wrote: On Sun, Feb 9, 2014 at 9:28 AM, Greg Horn
wrote: Is --enable-shared off by default?
It's supposed to be on by default in 7.8. That said, not sure how many people have played with ~/.cabal/config....
-- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
-- Regards,
Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

The new dynamism is pretty nice! I even used GHC 7.6.3 to build cabal-install 1.18, then played with PATH so that when I said "cabal install mtl" it saw GHC 7.8. It correctly added -dynamic-too and built both *.a and *.so in one go. This is very user-friendly. Still, I observed a few oddities. 1. Referring to the user guide section 2.3 "loading compiled code", it is now insufficient to "ghc -c D.hs". It has to be "ghc -c -dynamic D.hs". Perhaps more importantly, "ghc -c -dynamic-too D.hs" is also insufficient. Apparently, while TemplateHaskell considers both *.hi and *.dyn_hi, :load considers *.hi only. 2. My experiment began with just QuasiQuote. QuasiQuote needs as much dynamism as TemplateHaskell. Yet, QuasiQuote does not trigger a nice implicit -dynamic-too, unlike TemplateHaskell.

hey Albert, could you open a ticket on ghc Trac about the QuasiQuot thing?
On Tue, Feb 18, 2014 at 9:48 PM, Albert Y. C. Lai
The new dynamism is pretty nice! I even used GHC 7.6.3 to build cabal-install 1.18, then played with PATH so that when I said "cabal install mtl" it saw GHC 7.8. It correctly added -dynamic-too and built both *.a and *.so in one go. This is very user-friendly.
Still, I observed a few oddities.
1. Referring to the user guide section 2.3 "loading compiled code", it is now insufficient to "ghc -c D.hs". It has to be "ghc -c -dynamic D.hs".
Perhaps more importantly, "ghc -c -dynamic-too D.hs" is also insufficient. Apparently, while TemplateHaskell considers both *.hi and *.dyn_hi, :load considers *.hi only.
2. My experiment began with just QuasiQuote. QuasiQuote needs as much dynamism as TemplateHaskell. Yet, QuasiQuote does not trigger a nice implicit -dynamic-too, unlike TemplateHaskell.
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

On 14-02-18 10:44 PM, Carter Schonwald wrote:
hey Albert, could you open a ticket on ghc Trac about the QuasiQuot thing?
participants (10)
-
Albert Y. C. Lai
-
Austin Seipp
-
Brandon Allbery
-
Carter Schonwald
-
George Colpitts
-
Greg Horn
-
harry
-
Joachim Breitner
-
Michael Snoyman
-
Simon Marlow