Linker change in GHC 7.8 leads to widespread issues

Hi folks, It seems that something somewhere in linker-land changed in GHC 7.8 such that packages that include C components now need to be built with position-independent code on some platforms. This affects a number of my packages. I am not sure whether the correct fix is to pepper all of these packages (and presumably dozens to hundreds of other packages on Hackage) with "-fPIC" and issue new releases, or to instead teach Cabal to do this. It would certainly be vastly less disruptive to all of the affected package authors to have Cabal automatically do the right thing here. Could someone who's closer to both piles of code please suggest which approach they believe is more appropriate?

whats an example of such a package?
On Tue, Dec 2, 2014 at 4:06 PM, Bryan O'Sullivan
Hi folks,
It seems that something somewhere in linker-land changed in GHC 7.8 such that packages that include C components now need to be built with position-independent code on some platforms.
This affects a number of my packages. I am not sure whether the correct fix is to pepper all of these packages (and presumably dozens to hundreds of other packages on Hackage) with "-fPIC" and issue new releases, or to instead teach Cabal to do this. It would certainly be vastly less disruptive to all of the affected package authors to have Cabal automatically do the right thing here.
Could someone who's closer to both piles of code please suggest which approach they believe is more appropriate?
_______________________________________________ cabal-devel mailing list cabal-devel@haskell.org http://www.haskell.org/mailman/listinfo/cabal-devel

On 2014-12-02, at 21:31, Carter Schonwald
whats an example of such a package?
text-icu. https://github.com/bos/text-icu/pull/9 FWIW, I’m in favour of fixing this at the Cabal level. https://github.com/haskell/cabal/issues/2207 -- Miëtek

If someone can figure out what the right fix is and if it happens to be in
Cabal I'd be happy to merge any changes.
On Tue, Dec 2, 2014 at 10:43 PM, Miëtek Bak
On 2014-12-02, at 21:31, Carter Schonwald
wrote: whats an example of such a package?
text-icu. https://github.com/bos/text-icu/pull/9
FWIW, I’m in favour of fixing this at the Cabal level. https://github.com/haskell/cabal/issues/2207
-- Miëtek
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

On 02/12/2014 21:43, Miëtek Bak wrote:
On 2014-12-02, at 21:31, Carter Schonwald
wrote: whats an example of such a package?
text-icu. https://github.com/bos/text-icu/pull/9
FWIW, I’m in favour of fixing this at the Cabal level. https://github.com/haskell/cabal/issues/2207
I'm actually rather surprised we got away without doing this up to now. Why haven't there been more complaints? Cheers, Simon

I'm also quite surprised about this; Cabal was supposed to handle
-fPIC based on whether GHC was dynamically linked a long time ago, I
figured.
The fact it's broken this long is really unfortunate. I'll attempt to
take a look at this when I have time; we should very much fix this for
7.10 (and perhaps backport the fix to older Cabal versions too,
maybe).
On Wed, Dec 3, 2014 at 6:03 AM, Simon Marlow
On 02/12/2014 21:43, Miëtek Bak wrote:
On 2014-12-02, at 21:31, Carter Schonwald
wrote: whats an example of such a package?
text-icu. https://github.com/bos/text-icu/pull/9
FWIW, I’m in favour of fixing this at the Cabal level. https://github.com/haskell/cabal/issues/2207
I'm actually rather surprised we got away without doing this up to now. Why haven't there been more complaints?
Cheers, Simon
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

Hi,
There aren't more complaints because most people haven't tried 7.8 seriously yet. (I didn't notice the problem since I've only been using GHC 7.8 under Windows.) This points out the need for some sort of regular testing of canaries (full systems producing known results) to test the entire toolchain including common libraries (HP, perhaps) and Cabal.
At the risk of antagonizing the developers again, I will point out that there is a bit of disconnect between GHC developers, Cabal developers, library developers and users (application developers). Without slowing the GHC developers' creativity, there is a need to look at the experience of the application developers. FWIW, my sense is that there is too much turbulence for many potential application developers to adopt Haskell as their production platform unless they have significant in-house resources to deal with the rapid changes. (As as thought experiment, how many changes have to be made to Real World Haskell's programs to get them to run?)
Howard
----- Original Message -----
From: Simon Marlow

I agree, and I’m hoping to help improve these matters with my project, Halcyon (https://halcyon.sh/).
Halcyon allows all build-time and run-time dependencies of an application to be explicitly declared and installed with a single invocation. This builds on the great work done recently in Cabal to get us `cabal freeze`, and works around many long-standing Cabal issues, such as the inability to automatically install missing build-tools.
Halcyon can also be used to restore entire Haskell installations in seconds, and supports GHC versions including 7.8.3, 7.8.2, 7.6.3, 7.6.1, 7.4.2, 7.2.2, and 7.0.4. Hopefully, being able to swap GHC versions in and out, without needing to have them on disk, should help lessen the amount of work needed for comprehensive testing.
I am currently putting the final touches on my project before announcement. Please let me know what you think.
--
Miëtek
On 2014-12-03, at 18:25, Howard B. Golden
Hi,
There aren't more complaints because most people haven't tried 7.8 seriously yet. (I didn't notice the problem since I've only been using GHC 7.8 under Windows.) This points out the need for some sort of regular testing of canaries (full systems producing known results) to test the entire toolchain including common libraries (HP, perhaps) and Cabal.
At the risk of antagonizing the developers again, I will point out that there is a bit of disconnect between GHC developers, Cabal developers, library developers and users (application developers). Without slowing the GHC developers' creativity, there is a need to look at the experience of the application developers. FWIW, my sense is that there is too much turbulence for many potential application developers to adopt Haskell as their production platform unless they have significant in-house resources to deal with the rapid changes. (As as thought experiment, how many changes have to be made to Real World Haskell's programs to get them to run?)
Howard
----- Original Message ----- From: Simon Marlow
To: Miëtek Bak ; Carter Schonwald Cc: "cabal-devel@haskell.org" ; "ghc-devs@haskell.org" Sent: Wednesday, December 3, 2014 4:03 AM Subject: Re: Linker change in GHC 7.8 leads to widespread issues I'm actually rather surprised we got away without doing this up to now. Why haven't there been more complaints?
Cheers, Simon _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Hi, Am Mittwoch, den 03.12.2014, 12:03 +0000 schrieb Simon Marlow:
text-icu. https://github.com/bos/text-icu/pull/9
FWIW, I’m in favour of fixing this at the Cabal level. https://github.com/haskell/cabal/issues/2207
I'm actually rather surprised we got away without doing this up to now. Why haven't there been more complaints?
Debian, for example, hasn’t started using GHC-7.8 properly yet, due to a combination of (at first) „let it settle a bit due to the new dynamic linking scheme“ and (later) „we cannot tackle this this close to the Debian freeze“ and a bit of „these unresolved build problems on arm look scary, let’s wait and see if the are resolved automatically“. (Not sure if there is a conclusion to be drawn here.) Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata@debian.org

On 02/12/2014 21:43, Miëtek Bak wrote:
On 2014-12-02, at 21:31, Carter Schonwald
wrote: whats an example of such a package?
text-icu. https://github.com/bos/text-icu/pull/9
FWIW, I’m in favour of fixing this at the Cabal level. https://github.com/haskell/cabal/issues/2207
I've actually seen this same problem when building GHC 7.8.4rc1 from source, but I assumed it was some problem with my development environment given that this presumably works for others. Ganesh

Hi,
On 2 December 2014 at 22:06, Bryan O'Sullivan
Hi folks,
It seems that something somewhere in linker-land changed in GHC 7.8 such that packages that include C components now need to be built with position-independent code on some platforms.
I've now fixed this issue in Cabal HEAD and also pushed the fix to 1.20 and 1.18 branches. 1.20 and 1.18 point releases should be out soon.

On Wed, Dec 3, 2014 at 12:49 PM, Mikhail Glushenkov < the.dead.shall.rise@gmail.com> wrote:
I've now fixed this issue in Cabal HEAD and also pushed the fix to 1.20 and 1.18 branches. 1.20 and 1.18 point releases should be out soon.
Thanks, Mikhail! Does this imply that package authors can simply advise bug reporters to upgrade cabal-install?

Hi, Am Donnerstag, den 04.12.2014, 01:08 +0100 schrieb Mikhail Glushenkov:
On 4 December 2014 at 00:44, Bryan O'Sullivan
wrote: Thanks, Mikhail! Does this imply that package authors can simply advise bug reporters to upgrade cabal-install?
Yes, once the new point releases of Cabal/cabal-install are out (in a week or two, according to Johan).
is the bug in Cabal or cabal-install? If the former, are we going to include a fixed version of Cabal in GHC-7.8.4? Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata@debian.org

Hi, Am Donnerstag, den 04.12.2014, 17:44 +0100 schrieb Mikhail Glushenkov:
On 4 December 2014 at 09:45, Joachim Breitner
wrote: is the bug in Cabal or cabal-install?
It's in Cabal. So once the new point releases are out, the users will need to run `cabal install cabal-install --constraint="Cabal > 1.20.0.2"`.
in this case, can we get a fixed Cabal in 7.8.4? Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata@debian.org

Bryan O'Sullivan wrote:
It seems that something somewhere in linker-land changed in GHC 7.8 such that packages that include C components now need to be built with position-independent code on some platforms... I am not sure whether the correct fix is to pepper… packages... with "-fPIC"... or to instead teach Cabal to do this.
Aha! Thanks for this explanation of what I have been suffering from. Mikhail Glushenkov wrote:
I've now fixed this issue in Cabal HEAD and also pushed the fix to 1.20 and 1.18 branches. 1.20 and 1.18 point releases should be out soon.
Thanks a million for the speedy response to this serious issue! I can't wait for the point releases. I'll go with HEAD on one of the branches for now. -Yitz

I've merged the patches into the 1.20 branch. Look forward to a patch
release this weekend.
On Thu, Dec 4, 2014 at 1:43 PM, Yitzchak Gale
Bryan O'Sullivan wrote:
It seems that something somewhere in linker-land changed in GHC 7.8 such that packages that include C components now need to be built with position-independent code on some platforms... I am not sure whether the correct fix is to pepper… packages... with "-fPIC"... or to instead teach Cabal to do this.
Aha! Thanks for this explanation of what I have been suffering from.
Mikhail Glushenkov wrote:
I've now fixed this issue in Cabal HEAD and also pushed the fix to 1.20 and 1.18 branches. 1.20 and 1.18 point releases should be out soon.
Thanks a million for the speedy response to this serious issue!
I can't wait for the point releases. I'll go with HEAD on one of the branches for now.
-Yitz _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

I wrote:
Aha! Thanks [to Bryan] for this explanation of what I have been suffering from... I can't wait for the point releases. I'll go with HEAD on one of the branches for now.
Unfortunately, this patch did *not* fix what I have been suffering from. I'm still getting segfaults. I'll describe what I'm doing. Please let me know if either of the two problems below are actual problems, or if I'm just missing something. I'll submit them as issues if they are problems. Sample code: http://github.com/ygale/test-text-icu The following procedure worked fine for 32 bit Windows versions of GHC previous to 7.8.* and with all recent versions of cabal-install. Using Haskell Platform 2014.2.0.0 (GHC 7.8.3) 64 bit on Windows 7. Using rev. caf257cd96e766b293943bbac07d766ec2f552dd of cabal, the latest rev. at the moment, on the 1.20 branch. I locally changed the version numbers in the cabal files of both Cabal and cabal-install so I could verify that I really am using cabal.exe from that revision. I verified in the source code that the -fPIC fix is included. Clone the above sample code, and unzip ICU4C 54.1 (currently the latest) for 64 bit Windows in a nearby folder. ***Problem #1 (minor): Relative linker paths don't work anymore cabal sandbox init cabal install --extra-lib-dirs=..\relative\path\to\icu\lib64 --extra-include-dirs=..\relative\path\to\icu\include text-icu This fails with: Warning: 'extra-lib-dirs: ../relative/path/to/icu/lib64' directory does not exist. Warning: 'include-dirs: ../relative/path/to/icu/include' directory does not exist. cabal: Missing dependencies on foreign libraries: * Missing C libraries: icuuc, icuin, icudt Note: The same problem occurs if you copy the ICU folders into the source tree. That just gets rid of the warnings that sdist won't work. ***Problem #2 (major): segfault! cabal install --extra-lib-dirs=C:\absolute\path\to\icu\lib64 --extra-include-dirs=C:\absolute\path\to\icu\include text-icu cabal install Now, place .cabal-sandbox\bin\test-text-icu.exe together with the 3 DLLs icudt54.dll, icuin54.dll, and icuuc54.dll from icu\bin64 all together in the same folder. Run test-text-icu. Kaboom! Note: The same problem occurs if you place *all* of the DLLs from icu\bin64 in the folder together with the exe. So it's not a missing DLL. Thanks, Yitz

Here's an update to our efforts to get a working build of text-icu on Windows 64 bits: I am working together with Kyra on this on the café list. It is becoming clear that GHC is unable to use standard DLLs on Windows 64 bits. That means FFI is really broken on that platform at the moment. By "unable to use standard DLLs" I mean specifically this: When GHC creates an executable with FFI by linking against object files, the GHC runtime is unable to use the DLLs corresponding to the object files if those DLLs were created by standard Microsoft build tools and not MingW. It's not clear yet whether this incompatibility is due to a change in GHC, or a new incompatibility between MingW's binutils and Microsoft's native DLL tools. Currently I am trying create new ICU DLLs from the object files using binutils so that we can use text-icu. If successful, that will solve our particular problem in the short term. But it won't solve the problem of FFI on Windows 64 bits in general. Here is the bug report I submitted for text-icu, with a simple way to reproduce the problem: https://github.com/bos/text-icu/issues/12 Shall I report that as a GHC bug as well? Thanks, Yitz
participants (11)
-
Austin Seipp
-
Bryan O'Sullivan
-
Carter Schonwald
-
Ganesh Sittampalam
-
Howard B. Golden
-
Joachim Breitner
-
Johan Tibell
-
Mikhail Glushenkov
-
Miëtek Bak
-
Simon Marlow
-
Yitzchak Gale