#8919: Why is xhtml library installed but not exported to users? -------------------------------------+------------------------------------ Reporter: simons | Owner: Type: bug | Status: new Priority: normal | Milestone: 7.8.1 Component: Build System | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Changes (by juhpetersen): * cc: juhp@… (added) Comment: Right +1 In Fedora, so far I have been working around it by packaging the xhtml library along with the other ghc libraries. Since xhtml is in HP anyway it seems easier/cleaner just to move it to ghc now IMHO. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8919#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8919: Why is xhtml library installed but not exported to users? -------------------------------------+------------------------------------ Reporter: simons | Owner: Type: bug | Status: new Priority: normal | Milestone: 7.8.3 Component: Build System | Version: 7.8.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Comment (by hvr): The same applies to `terminfo-0.4.0.0` and `haskeline-0.7.1.2` which have only the DSOs installed (and no interface files). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8919#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8919: Why is xhtml library installed but not exported to users? -------------------------------------+------------------------------------ Reporter: simons | Owner: Type: bug | Status: new Priority: normal | Milestone: 7.8.3 Component: Build System | Version: 7.8.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Comment (by td123): Investigating this further, it looks like this isn't a case of files getting accidentally included. I found the following comment in ghc.mk # If we have built the programs with dynamic libraries, then # ghc will be dynamically linked against haskeline.so etc, so # we need the dynamic libraries of everything down to here Which looks like it includes these libs if DYNAMIC_GHC_PROGRAMS is set to "YES" -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8919#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8919: Why is xhtml library installed but not exported to users? -------------------------------------+------------------------------------ Reporter: simons | Owner: Type: bug | Status: new Priority: normal | Milestone: 7.8.3 Component: Build System | Version: 7.8.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Comment (by hvr): Replying to [comment:4 td123]:

Just encountered this issue on archlinux since mtl was getting installed but it's dep on transformers wasn't. My journey ended at this ticket.

Here is a full list of packages that are used internally, but contain files when you make install:

haskell-haskeline=0.7.1.2 haskell-hoopl=3.9.0.0 haskell-terminfo=0.4.0.0 haskell-transformers=0.3.0.0 haskell-xhtml=3000.2.1

Strange, `transformers` and `hoopl` shouldn't be in that list (i.e. as mentioned earlier in this ticket, only `haskeline`,`terminfo`, and `xhtml` are problematic), as those are actually needed by the `ghc` package (and they will show up in `ghc-pkg list` for a properly performed GHC installation) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8919#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8919: Why is xhtml library installed but not exported to users? -------------------------------------+------------------------------------ Reporter: simons | Owner: Type: bug | Status: new Priority: normal | Milestone: 7.8.3 Component: Build System | Version: 7.8.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Comment (by td123): Investigated this some more this morning: The following commits look like they wanted to make transformers and hoopl installed with ghc on purpose: "Ship transformers with GHC" https://github.com/ghc/ghc/commit/71feb1025eed0c3cc849c85e2b00e16bc1a21790 "Merge in new code generator branch" (introduces hoopl) https://github.com/ghc/ghc/commit/889c084e943779e76d19f2ef5e970ff655f511eb And looking back at the 7.8.1 release notes, hoopl is mentioned as a package. http://www.haskell.org/ghc/docs/7.8.1-rc1/html/users_guide/release-7-8-1.htm... But it doesn't look like transformers is mentioned. So I'm not sure if this is an issue anymore. Looks like they were both meant to be added to ghc. Hoopl is mentioned in the release notes. But transformers isn't but it seems like it should be. Thoughts? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8919#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8919: Why is xhtml library installed but not exported to users? -------------------------------------+------------------------------------ Reporter: simons | Owner: Type: bug | Status: new Priority: normal | Milestone: 7.8.3 Component: Build System | Version: 7.8.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Comment (by td123): All I'm saying there is that the release notes do not include transformers as a library that will be included with ghc. If you visit: http://www.haskell.org/ghc/docs/7.8.1/html/users_guide/index.html it doesn't mention any transformers library when it should under section 1.5.3 The rest of the packages I mentioned don't seem to be a problem at all. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8919#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8919: Why is xhtml library installed but not exported to users? -------------------------------------+------------------------------------ Reporter: simons | Owner: Type: bug | Status: new Priority: normal | Milestone: 7.8.3 Component: Build System | Version: 7.8.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Comment (by juhpetersen): Here is the small patch to ghc.mk I have been using for Fedora builds so far: https://github.com/fedora-haskell/ghc/blob/master/ghc-package-xhtml- terminfo-haskeline.patch -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8919#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8919: Why is xhtml library installed but not exported to users? -------------------------------------+------------------------------------ Reporter: simons | Owner: Type: bug | Status: new Priority: normal | Milestone: 7.8.3 Component: Build System | Version: 7.8.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Comment (by nomeata):

Shipping the shared library without devel files on Linux seems quite unnatural and awkward.

I’m not sure about this: If the `.so` file is put in a private location (not `/usr/lib`, what’s wrong with that)? I’d actually prefer if GHC would ship as few packages as little, so that as many packages as possible can be packaged and installed in the “common” way. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8919#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8919: Why is xhtml library installed but not exported to users? -------------------------------------+------------------------------------ Reporter: simons | Owner: Type: bug | Status: patch Priority: normal | Milestone: 7.8.3 Component: Build System | Version: 7.8.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Changes (by errge): * status: new => patch Old description:

GHC 7.8.1-rc2 installs the xhtml library, but doesn't export it, i.e. it's completely invisible to users. Is there a particular reason why this setup is necessary?

If GHC needs that library internally anywhere, would it be possible to link it statically and *not* install it?

Or, if that's not possible, if the library needs to be installed, please make it visible to users.

New description: GHC 7.8.1-rc2 installs the xhtml library, but doesn't export it, i.e. it's completely invisible to users. Is there a particular reason why this setup is necessary? If GHC needs that library internally anywhere, would it be possible to link it statically and *not* install it? Or, if that's not possible, if the library needs to be installed, please make it visible to users. See comment 17 for a discussion on why this is causing problems for distribution vendors. -- Comment: Setting this to "please review", because two different ways have been analyzed, patch has been proposed for one of them and decision is needed from GHC HQ on how to proceed. If GHC HQ decides on the solution, I'm happy to prepare/review/test solutions on an expedited bases if it has a chance to go into 7.6.3. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8919#comment:18 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

...

As nomeata said in comment 16, we can simply ship our .so files in a

Or via a „private version number” – it should be sufficient if they have a file name that’s different from any filename resulting from a separately

#8919: Why is xhtml library installed but not exported to users? -------------------------------------+------------------------------------ Reporter: simons | Owner: Type: bug | Status: patch Priority: normal | Milestone: 7.8.3 Component: Build System | Version: 7.8.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Comment (by hvr): Replying to [comment:19 nomeata]: private directory and make our dynamic programs search there (via RPATH or some other trick). packaged library. If the purpose of those DSO are actually not really shared, what about simply linking them statically so that no orphaned DSOs are left over? Is there any value in having DSOs that are used by only one executable? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8919#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

I've been asked to briefly summarize the issue, since simons didn't write too much about it in the ticket itself.

GHC 7.8 uses the xhtml, terminfo and haskeline packages internally. Because of the dynamic ghci transition, the related .so files have to be shipped with the build. By not shipping the devel files (not making the cabal packages visible), we force the distribution vendors to package up

#8919: Why is xhtml library installed but not exported to users? -------------------------------------+------------------------------------ Reporter: simons | Owner: Type: bug | Status: patch Priority: normal | Milestone: 7.8.3 Component: Build System | Version: 7.8.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Comment (by slyfox): Replying to [comment:17 errge]: these libraries separately and make them available in their repos. Nothing wrong with that of course (they do this for all the other important libraries, like lens and pipes too). The problem is that they can't ship the .so files in the respective packages, because it's usually not allowed for packages to overwrite each other's files in an adhoc manner. So when they build e.g. the xhtml deb or rpm package, they have to make sure that if the resulting .so files have the same name as the ghc shipped ones, then they don't include it in the package. These kind of workarounds seem extremely kludgy and fragile to me. This naming/overwrite issue causes some headaches for NixOS too, just in a slightly different setting.

I see two solutions going forward.

As nomeata said in comment 16, we can simply ship our .so files in a

private directory and make our dynamic programs search there (via RPATH or some other trick).

Or we can simply ship these packages with GHC as we ship base, time and

so many others. juhpetersen posted a patch for that in comment 14.

Alternatively, we can ignore it and just force people to go with the

workarounds. I'd prefer not doing that. There were also some discussion on the mailing list about discontinuing the dynamic change after 7.10 and going back to static, which would of course make this go away. Thanks for the detailed explanation! ghc's xhtml is already a private lib. You can't accidentally use/link against it: {{{ /usr/lib64/ghc-7.8.2/xhtml-3000.2.1/libHSxhtml-3000.2.1-ghc7.8.2.so }}} On gentoo we just install external haskell libs in other location: {{{ /usr/lib64/xhtml-3000.2.1/ghc-7.8.2/libHSxhtml-3000.2.1-ghc7.8.2.so }}} and it's the package registered with cabal/seen with '''ghci -package xhtml'''. (As you see gentoo ships both .so files, allowing user to rebuild xhtml with any random ghc options out there). I see a problem in one single case: if you install haskell libs in ghc's private subdir. I'd strongly suggest against that practice and pick any other random dir out there. If we would support multiple ghc versions --prefix might be a /usr/$(libdir)/gentoo-ghc-7.8.2 Anyway, cabal can find them anywhere. Or i've misunderstood the clash problem completely? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8919#comment:22 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8919: Why is xhtml library installed but not exported to users? -------------------------------------+------------------------------------ Reporter: simons | Owner: Type: bug | Status: merge Priority: normal | Milestone: 7.8.3 Component: Build System | Version: 7.8.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Changes (by thoughtpolice): * status: patch => merge -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8919#comment:24 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8919: Why is xhtml library installed but not exported to users? -------------------------------------+------------------------------------ Reporter: simons | Owner: Type: bug | Status: closed Priority: normal | Milestone: 7.8.3 Component: Build System | Version: 7.8.2 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Changes (by thoughtpolice): * status: merge => closed * resolution: => fixed -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8919#comment:26 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8919: Why is xhtml library installed but not exported to users? -------------------------------------+--------------------------------- Reporter: simons | Owner: (none) Type: bug | Status: closed Priority: normal | Milestone: 7.8.3 Component: Build System | Version: 7.8.2 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | -------------------------------------+--------------------------------- Comment (by Herbert Valerio Riedel ): In [changeset:"02ff70563e490d2a7f3141eab7229803c523da57/ghc" 02ff705/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="02ff70563e490d2a7f3141eab7229803c523da57" Add 'stm' package to the global package database This is a preparation for `haskeline` picking up a dependency on `stm` real soon now. See https://github.com/judah/haskeline/pull/61 for details. If we figure out a way to not bundle the libraries depended upon by the GHCi executable in the global package database (see #8919 for the original reason why we had to start bundling terminfo/haskeline in the first place) we can get rid of `stm` again... On the bright side, we were able to avoid uploading new `stm` releases for over two years already, so it shouldn't cause too much trouble if GHC imposes a strong preference on the `stm` package's version (this most likely will mostly affect Linux distributions & similiar). While at it, this also update the stm submodule to include relaxed bounds to allow the upcoming base-4.11 version. }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8919#comment:28 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler