You could work around the dlopen issue as long as the static library is compiled with -fPIC by using --whole-archive (assuming you permit dangling references which will need to be resolved later) and making a shared library out of the static code. But you'd have to create one shared library per static library and preserve the order so you don't end up with symbol collisions. And you'd likely not want to do it this on every relink. But i think the - fPIC is a much greater hurdle. Very few of the static libraries a user may want to use would have this likely. I think it'll end up being quite a messy situation.. On Thu, Jun 7, 2018, 22:01 Simon Marlow wrote:

There's a technical restriction. The static code would be compiled with the small memory model, so it would have 32-bit relocations for external references, assuming that those references would resolve to something in the low 2GB of the address space. But we would be trying to link it against shared libraries which could be loaded anywhere in the address space.

If the static code was compiled with -fPIC then it might be possible, but there's also the restriction that we wouldn't be able to dlopen() a shared library that depends on the statically linked code, because the system linker can't see the symbols that the RTS linker has loaded. GHC doesn't currently know about this restriction, so it would probably go ahead and try, and things would break.

Cheers Simon

On 29 May 2018 at 04:05, Moritz Angermann wrote:

...

Dear friends,

when we build GHC with DYNAMIC_GHC_PROGRAMS=YES, we essentially prevent ghc/ghci from using archives (.a). Is there a technical reason behind this? The only only reasoning so far I've came across was: insist on using dynamic/shared objects, because the user said so when building GHC.

In that case, we don't however prevent GHC from building archive (static) only libraries. And as a consequence when we later try to build another archive of a different library, that depends via TH on the former library, GHC will bail and complain that we don't have the relevant dynamic/shared object. Of course we don't we explicitly didn't build it. But the linker code we have in GHC is perfectly capable of loading archives. So why don't we want to fall back to archives?

Similarly, as @deech asked on twitter[1], why we prevent GHCi from loading static libraries?

I'd like to understand the technical reason/rational for this behavior. Can someone help me out here? If there is no fundamental reason for this behavior, I'd like to go ahead and try to lift it.

Thank you!

Cheers, Moritz

--- [1]: https://twitter.com/deech/status/1001182709555908608 _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

I'm not sure how you could make DYNAMIC_GHC_PROGRAMS into a runtime flag, since it controls whether GHC itself (and the other tools) are built as dynamic executables. If GHC is a dynamic executable, then it can only load -fPIC code to link at runtime. Cheers Simon On 13 June 2018 at 02:30, Moritz Angermann wrote:

Thank you both for the replies.

My issue with the current situation is that I can navigate myself into a situation where I’m stuck. By asking ghc to build static libraries, it will later fall over when it tries to load those.

Guess what I really want is to turn the DYNAMIC_GHC_PROGRAMS into a runtime flag.

That might help with getting out of the situation without resorting to building two ghcs.

Cheers, Moritz

Sent from my iPhone

On 12 Jun 2018, at 9:07 PM, Phyx wrote:

You could work around the dlopen issue as long as the static library is compiled with -fPIC by using --whole-archive (assuming you permit dangling references which will need to be resolved later) and making a shared library out of the static code. But you'd have to create one shared library per static library and preserve the order so you don't end up with symbol collisions.

And you'd likely not want to do it this on every relink. But i think the - fPIC is a much greater hurdle. Very few of the static libraries a user may want to use would have this likely.

I think it'll end up being quite a messy situation..

On Thu, Jun 7, 2018, 22:01 Simon Marlow wrote:

...

There's a technical restriction. The static code would be compiled with the small memory model, so it would have 32-bit relocations for external references, assuming that those references would resolve to something in the low 2GB of the address space. But we would be trying to link it against shared libraries which could be loaded anywhere in the address space.

If the static code was compiled with -fPIC then it might be possible, but there's also the restriction that we wouldn't be able to dlopen() a shared library that depends on the statically linked code, because the system linker can't see the symbols that the RTS linker has loaded. GHC doesn't currently know about this restriction, so it would probably go ahead and try, and things would break.

Cheers Simon

On 29 May 2018 at 04:05, Moritz Angermann wrote:

...

Dear friends,

when we build GHC with DYNAMIC_GHC_PROGRAMS=YES, we essentially prevent ghc/ghci from using archives (.a). Is there a technical reason behind this? The only only reasoning so far I've came across was: insist on using dynamic/shared objects, because the user said so when building GHC.

In that case, we don't however prevent GHC from building archive (static) only libraries. And as a consequence when we later try to build another archive of a different library, that depends via TH on the former library, GHC will bail and complain that we don't have the relevant dynamic/shared object. Of course we don't we explicitly didn't build it. But the linker code we have in GHC is perfectly capable of loading archives. So why don't we want to fall back to archives?

Similarly, as @deech asked on twitter[1], why we prevent GHCi from loading static libraries?

I'd like to understand the technical reason/rational for this behavior. Can someone help me out here? If there is no fundamental reason for this behavior, I'd like to go ahead and try to lift it.

Thank you!

Cheers, Moritz

--- [1]: https://twitter.com/deech/status/1001182709555908608 _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs