On 02/05/14 18:19, Edward Kmett wrote:I think we're getting confused here. External C++ libraries are, and always have been, loaded by the system linker, regardless of whether GHCi itself is dynamically linked. So there's no difference in 7.8 with respect to how external C++ code is loaded.
I may have to dig to find an example, but when I last checked it seemed
that c++ libraries would load fine, but there was a problem with static
initializers not getting called, when loading from ghci, so if your c++
library needed them then you'd have problems.
However, if you link the C++ code into the same library as the Haskell code as you're doing with MPFR, then of course there is a difference (let's call that *internal* C++ code, for want of a better name). I think that's a rare case (I understand why you need to do it), and most people should be linking their C++ code via separate shared libraries.Absolutely not, we're drowning in static initialisers :)
An earlier version of the MPFR hackery used a static initializer to
replace the GHC garbage collection hook for GMP with one that played
evil games to figure out if it was being called from the MPFR constant
cache.
That initialized and loaded fine from ghc, but not from ghci. Our static
initializer was never being called, despite the library being loaded.
If your C++ practices there forbid static initializers -- some places do
-- that may be why you aren't seeing the issue.
Cheers,
Simon
<mailto:marlowsd@gmail.com>> wrote:
On 02/05/2014 14:28, Edward Kmett wrote:
> Perhaps. We actually tried that originally, but had issues about
> where and how to get cabal to place it. We'd need it to go somewhere
> installed rather than the local build dir lest it not be there when
> we go to use the lib, but IIRC, cabal couldn't/wouldn't tell me where
> it was putting the final installed version and then there is the
> issue of the local in place runs vs. post cabal install runs and
> referencing that dir from subsequent projects. If we want a
> transparent ’cabal install rounded` It isn't clear to me how to get
> there down this path, but it is entirely possible I just missed
> something obvious.
>
> MPFR/rounded was just an off the cuff example. Another place where
> the dynamic linker really helps is external c++ libraries which
> should now actually get all of their initializers called in the right
> order when launched from ghci.
External C++ libraries work just fine with the statically linked GHCi -
it uses the system linker to load them, and all the initalizers get
called, in the right order, as they should. So this is where I'm
confused about what the problems actually are - there seems to be this
perception that GHCi didn't work with external C++ code, but as far as I
know it works just fine (indeed we're doing it a lot at Facebook, so
there's at least an existence proof that depending on a non-trivial
amount of external C++ with a statically-linked GHCi can work).
Cheers,
Simon
>
>
>> Cheers,
>> Simon
>>
>>> Switching to the system dynamic linker fo ghci seems to have
resolved
>>> all of that effortlessly.
>>>
>>> Dan Peebles has been talking to the MPFR folks to see if we can
get them
>>> to expose enough information about the 'hidden' allocations
they use
>>> that we can make them visible to GHC or have them do what our
local fix
>>> does and avoid using the MPFR allocator for their hidden
constant cache.
>>>
>>> If they do that then we can actually link to the library like
normal
>>> rather than link it in directly, but it isn't clear to me what
would
>>> happen even with those hooks if we rolled back to something
like the old
>>> custom linker.
>>>
>>> -Edward
>>>
>>>
>>> On Thu, May 1, 2014 at 5:30 PM, Simon Marlow
<marlowsd@gmail.com <mailto:marlowsd@gmail.com>>>> <mailto:marlowsd@gmail.com <mailto:marlowsd@gmail.com>>> wrote:
>>>
>>> On 01/05/14 15:27, Edward Kmett wrote:
>>>
>>> Figured I'd make one case for dynamic linking:
>>>
>>> https://github.com/ekmett/__rounded
>>> <https://github.com/ekmett/rounded>
>>>
>>> Dynamic linking is finally enabling us to build a
version of MPFR
>>> bindings for Haskell for scientific/high precision
computing
>>> with 7.8. I
>>> would really hate to lose it after all of these years
trying to
>>> get it
>>> work, as I have a rather large edifice being built atop
that
>>> platform.
>>> We tried and failed due to limitations of the old
linker for
>>> almost 3 years.
>>>
>>>
>>> I understand the issues with MPFR. But how is dynamic
linking helping?
>>>
>>>
>>> That said, -dynamic-too seems to cause me all sorts of
problems
>>> elsewhere. ^C'ing out of a build and restarting it will
often
>>> make a .o
>>> but lose the .dyn_o, leading to GHC + cabal getting
confused and
>>> refusing to build until I clean. This hits me several
times a day.
>>>
>>>
>>> We should fix this (or at least make it a lot less likely). Is
>>> there a ticket?
>>>
>>> Cheers,
>>> Simon
>>>
>>>
>>> -Edward
>>>
>>>
>>> On Thu, May 1, 2014 at 3:29 AM, Simon Peyton Jones
>>> <simonpj@microsoft.com <mailto:simonpj@microsoft.com>
<mailto:simonpj@microsoft.com <mailto:simonpj@microsoft.com>>>>> <mailto:simonpj@microsoft.com
<mailto:simonpj@microsoft.com> <mailto:simonpj@microsoft.com
<mailto:simonpj@microsoft.com>>>__>
>>> wrote:
>>>
>>> | Dynamic linking has been a huge headache in GHC,
and it's not
>>> clear that
>>> | it's an overall improvement compared with the static
>>> linker. Now that
>>> | 7.8 is out of the way, it's time to have a
conversation about
>>> whether we
>>> | want to do dynamic linking again for 7.10, or
revert to
>>> static
>>> linking.
>>>
>>> I echo this. Dynamic linking has had many
un-anticipated
>>> costs and
>>> it is still very far from sorted out. It
originally felt
>>> like a
>>> Fantastic Idea to give up our own linker and adopt
the system
>>> linker, but it now feels to me like a black hole,
endlessly
>>> sucking
>>> effort and increasing complexity.
>>>
>>> My viewpoint is highly un-informed about details;
I just
>>> watch the
>>> traffic going by. And of course it does have
benefits that
>>> doubtless generate less traffic.
>>>
>>> Simon
>>>
>>> |
>>> |
>>> |
>>> | >
>>> | > On Tue, Apr 29, 2014 at 6:13 PM, Simon Peyton
Jones
>>> | > <simonpj@microsoft.com
<mailto:simonpj@microsoft.com> <mailto:simonpj@microsoft.com
<mailto:simonpj@microsoft.com>>
>>> <mailto:simonpj@microsoft.com
<mailto:simonpj@microsoft.com> <mailto:simonpj@microsoft.com
<mailto:simonpj@microsoft.com>>>
>>> <mailto:simonpj@microsoft.com
<mailto:simonpj@microsoft.com>
>>> <mailto:simonpj@microsoft.com
<mailto:simonpj@microsoft.com>> <mailto:simonpj@microsoft.com
<mailto:simonpj@microsoft.com>
>>> <mailto:simonpj@microsoft.com
<mailto:simonpj@microsoft.com>>>__>> wrote:
>>> | >
>>> | > As Austin has told us, there's a draft of
the *GHC
>>> Status Report
>>> | for
>>> | > the HCAR*, here:____
>>> | >
>>> | >
>>> https://ghc.haskell.org/trac/__ghc/wiki/Status/May14____
>>> <https://ghc.haskell.org/trac/ghc/wiki/Status/May14____>
>>> | >
>>> | > Have we missed out something you have
been working
>>> hard on? Do
>>> | > take a moment to add a bullet in an
appropriate
>>> place (it's a
>>> | > wiki). I'd like to be sure that we are giving
>>> credit to all the
>>> | > appropriate people, so please help us fix
that too.
>>> GHC is
>>> a team
>>> | > effort.____
>>> | >
>>> | > Deadline is 1 May I think.____
>>> | >
>>> | > Thanks____
>>> | >
>>> | > Simon____
>>> | >
>>> | > __ __
>>> | >
>>> | >
>>> | >
_________________________________________________
>>> | > ghc-devs mailing list
>>> | > ghc-devs@haskell.org
<mailto:ghc-devs@haskell.org> <mailto:ghc-devs@haskell.org
<mailto:ghc-devs@haskell.org>>
>>> <mailto:ghc-devs@haskell.org
<mailto:ghc-devs@haskell.org> <mailto:ghc-devs@haskell.org
<mailto:ghc-devs@haskell.org>>>
>>> <mailto:ghc-devs@haskell.org
<mailto:ghc-devs@haskell.org> <mailto:ghc-devs@haskell.org
<mailto:ghc-devs@haskell.org>>
>>> <mailto:ghc-devs@haskell.org
<mailto:ghc-devs@haskell.org> <mailto:ghc-devs@haskell.org
<mailto:ghc-devs@haskell.org>>>>
>>>
>>> | > http://www.haskell.org/__mailman/listinfo/ghc-devs
>>> <http://www.haskell.org/mailman/listinfo/ghc-devs>
>>> | >
>>> | >
>>> | >
>>> | >
>>> | > _________________________________________________
>>> | > ghc-devs mailing list
>>> | > ghc-devs@haskell.org
<mailto:ghc-devs@haskell.org> <mailto:ghc-devs@haskell.org
<mailto:ghc-devs@haskell.org>>
>>> <mailto:ghc-devs@haskell.org
<mailto:ghc-devs@haskell.org> <mailto:ghc-devs@haskell.org
<mailto:ghc-devs@haskell.org>>>
>>> | > http://www.haskell.org/__mailman/listinfo/ghc-devs
>>> <http://www.haskell.org/mailman/listinfo/ghc-devs>
>>> | >
>>> _________________________________________________
>>> ghc-devs mailing list
>>> ghc-devs@haskell.org <mailto:ghc-devs@haskell.org>
<mailto:ghc-devs@haskell.org <mailto:ghc-devs@haskell.org>>
>>> <mailto:ghc-devs@haskell.org
<mailto:ghc-devs@haskell.org> <mailto:ghc-devs@haskell.org
<mailto:ghc-devs@haskell.org>>>
>>> http://www.haskell.org/__mailman/listinfo/ghc-devs
>>> <http://www.haskell.org/mailman/listinfo/ghc-devs>
>>>
>>>
>>>
>>>