Re: GHC status report

On 07/05/2014 05:11, Dominick Samperi wrote:

OK, so if I understand correctly, there is no point trying to narrow down my issue further as it is not supposed to work with DYNAMIC_GHC_PROGRAMS=NO (because this disables use of the system linker).

On the contrary, if the external C++ code that you depend on is in a shared library or DLL then DYNAMIC_GHC_PROGRAMS=NO should work fine.

Furthermore, if the plan to disable some form of GHC dynamic linkage is carried out, this should not affect how external C/C++ libs are linked to (using the system linker).

Correct. (but there's no plan at the moment)

From other comments in this thread it appears that there may be problems linking to C++ libs that require static initialization.

Only when both (a) the C++ code is in the same library as the Haskell code (b) DYNAMIC_GHC_PROGRAMS=NO And even then, GHC 7.8 has support for running static initializers in the linker so it might work now. Cheers, Simon

On Mon, May 5, 2014 at 4:58 PM, Simon Marlow wrote:

...

On 03/05/14 04:15, Dominick Samperi wrote:

...

I'm trying to understand the dynamic linking situation with the help of https://ghc.haskell.org/trac/ghc/wiki/Status/GHC-7.8, and according to this information I need to specify DYNAMIC_GHC_PROGRAMS=YES for ghci to use the system linker. I don't understand why you suggested I test with DYNAMIC_GHC_PROGRAMS=NO?

The question I'm trying to answer is "what stops working if we use DYNAMIC_GHC_PROGRAMS=NO?". So that's why I'm interested in what happens if you set this option.

Thanks for your help!

Cheers, Simon

...

Another interesting twist is that my experience under Windows is the opposite of the negative experience described on this page. What I mean by this is more versions of ghci seem to work under Windows (correctly linking to R.dll) than work under Linux!

On Fri, May 2, 2014 at 3:45 PM, Simon Marlow wrote:

...

Can you give me a quick summary of how to reproduce the problem? (not including the GHC build steps)

Cheers, Simon

On 02/05/14 18:18, Dominick Samperi wrote:

...

I downloaded HEAD and placed DYNAMIC_GHC_PROGRAMS=NO in the "quick" section of mk/build.mk (with BuildFlavour = quick), and set DYNAMIC_GHC_PROGRAMS=NO in my environment before running configure (just to be sure!). Near the end of the build I saw some messages like "Warning: vectorization failure," but the build completed.

The status at the end of configure doesn't say that dynamic linking via RTS has been turned off, and I don't know how to check that this is so. Nevertheless, I checked the linking issue and it is NOT fixed with this build, so DYNAMIC_GHC_PROGRAMS=YES is required to prevent the problems reported by me and others.

On Fri, May 2, 2014 at 9:09 AM, Simon Marlow wrote:

...

On 02/05/2014 01:09, Dominick Samperi wrote:

> If I understand your last comment correctly linking to libR should > continue to work, even if you revert to static linking of Haskell > compiled > code via RTS linker. Since you say this is the way things have always > been, perhaps the real explanation for why 7.8 fixed the issue is that > some bug was fixed along the way...

Indeed. To know for sure we would have to test 7.8 with DYNAMIC_GHC_PROGRAMS=NO with your setup - is there a way to do that?

Cheers, Simon

> Note that R is a C library, so the C++ issues that Carter mentions are > not a factor here. > > Thanks, > Dominick > > On Thu, May 1, 2014 at 5:27 PM, Simon Marlow > wrote: >> >> >> >> On 01/05/14 14:48, Dominick Samperi wrote: >>> >>> >>> >>> >>> The problem with some graphics libraries used via FFI (and other >>> libraries >>> that are not thread-safe), if I understand the situation correctly, >>> is >>> that ghci >>> forks a thread when it shouldn't, causing some programs to >>> miscalculate >>> the available stack space (because they think there is only one >>> thread). >> >> >> >> >>> >>> >>> The new dynamic linking support and the flag -fno-ghci-sandbox fixes >>> this problem, and I would not vote for the removal of these >>> features. >> >> >> >> >> >> So I understand how -fno-ghci-sandbox avoids problems with GUI >> libraries >> that use thread-local state. But how is dynamic linking involved >> here? >> What improved in GHC 7.8 relative to 7.6 for you? And could you >> clarify >> "miscalculate the available stack space"? What needs to calculate >> stack >> space? Why? C stack space? >> >> We can certainly make a smoother experience around -fno-ghci-sandbox >> for >> using GUI libraries. >> >> >>> It is not clear to me from Simon's original post how linking to all >>> of >>> those C++ libs can continue to work if dynamic linking is removed >>> in 7.10? Perhaps I misunderstand what you mean by "revert to >>> static linking"? >> >> >> >> >> >> When I say "revert to static linking" I mean make GHCi static linked, >> and >> have it load Haskell code compiled for static linking using the RTS >> linker >> (like it did in 7.6). Foreign libraries would still be loaded using >> the >> system linker, as they always have been. >> >> To be clear, I'm not officially proposing that we drop dynamic >> linking, >> but >> I think it's worthwhile exploring the design space again, given that >> we >> know >> dynamic linking has been tougher than we expected. >> >> Cheers, >> Simon >> >> >> >>> Thanks, >>> Dominick >>> >>> >>> On Wed, Apr 30, 2014 at 9:26 PM, George Colpitts >>> wrote: >>>> >>>> >>>> >>>> >>>> To elaborate, in the past, I had a lot of problems using libraries >>>> from >>>> the >>>> ghci prompt on the Mac but I haven't tried recently. >>>> >>>> As an example, on the web page for the book the Haskell School of >>>> Expression >>>> it says: >>>> >>>> Note for OS X users: running graphics applications from GHCi is no >>>> longer >>>> supported. Instead, one has to compile a graphics program using GHC >>>> in >>>> order >>>> to run it (see example/GMIExamples.lhs for an example). >>>> >>>> I had similar problems using the Yale Euterpea music program from >>>> ghci. >>>> When >>>> I inquired I was referred to >>>> https://ghc.haskell.org/trac/ghc/ticket/4244 >>>> and https://ghc.haskell.org/trac/ghc/ticket/781. I see that the >>>> latter >>>> is >>>> now scheduled for 7.10.1 >>>> >>>> >>>> On Wed, Apr 30, 2014 at 1:45 PM, Simon Marlow >>>> wrote: >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On 30/04/2014 01:35, George Colpitts wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> It doesn't have anything about the dynamic linking changes made >>>>>> for >>>>>> 7.8. >>>>>> I think it's worth mentioning the improvements we expect to get >>>>>> from >>>>>> that. The highlights of the release notes do mention it, so maybe >>>>>> that >>>>>> suffices. >>>>>> >>>>>> In particular, I'm hoping that it is going to fix a lot of >>>>>> problems >>>>>> with >>>>>> using foreign libraries such as OpenGL from ghci. I could be >>>>>> wrong >>>>>> about >>>>>> that though. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> I'd like to understand more about what those problems are. As a >>>>> data >>>>> point, at Facebook we're using static linking (I compiled GHC with >>>>> DYNAMIC_GHC_PROGRAMS=NO), we're loading upwards of 50 3rd-party >>>>> C++ >>>>> libraries and one gigantic shared library consisting of a ton of >>>>> in-house >>>>> C++ code, together with all our Haskell code into GHCi, and it >>>>> works >>>>> perfectly. The key to using the static linker is to not use it >>>>> for >>>>> C++ >>>>> code >>>>> - you want all your external C++ code in shared libraries and load >>>>> those >>>>> using the system linker. >>>>> >>>>> Dynamic linking has been a huge headache in GHC, and it's not >>>>> clear >>>>> that >>>>> it's an overall improvement compared with the static linker. Now >>>>> that >>>>> 7.8 >>>>> is out of the way, it's time to have a conversation about whether >>>>> we >>>>> want to >>>>> do dynamic linking again for 7.10, or revert to static linking. I >>>>> think >>>>> Austin is going to update >>>>> https://ghc.haskell.org/trac/ghc/wiki/DynamicGhcPrograms, and then >>>>> we'll >>>>> see >>>>> where we stand. >>>>> >>>>> Cheers, >>>>> Simon >>>>> >>>>> >>>>> >>>>>> >>>>>> On Tue, Apr 29, 2014 at 6:13 PM, Simon Peyton Jones >>>>>> mailto:simonpj@microsoft.com> wrote: >>>>>> >>>>>> As Austin has told us, there's a draft of the *GHC Status >>>>>> Report >>>>>> for >>>>>> the HCAR*, here:____ >>>>>> >>>>>> https://ghc.haskell.org/trac/ghc/wiki/Status/May14____ >>>>>> >>>>>> >>>>>> Have we missed out something you have been working hard >>>>>> on? >>>>>> Do >>>>>> take a moment to add a bullet in an appropriate place >>>>>> (it's >>>>>> a >>>>>> wiki). I'd like to be sure that we are giving credit to >>>>>> all >>>>>> the >>>>>> appropriate people, so please help us fix that too. GHC >>>>>> is >>>>>> a >>>>>> team >>>>>> effort.____ >>>>>> >>>>>> Deadline is 1 May I think.____ >>>>>> >>>>>> Thanks____ >>>>>> >>>>>> Simon____ >>>>>> >>>>>> __ __ >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> ghc-devs mailing list >>>>>> ghc-devs@haskell.org mailto:ghc-devs@haskell.org >>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> ghc-devs mailing list >>>>>> ghc-devs@haskell.org >>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs >>>>>> >>>> >>>> >>>> _______________________________________________ >>>> ghc-devs mailing list >>>> ghc-devs@haskell.org >>>> http://www.haskell.org/mailman/listinfo/ghc-devs >>>> >>