On Sun, Mar 15, 2009 at 10:49:20PM +0000, Ingmar Vanhassel wrote:

...

http://www.haskell.org/ghc/dist/6.10.2-rc1/

Could you add a testsuite tarball too please?

There's already one there. Thanks Ian

I've tried the 6.10.2 RC with some performance-sensitive work code. The code uses the non-threaded runtime, and makes extensive use of signals. The results look very good.

The slightly funny (but useful to us) benchmark measures bandwidth communicating between multiple unix processes. Here's a graph of how much better 6.10.2 is doing:

http://galois.com/~dons/images/bandwidth.png

Note we're over a 1G/s bandwidth now with 6.10.2 for the first time :) Thanks guys!

(Note also we use a slightly modified runtime that has thread deadlock detection disabled).

There have been quite a few runtime patches, enough that 6.10.2 is really quite a different runtime now:

http://galois.com/~dons/rts.patches http://galois.com/~dons/6101-6102.diff

Anyway, all those signal and thread handling changes are looking good.

Looking back through what has gone into 6.10.2, I think the credit probably goes to the patch below. It's nice to hear this improves performance too - that was an unexpected benefit. If you're able to test with the HEAD, I'd be very interested to see your results there too. Cheers, Simon Fri Feb 20 14:32:00 GMT 2009 Ian Lynagh * MERGED: Rewrite of signal-handling (ghc patch; see also base and unix patches) Simon Marlow **20090219103142 Ignore-this: aca7c3e258224fadc6f0f2fee86b2971 The API is the same (for now). The new implementation has the capability to define signal handlers that have access to the siginfo of the signal (#592), but this functionality is not exposed in this patch. #2451 is the ticket for the new API. The main purpose of bringing this in now is to fix race conditions in the old signal handling code (#2858). Later we can enable the new API in the HEAD. Implementation differences: - More of the signal-handling is moved into Haskell. We store the table of signal handlers in an MVar, rather than having a table of StablePtrs in the RTS. - In the threaded RTS, the siginfo of the signal is passed down the pipe to the IO manager thread, which manages the business of starting up new signal handler threads. In the non-threaded RTS, the siginfo of caught signals is stored in the RTS, and the scheduler starts new signal handler threads.

Hi Ian, Ian Lynagh wrote:

The buildbot is putting "SplitObjs=NO" in mk/build.mk. The above log has object splitting enabled, which means that there are a lot more .o files, which is presumably causing some problem with your ar.

If you have both a GNU ar and a Solaris ar then it might be worth seeing if the other one works. Otherwise, putting "SplitObjs=NO" in mk/build.mk will work, but at the expense of larger binaries.

I put just GNU ar into my PATH, so GHC build is using Sun ld and GNU ar and the build runs for much longer and breaks on: Building hsc2hs-0.67... [1 of 2] Compiling Paths_hsc2hs ( dist-install/build/autogen/Paths_hsc2hs.hs, dist-install/build/hsc2hs/hsc2hs-tmp/Paths_hsc2hs.o ) [2 of 2] Compiling Main ( Main.hs, dist-install/build/hsc2hs/hsc2hs-tmp/Main.o ) Linking dist-install/build/hsc2hs/hsc2hs ... gmake[3]: Leaving directory `/var/tmp/ghc-6.10.1.20090314/utils/hsc2hs' gmake -C haddock install-inplace gmake[3]: Entering directory `/var/tmp/ghc-6.10.1.20090314/utils/haddock' /var/tmp/ghc-6.10.1.20090314/utils/installPackage/install-inplace/bin/installPackage install '/var/tmp/ghc-6.10.1.20090314/utils/ghc-pkg/install-inplace/bin/ghc-pkg' '/var/tmp/ghc-6.10.1.20090314/inplace-datadir/package.conf' '' \ '/var/tmp/ghc-6.10.1.20090314/utils/haddock/install-inplace' \ '/var/tmp/ghc-6.10.1.20090314/utils/haddock/install-inplace' \ '$prefix/bin' \ '$prefix/lib' \ '$prefix/libexec' \ '$prefix/dynlib' \ '$prefix/share' \ '$prefix/doc' \ '$prefix/html' \ '$prefix/haddock' \ --distpref dist-install \ --enable-shell-wrappers Installing library in /var/tmp/ghc-6.10.1.20090314/utils/haddock/install-inplace/lib/haddock-2.4.2 Installing executable(s) in /var/tmp/ghc-6.10.1.20090314/utils/haddock/install-inplace/bin internal error: error_message(28) gmake[3]: *** [install-inplace] Error 100 gmake[3]: Leaving directory `/var/tmp/ghc-6.10.1.20090314/utils/haddock' gmake[2]: *** [with-stage-2] Error 2 gmake[2]: Leaving directory `/var/tmp/ghc-6.10.1.20090314/utils' gmake[1]: *** [stage2] Error 2 gmake[1]: Leaving directory `/var/tmp/ghc-6.10.1.20090314' gmake: *** [bootstrap2] Error 2 The message `internal error: error_message(28)' looks suspiciously, is there any Sun ar invoked behind the scene? Or isn't this message from the AR but coming from something else? Thanks, Karel

Christian Maeder wrote:

Testsuite results are bad for ghc-6.10.1.20090314, see http://hackage.haskell.org/trac/ghc/ticket/3106

Patch for the cygpath not found issue is attached to the ticket. Please (Windows/Cygwin/Mingw users especially) test it. Thanks, Karel

On Tue, 2009-03-17 at 11:09 +0100, Christian Maeder wrote:

GHC 6.10.2 will have a problem with cabal-install-0.6.2!

When I tried to install cabal-install-0.6.2 for ghc-6.10.1.20090314 I needed to change #!/bin/sh to #!/bin/bash in bootstrap.sh to avoid the following errors:

-bash-3.00$ ./bootstrap.sh Checking installed packages for ghc-6.10.1.20090314... ./bootstrap.sh: !: not found

Under Solaris sh is not bash!

Indeed. According to the OpenGroup that syntax should be fine: http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_... It works for me under Solaris 10. Perhaps Solaris 9 or older do not have a standard compliant /bin/sh program. What do you suggest we use instead as a workaround?

Next, ghc-6.10.1.20090314 comes with package unix-2.4.0.0, but cabal-install.cabal requests:

unix >= 2.0 && < 2.4

Changing to "<= 2.4" was not sufficient, so I changed it to "<= 2.5". This will affect any OS!

Hmm, it's a bit suspicious that the major version number is changing in a minor ghc release. Do we know what the API breakage is? This could affect any program. Duncan

On Tue, 2009-03-17 at 21:12 -0400, Brandon S. Allbery KF8NH wrote:

On 2009 Mar 17, at 20:28, Duncan Coutts wrote:

...

It works for me under Solaris 10. Perhaps Solaris 9 or older do not have a standard compliant /bin/sh program. What do you suggest we use instead as a workaround?

For backward compatibility reasons sh in Solaris 9 and earlier is not POSIX compliant. Use /usr/xpg4/bin/sh or /bin/bash instead.

(Unfortunately you can't cheat and define a shell function, although you could create a program called "!":

if ${1+"$@"}; then exit 1 else exit 0 fi

Actually this is what the script used 'til someone pointed out to me that sh has the ! syntax :-). I'll switch it back to using this style with a note to say why. Duncan - ! grep " ${PKG}-${VER_MATCH}" ghc-pkg.list > /dev/null 2>&1 + if grep " ${PKG}-${VER_MATCH}" ghc-pkg.list > /dev/null 2>&1 + then + return 1; + else + return 0; + fi + #Note: we cannot use "! grep" as Solaris 9 /bin/sh doesn't like it.

On Wed, Mar 18, 2009 at 12:28:50AM +0000, Duncan Coutts wrote:

On Tue, 2009-03-17 at 11:09 +0100, Christian Maeder wrote:

...

unix >= 2.0 && < 2.4

Changing to "<= 2.4" was not sufficient, so I changed it to "<= 2.5". This will affect any OS!

Hmm, it's a bit suspicious that the major version number is changing in a minor ghc release. Do we know what the API breakage is? This could affect any program.

This looks like a braino to me. Things have been added to the API, but I don't see anything that's been removed or change. I'll put the version number back to 2.3.2.0. Thanks Ian

Ian Lynagh wrote:

On Mon, Mar 16, 2009 at 04:00:09PM +0100, Christian Maeder wrote:

...

Under Solaris grep does not understand "-q" in configure:

checkMake380() { if $1 --version 2>&1 | head -1 | grep -q 'GNU Make 3\.80'

it fails with:

grep: illegal option -- q Usage: grep -hblcnsviw pattern file . . . grep: illegal option -- q Usage: grep -hblcnsviw pattern file . . .

OK, I'll just redirect stdout to /dev/null then.

Presumably these print 0 and 1 respectively?

echo foo | grep foo > /dev/null; echo $? echo foo | grep bar > /dev/null; echo $?

Yes, Christian -bash-3.00$ echo foo | grep foo > /dev/null; echo $? 0 -bash-3.00$ echo foo | grep bar > /dev/null; echo $? 1

On Wed, Mar 18, 2009 at 12:35:01PM +0100, Thorkil Naur wrote:

I have tried the Intel Mac installer and the source package on both FreeBSD and PPC Mac OS X. Some comments follow

Thanks!

1. An important property of such installers is that you are told, right from the start, that all the information you are presented with during the installation process can be accessed at any time, after completion of the installation, and you are told how. In case of GHC, something like a reference to a suitable spot, given as part of the output from ghc --help. I don't see any trace of such a facility on the introduction screen. (I know that other installers don't do this either, which is part of the reason why I try to avoid them.)

I didn't follow that; can you say exactly what text you want where, please?

2. The introduction screen says "This package must be installed on the system volume" which seems to imply that there is some kind of choice involved at a later stage. But I don't recall having to perform any choice that related to this. So perhaps this should be "This package will be installed on the system volume" instead.

Good point, done.

3. I can copy and paste the text of the introduction screen, excellent. I cannot copy and paste the header.

I don't think there's anything we can do about this.

4. On the License screen, GMP is denoted "GNU MP Bignum Library" in two places. I suggest using "GNU Multiple Precision Arithmetic Library" (from http://gmplib.org/) instead.

The title of that page is "The GNU MP Bignum Library", and the docs seem to be mostly consistent in calling it the "GNU MP Library", so I'm going to leave it as it is unless there's consensus for a change.

5. On the License screen, replace "licence" by "license", twice.

Thanks, changed to make it consistent.

6. The License screen explains "that by default the GMP will be statically linked into any binary produced by GHC. Software with a non-GPL compatible licence [sic] will have to ensure that the conditions of the LGPL are met; for example, by forcing GMP to link dynamically instead." Some details or a reference to an explanation of how to do this would be nice.

I don't think we want details in the installer, but we should have something in the docs somewhere.

Also, shouldn't "non-GPL compatible" be plain "non-GPL"?

No, I believe (IANAL etc) any license that provides the appropriate freedoms is OK.

7. I may very well be wrong, but as far as I have been able to tell, the ghc executable itself that is distributed (/Library/Frameworks/GHC.framework/Versions/610/usr/lib/ghc-6.10.1.20090314/ghc) contains GMP, statically linked. For example:

...

thorkil-naurs-intel-mac-mini:~/tn/test/GHC/release/GHC-6.10.2-rc1 thorkilnaur$ DYLD_PRINT_LIBRARIES=YES /usr/bin/ghc --version dyld: loaded: /bin/sh dyld: loaded: /usr/lib/libncurses.5.4.dylib dyld: loaded: /usr/lib/libiconv.2.dylib dyld: loaded: /usr/lib/libSystem.B.dylib dyld: loaded: /usr/lib/libgcc_s.1.dylib dyld: loaded: /usr/lib/system/libmathCommon.A.dylib dyld: loaded: /Library/Frameworks/GHC.framework/Versions/610/usr/lib/ghc-6.10.1.20090314/ghc dyld: loaded: /usr/lib/libedit.2.dylib dyld: loaded: /usr/lib/libncurses.5.4.dylib dyld: loaded: /usr/lib/libSystem.B.dylib dyld: loaded: /usr/lib/libgcc_s.1.dylib dyld: loaded: /usr/lib/system/libmathCommon.A.dylib The Glorious Glasgow Haskell Compilation System, version 6.10.1.20090314 thorkil-naurs-intel-mac-mini:~/tn/test/GHC/release/GHC-6.10.2-rc1 thorkilnaur$

where I fail to observe anything that seems to relate to GMP. This is in contrast to the corresponding information for an earlier GHC installation

...

thorkil-naurs-intel-mac-mini:~/tn/test/GHC/release/GHC-6.10.2-rc1 thorkilnaur$ DYLD_PRINT_LIBRARIES=YES ghc --version dyld: loaded: /bin/sh dyld: loaded: /usr/lib/libncurses.5.4.dylib dyld: loaded: /usr/lib/libiconv.2.dylib dyld: loaded: /usr/lib/libSystem.B.dylib dyld: loaded: /usr/lib/libgcc_s.1.dylib dyld: loaded: /usr/lib/system/libmathCommon.A.dylib dyld: loaded: /Users/thorkilnaur/tn/install/ghc-6.8.3/lib/ghc-6.8.3/ghc-6.8.3 dyld: loaded: /opt/local/lib/libreadline.5.2.dylib dyld: loaded: /opt/local/lib/libncurses.5.dylib dyld: loaded: /usr/lib/libSystem.B.dylib dyld: loaded: /opt/local/lib/libgmp.3.dylib dyld: loaded: /usr/lib/libgcc_s.1.dylib dyld: loaded: /usr/lib/system/libmathCommon.A.dylib The Glorious Glasgow Haskell Compilation System, version 6.8.3 thorkil-naurs-intel-mac-mini:~/tn/test/GHC/release/GHC-6.10.2-rc1 thorkilnaur$

where one observes the presence of "dyld: loaded: /opt/local/lib/libgmp.3.dylib" that loads the separately installed dynamic GMP library.

Hmm, I'm note sure why that changed. Manuel, do you know?

9. I cannot copy and paste the text or the header of the Installation Type screen, the one that says "This will take 448 MB of space on your computer. Click Install to perform a standard installation of this software for all users of this computer."

That looks like it's baked into the installer generator, i.e. i don't think we can change it.

10. On the Summary screen, I am told to find additional documentation at "/usr/share/doc/ghc/index.html". Should that, perhaps, be "file:///usr/share/doc/ghc/index.html" or something else, that more directly indicates how to access this documentation? And also mention the use of a browser?

I think /usr/share/doc/ghc/index.html will work in any web browser. I've tweaked the text a bit.

12. What happens if I have a GHC installed already? To check this, I tried running the installer twice, without uninstalling in between. I didn't notice any difference between the first and second install. So I guess that the first installation was simply being overwritten.

I realise that the /Library/Framework conventions allow multiple versions of the same thing to be installed. But the way the GHC package is supposed to be accessed is via symbolic links in /usr/bin (and elsewhere?) and these links are surely being overwritten. If there is a mechanism for changing between versions, I haven't noticed it.

I don't know enough about the OS X installers to know if we can do better on this. I'll take a look at your testsuite failures separately. Thanks Ian

Hello Simon, I've put a request about the issue on the gtk2hs users mailing list:

I've tried a gtk2hs app on ghc 6.10.2 release candidate. It crashes frequently and Simon (as you can read down here) assumes it is gtk2hs problem. My question is: Is this problem known to gtk2hs developers? Is it really a gtk2hs problem? How difficult is it to fix the problem? When will we have a patch to use gtk2hs with 6.10.2, is it already in the repo?

However, I'm a little surprised that a little version upgrade from 6.10.1 to 6.10.2 may break all gui apps based on gtk2hs. May it be that many more apps are affected because of this change? What's about wxhaskell e.g.? Well, maybe we have only few Haskell applications around, but usually I wouldn't expect such a dramatic effect from such a moderate upgrade. Is this fix so important to introduce it now? What does it help when it was never officially supported if it causes such troubles? Jürgen Simon Marlow-7 wrote:

jutaro wrote:

...

I've installed a GUI application based on gtk2hs.

It frequently crashes with the error: leksah: error: a C finalizer called back into Haskell. use Foreign.Concurrent.newForeignPtr for Haskell finalizers.

This error did never occur with the 6.10 released version. It was verified that this happens on different machines. I've no idea how to isolate this bug.

This will need to be fixed in gtk2hs. Previously GHC allowed finalizers to call back into Haskell, but this was never officially supported. Now it is officially unsupported, because finalizers created via Foreign.mkForeignPtr are run directly by the garbage collector.

See

http://hackage.haskell.org/trac/ghc/ticket/1364

Cheers, Simon _______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

-- View this message in context: http://www.nabble.com/ANNOUNCE%3A-GHC-6.10.2-Release-Candidate-1-tp22524567p... Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com.

On Thu, 2009-03-19 at 16:34 -0700, Don Stewart wrote:

We must have the gtk2hs team invovled in this discussion. They were using an undocumented feature. It may be trivial to fix.

...

...

This will need to be fixed in gtk2hs. Previously GHC allowed finalizers to call back into Haskell, but this was never officially supported. Now it is officially unsupported, because finalizers created via Foreign.mkForeignPtr are run directly by the garbage collector.

I had a quick look but so far I cannot see where any callback into Haskell is happening. The only interesting case I've found is one finaliser which is implemented in C and uses hs_free_stable_ptr. Duncan

Bulat Ziganshin wrote:

Hello Don,

Friday, March 20, 2009, 2:34:09 AM, you wrote:

the question is wider - how many other programs already used this behavior? it may be asked on main haskell list. what is the cost of reverting it back for 6.10.*?

We could revert it, but lots of people asked for this feature (predictable finalizers), which is why we put it in 6.10.2. The fix is fiarly easy: use Foreign.Concurrent.mkForeignPtr with a foreign import. Cheers, Simon

...

We must have the gtk2hs team invovled in this discussion. They were using an undocumented feature. It may be trivial to fix.

...

--Don

...

jnf:

...

Hello Simon, I've put a request about the issue on the gtk2hs users mailing list:

...

I've tried a gtk2hs app on ghc 6.10.2 release candidate. It crashes frequently and Simon (as you can read down here) assumes it is gtk2hs problem. My question is: Is this problem known to gtk2hs developers? Is it really a gtk2hs problem? How difficult is it to fix the problem? When will we have a patch to use gtk2hs with 6.10.2, is it already in the repo? However, I'm a little surprised that a little version upgrade from 6.10.1 to 6.10.2 may break all gui apps based on gtk2hs. May it be that many more apps are affected because of this change? What's about wxhaskell e.g.? Well, maybe we have only few Haskell applications around, but usually I wouldn't expect such a dramatic effect from such a moderate upgrade. Is this fix so important to introduce it now? What does it help when it was never officially supported if it causes such troubles?

Jürgen

Simon Marlow-7 wrote:

...

jutaro wrote:

...

I've installed a GUI application based on gtk2hs.

It frequently crashes with the error: leksah: error: a C finalizer called back into Haskell. use Foreign.Concurrent.newForeignPtr for Haskell finalizers.

This error did never occur with the 6.10 released version. It was verified that this happens on different machines. I've no idea how to isolate this bug. This will need to be fixed in gtk2hs. Previously GHC allowed finalizers to call back into Haskell, but this was never officially supported. Now it is officially unsupported, because finalizers created via Foreign.mkForeignPtr are run directly by the garbage collector.

See

http://hackage.haskell.org/trac/ghc/ticket/1364

Cheers, Simon _______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

-- View this message in context: http://www.nabble.com/ANNOUNCE%3A-GHC-6.10.2-Release-Candidate-1-tp22524567p... Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com.

_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

jutaro:

This is the first answer I got from the gtk2hs mailing list. Please consider this issue seriously.

Well there is a simple fix as Simon Marlow wrote,

The fix is fiarly easy: use Foreign.Concurrent.mkForeignPtr with a foreign import.

In fact, if as Axel writes, these finalisers are Haskell functions that are exported using foreign import wrapper, then using Foreign.Concurrent.mkForeignPtr is actually the *simpler* thing to do (you don't need any wrapping and exporting). Manuel

Axel Simon wrote:

...

Phew,

I think we're doomed. We have many, many little methods that take a user-given function, wrap it into a foreign export wrapper which is freed by using an on-destroy callback to Haskell. These functions are most likely installed into some widgets (or other reference-counted objects) that will be eventually destroyed by the Haskell garbage collector. So, basically, we can't easily change Gtk2Hs. It will involve many modifications. I can understand that not allowing callbacks during GC is a great simplification in the runtime but it seemed to be common practice to free Stable and function pointers from within Haskell using a callback.

So, unless I'm wrong on why finalizers call back into Haskell land, then this means that Gtk2Hs is fundamentally broken for the foreseeable future.

Axel.

On Thu, Mar 19, 2009 at 01:43:26PM -0700, jutaro wrote:

Is this fix so important to introduce it now? What does it help when it was never officially supported if it causes such troubles?

The reason it's now not allowed is that 6.10.2 provides guarantees about running C finalizers that 6.10.1 did not. See http://hackage.haskell.org/trac/ghc/ticket/1364 for more details. Thanks Ian