Validate broken on Snow Leopard and Mountain Lion

The Snow Leopard says: ---------------------- Loading package primitive-0.5.1.0 ... linking ... done. Loading package vector-0.10.9.1 ... <command line>: can't load .so/.DLL for: /Users/benl/devel/ghc/ghc-head-validate/libraries/vector/dist-install/build/libHSvector-0.10.9.1-ghc7.7.20131113.dylib (dlopen(/Users/benl/devel/ghc/ghc-head-validate/libraries/vector/dist-install/build/libHSvector-0.10.9.1-ghc7.7.20131113.dylib, 9): Library not loaded: @rpath/primitive-0.5.1.0/libHSprimitive-0.5.1.0-ghc7.7.20131113.dylib Referenced from: /Users/benl/devel/ghc/ghc-head-validate/libraries/vector/dist-install/build/libHSvector-0.10.9.1-ghc7.7.20131113.dylib Reason: image not found) make[1]: *** [libraries/dph/dph-lifted-copy/dist-install/build/Data/Array/Parallel/PArray/PData.o] Error 1 make[1]: *** Waiting for unfinished jobs.... make: *** [all] Error 2 The Mountain Lion says: ----------------------- libraries/primitive/cbits/primitive-memops.c:48:0: warning: dereferencing type-punned pointer will break strict-aliasing rules libraries/primitive/cbits/primitive-memops.c: In function ‘hsprimitive_memset_Float’: libraries/primitive/cbits/primitive-memops.c:49:0: warning: dereferencing type-punned pointer will break strict-aliasing rules libraries/primitive/cbits/primitive-memops.c: In function ‘hsprimitive_memset_Double’: libraries/primitive/cbits/primitive-memops.c:50:0: warning: dereferencing type-punned pointer will break strict-aliasing rules make[1]: *** [libraries/primitive/dist-install/build/cbits/primitive-memops.dyn_o] Error 1 make: *** [all] Error 2 Neither seems pleased. Ben.

The error in snow leopard is the result of my patch [1] for #8266 [2], which ensures that the original build directory is no longer referenced in installed packages/libs.
The patch is just a proper implementation of the original design though.
Perhaps the "relative-dynlib-references" procedure should be called just before the install phase, instead of after the build phase.
That, or the validate-related scripts/binaries should references the libraries in "inplace/lib" instead of the build dirs?
Does validate work for the DPH packages on Linux?
-- Christiaan
[1] https://ghc.haskell.org/trac/ghc/changeset/f213e48447050bf468bc4d91fc4d81040...
[2] https://ghc.haskell.org/trac/ghc/ticket/8266
On Nov 13, 2013, at 4:46 AM, Ben Lippmeier
The Snow Leopard says: ---------------------- Loading package primitive-0.5.1.0 ... linking ... done. Loading package vector-0.10.9.1 ... <command line>: can't load .so/.DLL for: /Users/benl/devel/ghc/ghc-head-validate/libraries/vector/dist-install/build/libHSvector-0.10.9.1-ghc7.7.20131113.dylib (dlopen(/Users/benl/devel/ghc/ghc-head-validate/libraries/vector/dist-install/build/libHSvector-0.10.9.1-ghc7.7.20131113.dylib, 9): Library not loaded: @rpath/primitive-0.5.1.0/libHSprimitive-0.5.1.0-ghc7.7.20131113.dylib Referenced from: /Users/benl/devel/ghc/ghc-head-validate/libraries/vector/dist-install/build/libHSvector-0.10.9.1-ghc7.7.20131113.dylib Reason: image not found) make[1]: *** [libraries/dph/dph-lifted-copy/dist-install/build/Data/Array/Parallel/PArray/PData.o] Error 1 make[1]: *** Waiting for unfinished jobs.... make: *** [all] Error 2
The Mountain Lion says: ----------------------- libraries/primitive/cbits/primitive-memops.c:48:0: warning: dereferencing type-punned pointer will break strict-aliasing rules libraries/primitive/cbits/primitive-memops.c: In function ‘hsprimitive_memset_Float’:
libraries/primitive/cbits/primitive-memops.c:49:0: warning: dereferencing type-punned pointer will break strict-aliasing rules libraries/primitive/cbits/primitive-memops.c: In function ‘hsprimitive_memset_Double’:
libraries/primitive/cbits/primitive-memops.c:50:0: warning: dereferencing type-punned pointer will break strict-aliasing rules make[1]: *** [libraries/primitive/dist-install/build/cbits/primitive-memops.dyn_o] Error 1 make: *** [all] Error 2
Neither seems pleased.
Ben.
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Ben has an excellent point:
if things are broken on mac, lets fix it! Especially since we should be
hitting RC status shortly.... any breakages should be smashed ASAP.
This does raise a point I hope we'll address more aggressively after the
7.8 release, namely trying to keep ghc less broken by default! GHC being
really really really broken between releases is probably the single HUGEST
barrier to more people getting involved in helping GHC dev. And I think
thats a huge problem we need to take more seriously.
On Wed, Nov 13, 2013 at 11:09 PM, Ben Lippmeier
On 13/11/2013, at 8:02 PM, Christiaan Baaij wrote:
Does validate work for the DPH packages on Linux?
"sh validate" runs fine on my Linux machine, but neither of my Macs.
I can also run the dph tests manually on Linux, and they work fine.
Ben.
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

This does raise a point I hope we'll address more aggressively after the 7.8 release, namely trying to keep ghc less broken by default! GHC being really really really broken between releases is probably the single HUGEST barrier to more people getting involved in helping GHC dev. And I think thats a huge problem we need to take more seriously.
I totally agree with this. What it needs is a Mac SWAT team who build GHC on Macs nightly, warn about problems and help fix them.
One recent total Mac breakage was due to the compulsory move to clang instead of gcc. I gather that has caused a lot of pain. I’ve seen a lot about Xcode too, whatever that is.
Anyway, Mac folk, we need you!
Simon
From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Carter Schonwald
Sent: 14 November 2013 04:24
To: Ben Lippmeier
Cc: ghc-devs@haskell.org
Subject: Re: Validate broken on Snow Leopard and Mountain Lion
Ben has an excellent point:
if things are broken on mac, lets fix it! Especially since we should be hitting RC status shortly.... any breakages should be smashed ASAP.
This does raise a point I hope we'll address more aggressively after the 7.8 release, namely trying to keep ghc less broken by default! GHC being really really really broken between releases is probably the single HUGEST barrier to more people getting involved in helping GHC dev. And I think thats a huge problem we need to take more seriously.
On Wed, Nov 13, 2013 at 11:09 PM, Ben Lippmeier
Does validate work for the DPH packages on Linux? "sh validate" runs fine on my Linux machine, but neither of my Macs.
I can also run the dph tests manually on Linux, and they work fine. Ben. _______________________________________________ ghc-devs mailing list ghc-devs@haskell.orgmailto:ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Good idea!
I've setup nightly builds from my Mountain Lion machine.
You can see the result and log files at http://maxsbot.net
If anyone else has machines they can add, email me.
Hopefully we can find one for Snow Leopard and one for Mavericks.
On 14/11/2013, at 8:03 PM, Simon Peyton-Jones
This does raise a point I hope we'll address more aggressively after the 7.8 release, namely trying to keep ghc less broken by default! GHC being really really really broken between releases is probably the single HUGEST barrier to more people getting involved in helping GHC dev. And I think thats a huge problem we need to take more seriously.
I totally agree with this. What it needs is a Mac SWAT team who build GHC on Macs nightly, warn about problems and help fix them.
One recent total Mac breakage was due to the compulsory move to clang instead of gcc. I gather that has caused a lot of pain. I’ve seen a lot about Xcode too, whatever that is.
Anyway, Mac folk, we need you!
Simon
From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Carter Schonwald Sent: 14 November 2013 04:24 To: Ben Lippmeier Cc: ghc-devs@haskell.org Subject: Re: Validate broken on Snow Leopard and Mountain Lion
Ben has an excellent point:
if things are broken on mac, lets fix it! Especially since we should be hitting RC status shortly.... any breakages should be smashed ASAP.
This does raise a point I hope we'll address more aggressively after the 7.8 release, namely trying to keep ghc less broken by default! GHC being really really really broken between releases is probably the single HUGEST barrier to more people getting involved in helping GHC dev. And I think thats a huge problem we need to take more seriously.
On Wed, Nov 13, 2013 at 11:09 PM, Ben Lippmeier
wrote: On 13/11/2013, at 8:02 PM, Christiaan Baaij wrote:
Does validate work for the DPH packages on Linux?
"sh validate" runs fine on my Linux machine, but neither of my Macs.
I can also run the dph tests manually on Linux, and they work fine.
Ben.
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Looking at the "offending" c-code of the primitive library, with my limited C knowledge and understanding of this strict-aliasing, I think the gcc warning on Lion is correct: the strict-aliasing rule is violated.
However, after asking my colleague, we don't think the gcc optimizer will break the code in this particular case.
It's weird that the warning doesn't show on your linux install.
What version of gcc are you using on linux? Is it higher or lower than the on on OS X?
Changing line 22 in primitive-memops.c to:
const int *r = (const int *)(void *)&x;
Makes the warning go away.
For reference, my system:
OS X 10.8.5 (Mountain Lion), X Code 4.6.2, gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)
On Nov 14, 2013, at 5:23 AM, Carter Schonwald
Ben has an excellent point:
if things are broken on mac, lets fix it! Especially since we should be hitting RC status shortly.... any breakages should be smashed ASAP.
This does raise a point I hope we'll address more aggressively after the 7.8 release, namely trying to keep ghc less broken by default! GHC being really really really broken between releases is probably the single HUGEST barrier to more people getting involved in helping GHC dev. And I think thats a huge problem we need to take more seriously.
On Wed, Nov 13, 2013 at 11:09 PM, Ben Lippmeier
wrote: On 13/11/2013, at 8:02 PM, Christiaan Baaij wrote:
Does validate work for the DPH packages on Linux?
"sh validate" runs fine on my Linux machine, but neither of my Macs.
I can also run the dph tests manually on Linux, and they work fine.
Ben.
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

After fixing the warning in the C-code of the primitive package, validate (including dph tests) seems to be working for me:
OVERALL SUMMARY for test run started at Thu Nov 14 11:03:26 2013 CET
0:46:34 spent to go through
3818 total tests, which gave rise to
14930 test cases, of which
11445 were skipped
28 had missing libraries
3394 expected passes
58 expected failures
0 caused framework failures
1 unexpected passes
4 unexpected failures
Unexpected passes:
ghci/linking T3333 (normal)
Unexpected failures:
driver static001 [bad stderr] (normal)
driver/T3007 T3007 [bad exit code] (normal)
perf/compiler T3064 [stat not good enough] (normal)
perf/compiler T4801 [stat too good] (normal)
I'll try again with a 'make maintainer-clean', include profiling libs, and do a ./validate --slow.
On Nov 14, 2013, at 5:23 AM, Carter Schonwald
Ben has an excellent point:
if things are broken on mac, lets fix it! Especially since we should be hitting RC status shortly.... any breakages should be smashed ASAP.
This does raise a point I hope we'll address more aggressively after the 7.8 release, namely trying to keep ghc less broken by default! GHC being really really really broken between releases is probably the single HUGEST barrier to more people getting involved in helping GHC dev. And I think thats a huge problem we need to take more seriously.
On Wed, Nov 13, 2013 at 11:09 PM, Ben Lippmeier
wrote: On 13/11/2013, at 8:02 PM, Christiaan Baaij wrote:
Does validate work for the DPH packages on Linux?
"sh validate" runs fine on my Linux machine, but neither of my Macs.
I can also run the dph tests manually on Linux, and they work fine.
Ben.
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

I'm still puzzled as to how validate failed on snow leopard, but is working correctly on my mountain lion machine after fixing the warning in the primitive package.
Ben: does validate go through on your machine after patching the primitive package?
Here are my results for 'validate --slow':
OVERALL SUMMARY for test run started at Thu Nov 14 14:32:18 2013 CET
4:15:58 spent to go through
3818 total tests, which gave rise to
14934 test cases, of which
3146 were skipped
145 had missing libraries
11465 expected passes
137 expected failures
0 caused framework failures
5 unexpected passes
36 unexpected failures
Unexpected passes:
codeGen/should_run cgrun071 (optllvm)
ghci/linking T3333 (normal)
rename/should_compile/T3103 T3103 (hpc,optasm,optllvm)
Unexpected failures:
../../libraries/unix/tests signals004 [bad exit code] (threaded2)
concurrent/should_run conc012 [bad exit code] (ghci)
driver static001 [bad stderr] (normal)
gadt type-rep [exit code non-0] (hpc,optasm,threaded2,dyn,optllvm)
ghc-api/apirecomp001 apirecomp001 [bad stderr] (normal)
lib/integer integerConstantFolding [bad stderr] (normal)
perf/compiler T3064 [stat not good enough] (normal)
perf/compiler T3294 [stderr mismatch] (normal)
perf/compiler T4801 [stat too good] (normal)
perf/compiler T5642 [stat not good enough] (normal)
perf/haddock haddock.Cabal [stat not good enough] (normal)
perf/haddock haddock.compiler [stat not good enough] (normal)
perf/space_leaks space_leak_001 [stat too good] (hpc,optasm,dyn,optllvm)
polykinds T6068 [bad stderr] (ghci)
rts T7919 [exit code non-0] (normal,hpc,optasm,threaded1,threaded2,optllvm)
th/TH_import_loop TH_import_loop [bad stdout or stderr] (ghci)
typecheck/should_run T7861 [bad stderr] (normal,hpc,optasm,threaded1,threaded2,dyn,optllvm)
typecheck/should_run T7861 [bad stdout or stderr] (ghci)
On Nov 14, 2013, at 12:04 PM, Christiaan Baaij
After fixing the warning in the C-code of the primitive package, validate (including dph tests) seems to be working for me:
OVERALL SUMMARY for test run started at Thu Nov 14 11:03:26 2013 CET 0:46:34 spent to go through 3818 total tests, which gave rise to 14930 test cases, of which 11445 were skipped
28 had missing libraries 3394 expected passes 58 expected failures
0 caused framework failures 1 unexpected passes 4 unexpected failures
Unexpected passes: ghci/linking T3333 (normal)
Unexpected failures: driver static001 [bad stderr] (normal) driver/T3007 T3007 [bad exit code] (normal) perf/compiler T3064 [stat not good enough] (normal) perf/compiler T4801 [stat too good] (normal)
I'll try again with a 'make maintainer-clean', include profiling libs, and do a ./validate --slow.
On Nov 14, 2013, at 5:23 AM, Carter Schonwald
wrote: Ben has an excellent point:
if things are broken on mac, lets fix it! Especially since we should be hitting RC status shortly.... any breakages should be smashed ASAP.
This does raise a point I hope we'll address more aggressively after the 7.8 release, namely trying to keep ghc less broken by default! GHC being really really really broken between releases is probably the single HUGEST barrier to more people getting involved in helping GHC dev. And I think thats a huge problem we need to take more seriously.
On Wed, Nov 13, 2013 at 11:09 PM, Ben Lippmeier
wrote: On 13/11/2013, at 8:02 PM, Christiaan Baaij wrote:
Does validate work for the DPH packages on Linux?
"sh validate" runs fine on my Linux machine, but neither of my Macs.
I can also run the dph tests manually on Linux, and they work fine.
Ben.
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Is this pushed? Edward Excerpts from Christiaan Baaij's message of 2013-11-15 00:04:30 -0800:
I'm still puzzled as to how validate failed on snow leopard, but is working correctly on my mountain lion machine after fixing the warning in the primitive package. Ben: does validate go through on your machine after patching the primitive package?
Here are my results for 'validate --slow':
OVERALL SUMMARY for test run started at Thu Nov 14 14:32:18 2013 CET 4:15:58 spent to go through 3818 total tests, which gave rise to 14934 test cases, of which 3146 were skipped
145 had missing libraries 11465 expected passes 137 expected failures
0 caused framework failures 5 unexpected passes 36 unexpected failures
Unexpected passes: codeGen/should_run cgrun071 (optllvm) ghci/linking T3333 (normal) rename/should_compile/T3103 T3103 (hpc,optasm,optllvm)
Unexpected failures: ../../libraries/unix/tests signals004 [bad exit code] (threaded2) concurrent/should_run conc012 [bad exit code] (ghci) driver static001 [bad stderr] (normal) gadt type-rep [exit code non-0] (hpc,optasm,threaded2,dyn,optllvm) ghc-api/apirecomp001 apirecomp001 [bad stderr] (normal) lib/integer integerConstantFolding [bad stderr] (normal) perf/compiler T3064 [stat not good enough] (normal) perf/compiler T3294 [stderr mismatch] (normal) perf/compiler T4801 [stat too good] (normal) perf/compiler T5642 [stat not good enough] (normal) perf/haddock haddock.Cabal [stat not good enough] (normal) perf/haddock haddock.compiler [stat not good enough] (normal) perf/space_leaks space_leak_001 [stat too good] (hpc,optasm,dyn,optllvm) polykinds T6068 [bad stderr] (ghci) rts T7919 [exit code non-0] (normal,hpc,optasm,threaded1,threaded2,optllvm) th/TH_import_loop TH_import_loop [bad stdout or stderr] (ghci) typecheck/should_run T7861 [bad stderr] (normal,hpc,optasm,threaded1,threaded2,dyn,optllvm) typecheck/should_run T7861 [bad stdout or stderr] (ghci)
On Nov 14, 2013, at 12:04 PM, Christiaan Baaij
wrote: After fixing the warning in the C-code of the primitive package, validate (including dph tests) seems to be working for me:
OVERALL SUMMARY for test run started at Thu Nov 14 11:03:26 2013 CET 0:46:34 spent to go through 3818 total tests, which gave rise to 14930 test cases, of which 11445 were skipped
28 had missing libraries 3394 expected passes 58 expected failures
0 caused framework failures 1 unexpected passes 4 unexpected failures
Unexpected passes: ghci/linking T3333 (normal)
Unexpected failures: driver static001 [bad stderr] (normal) driver/T3007 T3007 [bad exit code] (normal) perf/compiler T3064 [stat not good enough] (normal) perf/compiler T4801 [stat too good] (normal)
I'll try again with a 'make maintainer-clean', include profiling libs, and do a ./validate --slow.
On Nov 14, 2013, at 5:23 AM, Carter Schonwald
wrote: Ben has an excellent point:
if things are broken on mac, lets fix it! Especially since we should be hitting RC status shortly.... any breakages should be smashed ASAP.
This does raise a point I hope we'll address more aggressively after the 7.8 release, namely trying to keep ghc less broken by default! GHC being really really really broken between releases is probably the single HUGEST barrier to more people getting involved in helping GHC dev. And I think thats a huge problem we need to take more seriously.
On Wed, Nov 13, 2013 at 11:09 PM, Ben Lippmeier
wrote: On 13/11/2013, at 8:02 PM, Christiaan Baaij wrote:
Does validate work for the DPH packages on Linux?
"sh validate" runs fine on my Linux machine, but neither of my Macs.
I can also run the dph tests manually on Linux, and they work fine.
Ben.
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Raised as issue on github: https://github.com/haskell/primitive/issues/4
On Nov 19, 2013, at 1:17 AM, Edward Z. Yang
Is this pushed?
Edward
Excerpts from Christiaan Baaij's message of 2013-11-15 00:04:30 -0800:
I'm still puzzled as to how validate failed on snow leopard, but is working correctly on my mountain lion machine after fixing the warning in the primitive package. Ben: does validate go through on your machine after patching the primitive package?
Here are my results for 'validate --slow':
OVERALL SUMMARY for test run started at Thu Nov 14 14:32:18 2013 CET 4:15:58 spent to go through 3818 total tests, which gave rise to 14934 test cases, of which 3146 were skipped
145 had missing libraries 11465 expected passes 137 expected failures
0 caused framework failures 5 unexpected passes 36 unexpected failures
Unexpected passes: codeGen/should_run cgrun071 (optllvm) ghci/linking T3333 (normal) rename/should_compile/T3103 T3103 (hpc,optasm,optllvm)
Unexpected failures: ../../libraries/unix/tests signals004 [bad exit code] (threaded2) concurrent/should_run conc012 [bad exit code] (ghci) driver static001 [bad stderr] (normal) gadt type-rep [exit code non-0] (hpc,optasm,threaded2,dyn,optllvm) ghc-api/apirecomp001 apirecomp001 [bad stderr] (normal) lib/integer integerConstantFolding [bad stderr] (normal) perf/compiler T3064 [stat not good enough] (normal) perf/compiler T3294 [stderr mismatch] (normal) perf/compiler T4801 [stat too good] (normal) perf/compiler T5642 [stat not good enough] (normal) perf/haddock haddock.Cabal [stat not good enough] (normal) perf/haddock haddock.compiler [stat not good enough] (normal) perf/space_leaks space_leak_001 [stat too good] (hpc,optasm,dyn,optllvm) polykinds T6068 [bad stderr] (ghci) rts T7919 [exit code non-0] (normal,hpc,optasm,threaded1,threaded2,optllvm) th/TH_import_loop TH_import_loop [bad stdout or stderr] (ghci) typecheck/should_run T7861 [bad stderr] (normal,hpc,optasm,threaded1,threaded2,dyn,optllvm) typecheck/should_run T7861 [bad stdout or stderr] (ghci)
On Nov 14, 2013, at 12:04 PM, Christiaan Baaij
wrote: After fixing the warning in the C-code of the primitive package, validate (including dph tests) seems to be working for me:
OVERALL SUMMARY for test run started at Thu Nov 14 11:03:26 2013 CET 0:46:34 spent to go through 3818 total tests, which gave rise to 14930 test cases, of which 11445 were skipped
28 had missing libraries 3394 expected passes 58 expected failures
0 caused framework failures 1 unexpected passes 4 unexpected failures
Unexpected passes: ghci/linking T3333 (normal)
Unexpected failures: driver static001 [bad stderr] (normal) driver/T3007 T3007 [bad exit code] (normal) perf/compiler T3064 [stat not good enough] (normal) perf/compiler T4801 [stat too good] (normal)
I'll try again with a 'make maintainer-clean', include profiling libs, and do a ./validate --slow.
On Nov 14, 2013, at 5:23 AM, Carter Schonwald
wrote: Ben has an excellent point:
if things are broken on mac, lets fix it! Especially since we should be hitting RC status shortly.... any breakages should be smashed ASAP.
This does raise a point I hope we'll address more aggressively after the 7.8 release, namely trying to keep ghc less broken by default! GHC being really really really broken between releases is probably the single HUGEST barrier to more people getting involved in helping GHC dev. And I think thats a huge problem we need to take more seriously.
On Wed, Nov 13, 2013 at 11:09 PM, Ben Lippmeier
wrote: On 13/11/2013, at 8:02 PM, Christiaan Baaij wrote:
Does validate work for the DPH packages on Linux?
"sh validate" runs fine on my Linux machine, but neither of my Macs.
I can also run the dph tests manually on Linux, and they work fine.
Ben.
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
participants (6)
-
Ben Lippmeier
-
Carter Schonwald
-
Christiaan Baaij
-
Edward Z. Yang
-
Maxwell Swadling
-
Simon Peyton-Jones