Hugs faster and more reliable than GHC for large list monad 'do' block

older
Haskell Weekly News: Issue 127 -...

Henning Thielemann

6 Aug 2009 6 Aug '09

4:39 p.m.

Today a student has shown me a program that consists of a large 'do' block for the list monad. The program looks like do x1 <- [0..3] x2 <- [0..2] ... x600 <- [0..5] guard (x1+x2+2*x3 >= 0) ... return (x1,x2,....,x600) It was actually generated by another program. The results were: GHC-6.4 was not able to compile that program at all, because it stopped because of memory exhaustion. GHC-6.8.2 finished compilation after two minutes but the program aborted quickly because of a corrupt thunk identifier. GHC-6.10 not yet tested. Hugs-2006 executed the program without complaining and showed the first result after a few seconds: (0,0,0,0,0,...,0). Eventually the program must run on a Linux cluster with a not up-to-date Linux kernel, that is, I suspect newer GHC versions cannot be used due to the 'timer_create' problem. (At least, the 'cabal' executable that I generated with a GHC-6.8.2 had this problem when running on the cluster which reminded me on the problems with GHC-6.8 itself running on older Linux kernels.) Since the list monad sorts the variable values in lexicographic order which is inappropriate for the considered problem, I recommended the use of control-monad-omega. Luke, I hope this monad can cope with 600 variables. :-)

Show replies by date

Jason Dagit

6 Aug 6 Aug

4:51 p.m.

On Thu, Aug 6, 2009 at 1:39 PM, Henning Thielemann < lemming@henning-thielemann.de> wrote:

...

Today a student has shown me a program that consists of a large 'do' block for the list monad. The program looks like

do x1 <- [0..3] x2 <- [0..2] ... x600 <- [0..5] guard (x1+x2+2*x3 >= 0) ... return (x1,x2,....,x600)

It was actually generated by another program. The results were:

GHC-6.4 was not able to compile that program at all, because it stopped because of memory exhaustion.

Did you try playing with optimization options? I think someone just mentioned that compiling large amounts of static data in GHC is problematic and recommended something like -O0 to prevent GHC from analyzing it. Of course, I wouldn't recommend having unoptimized code, but I'm curious how that changes the results here.

...

GHC-6.8.2 finished compilation after two minutes but the program aborted quickly because of a corrupt thunk identifier.

Oh that's icky.

...

GHC-6.10 not yet tested. Hugs-2006 executed the program without complaining and showed the first result after a few seconds: (0,0,0,0,0,...,0).

Eventually the program must run on a Linux cluster with a not up-to-date Linux kernel, that is, I suspect newer GHC versions cannot be used due to the 'timer_create' problem. (At least, the 'cabal' executable that I generated with a GHC-6.8.2 had this problem when running on the cluster which reminded me on the problems with GHC-6.8 itself running on older Linux kernels.)

I just googled this and found this: http://www.haskell.org/pipermail/glasgow-haskell-users/2008-March/014498.htm... That was linked from this discussion: http://www.nabble.com/ghc-6.8.2-and-timer_create-error-td16160635.html The second link seems to indicate that it's not a problem of antiquity but instead one related to what configure finds when building GHC for that system. In other words, it should be possible to make a version of GHC locally that doesn't use timer_create and then your student should be good to go. Hope that helps, Jason

Henning Thielemann

5:59 p.m.

On Thu, 6 Aug 2009, Jason Dagit wrote:

...

'timer_create' problem. (At least, the 'cabal' executable that I generated with a GHC-6.8.2 had this problem when running on the cluster which reminded me on the problems with GHC-6.8 itself running on older Linux kernels.)

I just googled this and found this: http://www.haskell.org/pipermail/glasgow-haskell-users/2008-March/014498.htm... That was linked from this discussion: http://www.nabble.com/ghc-6.8.2-and-timer_create-error-td16160635.html

The second link seems to indicate that it's not a problem of antiquity but instead one related to what configure finds when building GHC for that system. In other words, it should be possible to make a version of GHC locally that doesn't use timer_create and then your student should be good to go.

Thanks for the pointer. But it means building GHC myself, which I have no experience with. :-(

Jeff Heard

7:34 p.m.

Yes, the GHC compiler will work on older kernels and CentOS kernels if you bootstrap it with 6.4 On Thu, Aug 6, 2009 at 4:51 PM, Jason Dagit wrote:

...

On Thu, Aug 6, 2009 at 1:39 PM, Henning Thielemann wrote:

...
Today a student has shown me a program that consists of a large 'do' block for the list monad. The program looks like

do x1 <- [0..3] x2 <- [0..2] ... x600 <- [0..5] guard (x1+x2+2*x3 >= 0) ... return (x1,x2,....,x600)

It was actually generated by another program. The results were:

GHC-6.4 was not able to compile that program at all, because it stopped because of memory exhaustion.

Did you try playing with optimization options? I think someone just mentioned that compiling large amounts of static data in GHC is problematic and recommended something like -O0 to prevent GHC from analyzing it. Of course, I wouldn't recommend having unoptimized code, but I'm curious how that changes the results here.

...
GHC-6.8.2 finished compilation after two minutes but the program aborted quickly because of a corrupt thunk identifier.

Oh that's icky.

...
GHC-6.10 not yet tested. Hugs-2006 executed the program without complaining and showed the first result after a few seconds: (0,0,0,0,0,...,0).

Eventually the program must run on a Linux cluster with a not up-to-date Linux kernel, that is, I suspect newer GHC versions cannot be used due to the 'timer_create' problem. (At least, the 'cabal' executable that I generated with a GHC-6.8.2 had this problem when running on the cluster which reminded me on the problems with GHC-6.8 itself running on older Linux kernels.)

I just googled this and found this: http://www.haskell.org/pipermail/glasgow-haskell-users/2008-March/014498.htm... That was linked from this discussion: http://www.nabble.com/ghc-6.8.2-and-timer_create-error-td16160635.html

The second link seems to indicate that it's not a problem of antiquity but instead one related to what configure finds when building GHC for that system. In other words, it should be possible to make a version of GHC locally that doesn't use timer_create and then your student should be good to go.

Hope that helps, Jason

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Dan Weston

4:59 p.m.

I assume for the return line, you meant to return a list, not a tuple. ghc doesn't support a 600-tuple. In any case, returning a list, I have verified that this problem exists in ghc 6.10.3, for -O0 and -O2. For -O0, it compiles and links fine, but gives this runtime message: z: internal error: scavenge: unimplemented/strange closure type -1 @ 0x2a95a8e000 (GHC version 6.10.3 for x86_64_unknown_linux) Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug Abort Maybe it is attempting to unroll these loops, even with -O0? Dan Henning Thielemann wrote:

...

Today a student has shown me a program that consists of a large 'do' block for the list monad. The program looks like

do x1 <- [0..3] x2 <- [0..2] ... x600 <- [0..5] guard (x1+x2+2*x3 >= 0) ... return (x1,x2,....,x600)

It was actually generated by another program. The results were:

GHC-6.4 was not able to compile that program at all, because it stopped because of memory exhaustion. GHC-6.8.2 finished compilation after two minutes but the program aborted quickly because of a corrupt thunk identifier. GHC-6.10 not yet tested. Hugs-2006 executed the program without complaining and showed the first result after a few seconds: (0,0,0,0,0,...,0).

Eventually the program must run on a Linux cluster with a not up-to-date Linux kernel, that is, I suspect newer GHC versions cannot be used due to the 'timer_create' problem. (At least, the 'cabal' executable that I generated with a GHC-6.8.2 had this problem when running on the cluster which reminded me on the problems with GHC-6.8 itself running on older Linux kernels.)

Since the list monad sorts the variable values in lexicographic order which is inappropriate for the considered problem, I recommended the use of control-monad-omega. Luke, I hope this monad can cope with 600 variables. :-) _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Neil Mitchell

5:26 p.m.

Hi I think the issue you're running in to with 6.4 is this one: http://hackage.haskell.org/trac/ghc/ticket/830 - known and fixed a while back. Thanks Neil On Thu, Aug 6, 2009 at 9:59 PM, Dan Weston wrote:

...

I assume for the return line, you meant to return a list, not a tuple. ghc doesn't support a 600-tuple. In any case, returning a list, I have verified that this problem exists in ghc 6.10.3, for -O0 and -O2.

For -O0, it compiles and links fine, but gives this runtime message:

z: internal error: scavenge: unimplemented/strange closure type -1 @ 0x2a95a8e000 (GHC version 6.10.3 for x86_64_unknown_linux) Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug Abort

Maybe it is attempting to unroll these loops, even with -O0?

Dan

Henning Thielemann wrote:

...
Today a student has shown me a program that consists of a large 'do' block for the list monad. The program looks like

do x1 <- [0..3] x2 <- [0..2] ... x600 <- [0..5] guard (x1+x2+2*x3 >= 0) ... return (x1,x2,....,x600)

It was actually generated by another program. The results were:

GHC-6.4 was not able to compile that program at all, because it stopped because of memory exhaustion. GHC-6.8.2 finished compilation after two minutes but the program aborted quickly because of a corrupt thunk identifier. GHC-6.10 not yet tested. Hugs-2006 executed the program without complaining and showed the first result after a few seconds: (0,0,0,0,0,...,0).

Eventually the program must run on a Linux cluster with a not up-to-date Linux kernel, that is, I suspect newer GHC versions cannot be used due to the 'timer_create' problem. (At least, the 'cabal' executable that I generated with a GHC-6.8.2 had this problem when running on the cluster which reminded me on the problems with GHC-6.8 itself running on older Linux kernels.)

Since the list monad sorts the variable values in lexicographic order which is inappropriate for the considered problem, I recommended the use of control-monad-omega. Luke, I hope this monad can cope with 600 variables. :-) _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Dan Weston

7:25 p.m.

No, I am using the latest released ghc:

...

ghc --version The Glorious Glasgow Haskell Compilation System, version 6.10.4

[ z.hs is attached ]

...

time ghc -O0 --make z.hs [1 of 1] Compiling Main ( z.hs, z.o ) Linking z ... 14.422u 0.630s 0:15.10 99.6% 0+0k 0+0io 0pf+0w

...

time ./z z: internal error: scavenge: unimplemented/strange closure type -1 @ 0x2a95a8e000 (GHC version 6.10.4 for x86_64_unknown_linux) Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug Abort 0.007u 0.007s 0:00.02 0.0% 0+0k 0+0io 0pf+0w

Dan Neil Mitchell wrote:

...

Hi

I think the issue you're running in to with 6.4 is this one: http://hackage.haskell.org/trac/ghc/ticket/830 - known and fixed a while back.

Thanks

Neil

On Thu, Aug 6, 2009 at 9:59 PM, Dan Weston wrote:

...
I assume for the return line, you meant to return a list, not a tuple. ghc doesn't support a 600-tuple. In any case, returning a list, I have verified that this problem exists in ghc 6.10.3, for -O0 and -O2.

For -O0, it compiles and links fine, but gives this runtime message:

z: internal error: scavenge: unimplemented/strange closure type -1 @ 0x2a95a8e000 (GHC version 6.10.3 for x86_64_unknown_linux) Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug Abort

Maybe it is attempting to unroll these loops, even with -O0?

Dan

Henning Thielemann wrote:

...
Today a student has shown me a program that consists of a large 'do' block for the list monad. The program looks like

do x1 <- [0..3] x2 <- [0..2] ... x600 <- [0..5] guard (x1+x2+2*x3 >= 0) ... return (x1,x2,....,x600)

It was actually generated by another program. The results were:

GHC-6.4 was not able to compile that program at all, because it stopped because of memory exhaustion. GHC-6.8.2 finished compilation after two minutes but the program aborted quickly because of a corrupt thunk identifier. GHC-6.10 not yet tested. Hugs-2006 executed the program without complaining and showed the first result after a few seconds: (0,0,0,0,0,...,0).

Eventually the program must run on a Linux cluster with a not up-to-date Linux kernel, that is, I suspect newer GHC versions cannot be used due to the 'timer_create' problem. (At least, the 'cabal' executable that I generated with a GHC-6.8.2 had this problem when running on the cluster which reminded me on the problems with GHC-6.8 itself running on older Linux kernels.)

Since the list monad sorts the variable values in lexicographic order which is inappropriate for the considered problem, I recommended the use of control-monad-omega. Luke, I hope this monad can cope with 600 variables. :-) _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Dan Weston

7:37 p.m.

I should clarify that this was done on an older kernel (bootstrapped from ghc 6.4 as Jeff Heard suggested), in case that makes any difference: Main memory size: 7971 Mbytes 1 GenuineIntel Intel(R) Xeon(TM) CPU 3.40GHz processor Swap Size: 2047 Mbytes Num Processors: 1 Processor Type: Intel(R) Xeon(TM) CPU 3.40GHz x64 Clock Speed: 3400 MHZ OS: Linux 2.6.9-42.0.3.EL.spi OS-VERSION: CentOS release 4.4 (Final) OS-HW: x86_64 Dan Weston wrote:

...

No, I am using the latest released ghc:

...
ghc --version The Glorious Glasgow Haskell Compilation System, version 6.10.4

[ z.hs is attached ]

...
time ghc -O0 --make z.hs [1 of 1] Compiling Main ( z.hs, z.o ) Linking z ... 14.422u 0.630s 0:15.10 99.6% 0+0k 0+0io 0pf+0w

...
time ./z z: internal error: scavenge: unimplemented/strange closure type -1 @ 0x2a95a8e000 (GHC version 6.10.4 for x86_64_unknown_linux) Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug Abort 0.007u 0.007s 0:00.02 0.0% 0+0k 0+0io 0pf+0w

Dan

Neil Mitchell wrote:

...
Hi

I think the issue you're running in to with 6.4 is this one: http://hackage.haskell.org/trac/ghc/ticket/830 - known and fixed a while back.

Thanks

Neil

On Thu, Aug 6, 2009 at 9:59 PM, Dan Weston wrote:

...
I assume for the return line, you meant to return a list, not a tuple. ghc doesn't support a 600-tuple. In any case, returning a list, I have verified that this problem exists in ghc 6.10.3, for -O0 and -O2.

For -O0, it compiles and links fine, but gives this runtime message:

z: internal error: scavenge: unimplemented/strange closure type -1 @ 0x2a95a8e000 (GHC version 6.10.3 for x86_64_unknown_linux) Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug Abort

Maybe it is attempting to unroll these loops, even with -O0?

Dan

Henning Thielemann wrote:

...
Today a student has shown me a program that consists of a large 'do' block for the list monad. The program looks like

do x1 <- [0..3] x2 <- [0..2] ... x600 <- [0..5] guard (x1+x2+2*x3 >= 0) ... return (x1,x2,....,x600)

It was actually generated by another program. The results were:

GHC-6.4 was not able to compile that program at all, because it stopped because of memory exhaustion. GHC-6.8.2 finished compilation after two minutes but the program aborted quickly because of a corrupt thunk identifier. GHC-6.10 not yet tested. Hugs-2006 executed the program without complaining and showed the first result after a few seconds: (0,0,0,0,0,...,0).

Eventually the program must run on a Linux cluster with a not up-to-date Linux kernel, that is, I suspect newer GHC versions cannot be used due to the 'timer_create' problem. (At least, the 'cabal' executable that I generated with a GHC-6.8.2 had this problem when running on the cluster which reminded me on the problems with GHC-6.8 itself running on older Linux kernels.)

Since the list monad sorts the variable values in lexicographic order which is inappropriate for the considered problem, I recommended the use of control-monad-omega. Luke, I hope this monad can cope with 600 variables. :-) _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Felipe Lessa

10:02 p.m.

New subject: Hugs faster and more reliable than GHC for large list monad 'do' block

On Thu, Aug 06, 2009 at 04:25:07PM -0700, Dan Weston wrote:

...

...
ghc --version The Glorious Glasgow Haskell Compilation System, version 6.10.4

Confirmed on Gento amd64 with custom-built GHC from Haskell overlay: $ ghc --version The Glorious Glasgow Haskell Compilation System, version 6.10.4 $ uname -a Linux kira 2.6.30-tuxonice-r4 #1 SMP Sat Jul 25 15:28:05 BRT 2009 x86_64 Intel(R) Core(TM)2 Duo CPU T5450 @ 1.66GHz GenuineIntel GNU/Linux -- Felipe.

Don Stewart

10:34 p.m.

New subject: Hugs faster and more reliable than GHC for large list monad 'do' block

felipe.lessa:

...

On Thu, Aug 06, 2009 at 04:25:07PM -0700, Dan Weston wrote:

...
...
ghc --version The Glorious Glasgow Haskell Compilation System, version 6.10.4

Confirmed on Gento amd64 with custom-built GHC from Haskell overlay:

$ ghc --version The Glorious Glasgow Haskell Compilation System, version 6.10.4

$ uname -a Linux kira 2.6.30-tuxonice-r4 #1 SMP Sat Jul 25 15:28:05 BRT 2009 x86_64 Intel(R) Core(TM)2 Duo CPU T5450 @ 1.66GHz GenuineIntel GNU/Linux

Someone please file a bug report, http://hackage.haskell.org/trac/ghc/newticket?type=bug

Henning Thielemann

8 Aug 8 Aug

1:33 p.m.

New subject: Hugs faster and more reliable than GHC for large list monad 'do' block

On Thu, 6 Aug 2009, Don Stewart wrote:

...

Someone please file a bug report,

http://hackage.haskell.org/trac/ghc/newticket?type=bug

Done, Don! http://hackage.haskell.org/trac/ghc/ticket/3424

Henning Thielemann

7 Aug 7 Aug

5:29 a.m.

On Thu, 6 Aug 2009, Dan Weston wrote:

...

Neil Mitchell wrote:

...
Hi

I think the issue you're running in to with 6.4 is this one: http://hackage.haskell.org/trac/ghc/ticket/830 - known and fixed a while back.

No, I am using the latest released ghc:

I think Neil refered to my experiences with GHC-6.4, namely that the 'do' block could not be compiled, at all. Neil, thanks for pointing to the ticket. I think it is the problem we encountered.

Henning Thielemann

6 Aug 6 Aug

5:58 p.m.

On Thu, 6 Aug 2009, Dan Weston wrote:

...

I assume for the return line, you meant to return a list, not a tuple. ghc doesn't support a 600-tuple.

Maybe that it was a list.

...

In any case, returning a list, I have verified that this problem exists in ghc 6.10.3, for -O0 and -O2.

For -O0, it compiles and links fine, but gives this runtime message:

z: internal error: scavenge: unimplemented/strange closure type -1 @ 0x2a95a8e000 (GHC version 6.10.3 for x86_64_unknown_linux) Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug Abort

Indeed, this is also what we got from the executable built by GHC-6.8.2.

5769

Age (days ago)

5771

Last active (days ago)

List overview

Download

12 comments

7 participants

participants (7)

Dan Weston
Don Stewart
Felipe Lessa
Henning Thielemann
Jason Dagit
Jeff Heard
Neil Mitchell

Hugs faster and more reliable than GHC for large list monad 'do' block

tags

participants (7)