Debugging a test suite freeze

David Feuer

30 Nov 2022 30 Nov '22

5:33 p.m.

The logict-sequence test suite consistently freezes up on the last test when compiled with GHC 9.4.3. logict-sequence and its test suite don't do anything special with that GHC/base version as far as I can see, or anything particularly strange in general, so I'm pretty confident the problem lies elsewhere. My best guesses are hedgehog, tasty, and tasty-hedgehog. Has anyone else run into weird issues with any/all of these on GHC 9.4.3? David

Show replies by date

David Feuer

30 Nov 30 Nov

5:46 p.m.

In particular, I think tasty does some tricky concurrency. Have there been changes in that recently, either in GHC itself or in async, that might be responsible? On Wed, Nov 30, 2022, 5:33 PM David Feuer wrote:

...

The logict-sequence test suite consistently freezes up on the last test when compiled with GHC 9.4.3. logict-sequence and its test suite don't do anything special with that GHC/base version as far as I can see, or anything particularly strange in general, so I'm pretty confident the problem lies elsewhere. My best guesses are hedgehog, tasty, and tasty-hedgehog. Has anyone else run into weird issues with any/all of these on GHC 9.4.3?

David

Doug Burke

10:36 p.m.

I had a test suite loop indefinitely with ghc 9.4.3 - see https://gitlab.haskell.org/ghc/ghc/-/issues/22416 - but to be honest I don't really understand what the code did that lead to the problem, so dont know how relevant it is for you. Doug On Wed, Nov 30, 2022 at 5:46 PM David Feuer wrote:

...

In particular, I think tasty does some tricky concurrency. Have there been changes in that recently, either in GHC itself or in async, that might be responsible?

On Wed, Nov 30, 2022, 5:33 PM David Feuer wrote:

...
The logict-sequence test suite consistently freezes up on the last test when compiled with GHC 9.4.3. logict-sequence and its test suite don't do anything special with that GHC/base version as far as I can see, or anything particularly strange in general, so I'm pretty confident the problem lies elsewhere. My best guesses are hedgehog, tasty, and tasty-hedgehog. Has anyone else run into weird issues with any/all of these on GHC 9.4.3?

David

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

David Feuer

11:15 p.m.

I don't see anything about test suites freezing/looping in that ticket. Did you mean a different one? Note: it seems pretty clear that my test suite is *not* looping: when it freezes, the process doesn't burn CPU or memory. It seems there's a deadlock somewhere, but it's all *extremely* mysterious to me. Most mysteriously, only the last test case seems to lead to this problem, and it does so even when *dramatically* simplified in various ways (e.g., if I just `liftIO (print s)` instead of traversing with an arbitrary function, and then finish off with `3 === 3`). It would be easy to believe that the problem was in my Hedgehog test case generation code if not for the fact that it works reliably on all versions of GHC from 7.8 through 9.2! So either something is wrong with GHC 9.4 or using 9.4 is getting me a bad version of some dependency. On Wed, Nov 30, 2022 at 10:36 PM Doug Burke wrote:

...

I had a test suite loop indefinitely with ghc 9.4.3 - see https://gitlab.haskell.org/ghc/ghc/-/issues/22416 - but to be honest I don't really understand what the code did that lead to the problem, so dont know how relevant it is for you.

Doug

On Wed, Nov 30, 2022 at 5:46 PM David Feuer wrote:

...
In particular, I think tasty does some tricky concurrency. Have there been changes in that recently, either in GHC itself or in async, that might be responsible?

On Wed, Nov 30, 2022, 5:33 PM David Feuer wrote:

...
The logict-sequence test suite consistently freezes up on the last test when compiled with GHC 9.4.3. logict-sequence and its test suite don't do anything special with that GHC/base version as far as I can see, or anything particularly strange in general, so I'm pretty confident the problem lies elsewhere. My best guesses are hedgehog, tasty, and tasty-hedgehog. Has anyone else run into weird issues with any/all of these on GHC 9.4.3?

David

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Doug Burke

1 Dec 1 Dec

8:38 a.m.

Sorry, mea culpa. I got confused - there was a test loop with 9.2 (which sounded a bit like your case, but it was in 9.2 and not 9.4) and an unrelated issue with 9.4 that was found at the same time and I got that mixed up Doug On Wed, Nov 30, 2022 at 11:15 PM David Feuer wrote:

...

I don't see anything about test suites freezing/looping in that ticket. Did you mean a different one? Note: it seems pretty clear that my test suite is *not* looping: when it freezes, the process doesn't burn CPU or memory. It seems there's a deadlock somewhere, but it's all *extremely* mysterious to me. Most mysteriously, only the last test case seems to lead to this problem, and it does so even when *dramatically* simplified in various ways (e.g., if I just `liftIO (print s)` instead of traversing with an arbitrary function, and then finish off with `3 === 3`). It would be easy to believe that the problem was in my Hedgehog test case generation code if not for the fact that it works reliably on all versions of GHC from 7.8 through 9.2! So either something is wrong with GHC 9.4 or using 9.4 is getting me a bad version of some dependency.

On Wed, Nov 30, 2022 at 10:36 PM Doug Burke wrote:

...
I had a test suite loop indefinitely with ghc 9.4.3 - see https://gitlab.haskell.org/ghc/ghc/-/issues/22416 - but to be honest I don't really understand what the code did that lead to the problem, so dont know how relevant it is for you.

Doug

On Wed, Nov 30, 2022 at 5:46 PM David Feuer wrote:

...
In particular, I think tasty does some tricky concurrency. Have there been changes in that recently, either in GHC itself or in async, that might be responsible?

On Wed, Nov 30, 2022, 5:33 PM David Feuer wrote:

...
The logict-sequence test suite consistently freezes up on the last test when compiled with GHC 9.4.3. logict-sequence and its test suite don't do anything special with that GHC/base version as far as I can see, or anything particularly strange in general, so I'm pretty confident the problem lies elsewhere. My best guesses are hedgehog, tasty, and tasty-hedgehog. Has anyone else run into weird issues with any/all of these on GHC 9.4.3?

David

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Li-yao Xia

7:14 p.m.

GHC 9.4.3 with -O2 is miscompiling the derived Show and Eq instances for SeqT in logict-sequence. main :: IO () main = print (pure 0 :: Seq Int) -- -O0, -O1: prints the Seq -- -O2: <<loop>> On top of that, both hedgehog and hspec fail to catch `<<loop>>`. main :: IO () main = defaultMain [ check $ property $ do let x = x :: Int -- obvious loop compiles to a thrown exception <<loop>> if x == x then pure () else pure () ] -- "thread blocked indefinitely in an STM transaction" -- meaning some thread is not getting cleaned up properly main :: IO () main = hspec $ do describe "work" $ do it "works" $ do let x = x :: Int -- obvious loop compiles to a thrown exception <<loop>> if x == x then pure () else pure () :: IO () --- hangs Cheers, Li-yao On 2022-11-30 10:33 PM, David Feuer wrote: The logict-sequence test suite consistently freezes up on the last test when compiled with GHC 9.4.3. logict-sequence and its test suite don't do anything special with that GHC/base version as far as I can see, or anything particularly strange in general, so I'm pretty confident the problem lies elsewhere. My best guesses are hedgehog, tasty, and tasty-hedgehog. Has anyone else run into weird issues with any/all of these on GHC 9.4.3? David _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

David Feuer

9:51 p.m.

Wow, that's ... interesting. Could this have to do with the way the constraints are written? Has there been some change in UndecidableInstances? I'll try simplifying the Eq, Ord, Show, etc. constraints manually and see if that fixes it. On Thu, Dec 1, 2022, 7:14 PM Li-yao Xia wrote:

...

GHC 9.4.3 with -O2 is miscompiling the derived Show and Eq instances for SeqT in logict-sequence.

main :: IO () main = print (pure 0 :: Seq Int) -- -O0, -O1: prints the Seq -- -O2: <<loop>>

On top of that, both hedgehog and hspec fail to catch `<<loop>>`.

main :: IO () main = defaultMain [ check $ property $ do let x = x :: Int -- obvious loop compiles to a thrown exception <<loop>> if x == x then pure () else pure () ] -- "thread blocked indefinitely in an STM transaction" -- meaning some thread is not getting cleaned up properly

main :: IO () main = hspec $ do describe "work" $ do it "works" $ do let x = x :: Int -- obvious loop compiles to a thrown exception <<loop>> if x == x then pure () else pure () :: IO () --- hangs

Cheers, Li-yao

On 2022-11-30 10:33 PM, David Feuer wrote:

The logict-sequence test suite consistently freezes up on the last test when compiled with GHC 9.4.3. logict-sequence and its test suite don't do anything special with that GHC/base version as far as I can see, or anything particularly strange in general, so I'm pretty confident the problem lies elsewhere. My best guesses are hedgehog, tasty, and tasty-hedgehog. Has anyone else run into weird issues with any/all of these on GHC 9.4.3?

David _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to:http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

David Feuer

11:13 p.m.

Thank you so much for figuring this out. I've filed a bug at https://gitlab.haskell.org/ghc/ghc/-/issues/22549 In the mean time, I'll adjust the upper bounds on Hackage and release a version of logict-sequence that hopefully won't have this issue. On Thu, Dec 1, 2022 at 7:14 PM Li-yao Xia wrote:

...

GHC 9.4.3 with -O2 is miscompiling the derived Show and Eq instances for SeqT in logict-sequence.

main :: IO () main = print (pure 0 :: Seq Int) -- -O0, -O1: prints the Seq -- -O2: <<loop>>

On top of that, both hedgehog and hspec fail to catch `<<loop>>`.

main :: IO () main = defaultMain [ check $ property $ do let x = x :: Int -- obvious loop compiles to a thrown exception <<loop>> if x == x then pure () else pure () ] -- "thread blocked indefinitely in an STM transaction" -- meaning some thread is not getting cleaned up properly

main :: IO () main = hspec $ do describe "work" $ do it "works" $ do let x = x :: Int -- obvious loop compiles to a thrown exception <<loop>> if x == x then pure () else pure () :: IO () --- hangs

Cheers, Li-yao

On 2022-11-30 10:33 PM, David Feuer wrote:

The logict-sequence test suite consistently freezes up on the last test when compiled with GHC 9.4.3. logict-sequence and its test suite don't do anything special with that GHC/base version as far as I can see, or anything particularly strange in general, so I'm pretty confident the problem lies elsewhere. My best guesses are hedgehog, tasty, and tasty-hedgehog. Has anyone else run into weird issues with any/all of these on GHC 9.4.3?

David _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

900

Age (days ago)

902

Last active (days ago)

List overview

Download

7 comments

3 participants

participants (3)

David Feuer
Doug Burke
Li-yao Xia

Debugging a test suite freeze

tags

participants (3)