When I tried this it never returned, there was no error. -deech On Thu, Aug 12, 2010 at 4:04 PM, Yitzchak Gale wrote:

Pierre-Etienne Meunier wrote:

...

Does it still work with : writeFile "output" $! process inp

You're right, that changes things. Then the program prints:

<<loop>>

That would have given it away, of course. :)

Regards, Yitz _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Sebastian Fischer wrote:

   process _ = error "output: hClose: illegal operation (handle is finalized)"

Haha, very good! But I'm thinking of a case where GHC really does print out the error message. -Yitz

Anyone care to explain why? I also tested a slightly changed program pasted below, and am very confused.

main = do -- This call doesn't terminate, why? print $ nonTermination "a" -- Comment the above line to test the rest of the code -- RTS detects the loop and bails out print $ process "abc" -- Comment the above line to test the rest of the code inp <- readFile "input" -- I'm guessing (process inp) throws an exception, -- and the handler closes all files, making a second close fail? writeFile "output" $ process inp

process :: String -> String process xs = blackhole where blackhole = tail blackhole

nonTermination :: String -> String nonTermination _ = blackhole where blackhole = blackhole

On Thu, Aug 12, 2010 at 3:38 PM, Simon Marlow wrote:

On 12/08/10 15:09, Yitzchak Gale wrote:

...

The file "error_puzzle.hs" begins like this:

main = do   inp<- readFile "input"   writeFile "output" $ process inp

process :: String ->  String

When compiled with GHC 6.12.3 and run, it gives the following result:

$ ./error_puzzle error_puzzle: output: hClose: illegal operation (handle is finalized)

process xs = blackhole where blackhole = tail blackhole

Cheers,        Simon _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Threaded or not doesn't seem to make a difference. I just noticed with optimization turned on, RTS does detect the loop. On Thu, Aug 12, 2010 at 4:28 PM, Jason Dagit wrote:

On Thu, Aug 12, 2010 at 1:25 PM, Wei Hu wrote:

...

Anyone care to explain why? I also tested a slightly changed program pasted below, and am very confused.

...

main = do   -- This call doesn't terminate, why?   print $ nonTermination "a"   -- Comment the above line to test the rest of the code   -- RTS detects the loop and bails out   print $ process "abc"   -- Comment the above line to test the rest of the code   inp <- readFile "input"   -- I'm guessing (process inp) throws an exception,   -- and the handler closes all files, making a second close fail?   writeFile "output" $ process inp

...

process :: String ->  String process xs = blackhole where blackhole = tail blackhole

...

nonTermination :: String -> String nonTermination _ = blackhole where blackhole = blackhole

If you compile the program to use the non-threaded RTS does the behavior change? Jason

Wei Hu wrote:

...

nonTermination _ = blackhole where blackhole = blackhole

My original example was actually:

process :: String -> String process = let x = x in x

process = process works just as well. What about the other part of the solution:

What is the cause of the error?

Of course, the cause is the black hole. But why is it not reported? Sebastian -- Underestimating the novelty of the future is a time-honored tradition. (D.G.)

Hi, the reading is not needed to make it happen. main = writeFile "output" blackhole where blackhole = blackhole In fact, writing is not needed either. main = bracket (openFile "output" WriteMode) hClose (\hdl -> blackhole `seq` return ()) blackhole = blackhole Note that writeFile is indeed like this, so this seems to be the same problem. Sebastian Fischer wrote:

Of course, the cause is the black hole. But why is it not reported?

I guess the following happens: When the blackhole is detected in the third argument of bracket, the second argument is executed. But for some reason, the handle is already finalized at this point, so hClose raises an exception itself. But bracket reraises the exception from its third argument only if its second argument does not raise an exception itself. So in this case, the "handle is finalized" exception seems to eat the "blackhole" exception. So the question remains: Why is the handle finalized? (And what is "finalizing an handle"?) Tillmann

On 13/08/2010 09:53, Simon Marlow wrote:

On 12/08/2010 21:59, Yitzchak Gale wrote:

...

Wei Hu wrote:

...

nonTermination _ = blackhole where blackhole = blackhole

My original example was actually:

process :: String -> String process = let x = x in x

Ah yes, that works too. But other similar versions don't, like this one:

process :: String -> String process _ = let x = x in x

Hence why I added the "tail" in my version.

So what happens is this:

- the recursive definition causes the main thread to block on itself (known as a "black hole")

- the program is deadlocked (no threads to run), so the runtime invokes the GC to see if any threads are unreachable

- the GC finds that (a) the main thread is unreachable and blocked on a blackhole, so it gets a NonTermination exception (b) the Handle is unreachable, so its finalizer is started

- the finalizer runs first, and closes the Handle

- the main thread runs next, and the exception handler for writeFile tries to close the Handle, which has already been finalized

Really hClose shouldn't complain about a finalized handle, I'll see if I can fix that.

One other thing: this has nothing to do with lazy IO, the readFile is a red herring, you can get the same result without it. Cheers, Simon

On 13/08/10 17:00, Tillmann Rendel wrote:

Simon Marlow wrote:

...

Really hClose shouldn't complain about a finalized handle, I'll see if I can fix that.

That sounds like a work-around to me, not a fix, because it would not fix more complicated exception handlers.

I don't think there's a problem with more complicated exception handlers. The fix is to make the finalized Handle look like a closed Handle; it's a fix to the finalizer, not the exception handler, so it's not a workaround for this one particular symptom. Cheers, Simon

On Fri, Aug 13, 2010 at 4:53 AM, Simon Marlow wrote:

Ah yes, that works too.  But other similar versions don't, like this one:

process :: String ->  String process _ = let x = x in x

Hence why I added the "tail" in my version.

Without "-O", this version doesn't work, but with "-O" it does. Why?

So what happens is this:

 - the recursive definition causes the main thread to block on itself   (known as a "black hole")

 - the program is deadlocked (no threads to run), so the runtime   invokes the GC to see if any threads are unreachable

 - the GC finds that   (a) the main thread is unreachable and blocked on a blackhole, so it       gets a NonTermination exception   (b) the Handle is unreachable, so its finalizer is started

 - the finalizer runs first, and closes the Handle

 - the main thread runs next, and the exception handler for writeFile   tries to close the Handle, which has already been finalized

Really hClose shouldn't complain about a finalized handle, I'll see if I can fix that.

Cheers,        Simon

On 14/08/10 02:30, Lennart Augustsson wrote:

So it's a bug in the garbage collector. It's closing a handle that clearly is still reachable, otherwise this would not have happened.

The handle is in fact not reachable from the roots, because the thread that points to it is also not reachable. More about it in this ticket: http://hackage.haskell.org/trac/ghc/ticket/2161 Some rationale from that ticket: "You may argue that a reference from a thread, even an unreachable one, should keep a ForeignPtr alive. But it's equally valid to argue the reverse: a reference from a finalizer should keep a thread alive - you don't want a thread to be considered deadlocked if it can be unblocked by a finalizer. We can't have both, because then a cycle between a finalizer and a thread would never be collected even when both were unreachable." So while we can flip this decision, it would have other undesirable consequences. I suppose I don't really have a good handle on how often you want one behaviour vs. the other - feedback welcome. Either way the GC has to do two extra phases after tracing from the roots: trace from the finalizers and then the unreachable threads, or vice versa. Cheers, Simon

On Fri, Aug 13, 2010 at 10:53 AM, Simon Marlow wrote:

...

On 12/08/2010 21:59, Yitzchak Gale wrote:

...

Wei Hu wrote:

...

nonTermination _ = blackhole where blackhole = blackhole

My original example was actually:

process :: String -> String process = let x = x in x

Ah yes, that works too. But other similar versions don't, like this one:

process :: String -> String process _ = let x = x in x

Hence why I added the "tail" in my version.

So what happens is this:

- the recursive definition causes the main thread to block on itself (known as a "black hole")

- the program is deadlocked (no threads to run), so the runtime invokes the GC to see if any threads are unreachable

- the GC finds that (a) the main thread is unreachable and blocked on a blackhole, so it gets a NonTermination exception (b) the Handle is unreachable, so its finalizer is started

- the finalizer runs first, and closes the Handle

- the main thread runs next, and the exception handler for writeFile tries to close the Handle, which has already been finalized

Really hClose shouldn't complain about a finalized handle, I'll see if I can fix that.

Cheers, Simon _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On 14/08/2010 22:29, Yitzchak Gale wrote:

Lennart Augustsson wrote:

...

...

So it's a bug in the garbage collector. It's closing a handle that clearly is still reachable, otherwise this would not have happened.

Simon Marlow wrote:

...

The handle is in fact not reachable from the roots, because the thread that points to it is also not reachable.

I think I agree with Lennart here. A thread with an active exception handler is not permanently unreachable, only temporarily unreachable, unless we know that it is impossible for any exception to be thrown at it that it is capable of catching. In this case, the handler can catch a NonTermination, so it is not unreachable. A keyboard interrupt is another possibility.

You could also argue that a thread should not be considered deadlocked if a finalizer can wake it up - the problem is that these two requirements are contradictory. It's hard for me to tell which is more important, though certainly the evidence from this thread has swung things towards flipping the current behaviour. I wonder whether anyone is relying on the way things currently work, e.g. if you had a thread that waits for finalizers to run, then it would receive a BlockedIndefinitely exception under the changed semantics. Cheers, Simon

If we know that things are really stuck it could still be helpful for us to throw the NonTermination, but that doesn't mean that the handle should be considered unreachable at the time.

It is very important to do the best we can to allow people's exception handlers to run successfully, especially in a bracket. That is a much higher priority than protecting people from hanging their app when they write poor exception handlers. By finalizing the handle, you are significantly lowering the chance of exception handlers being able to do a proper job.

I do agree, though, that hClose should be less finicky about finalized handles, which would solve the problem in this particular case.

Thanks, Yitz