ghci: Difference in garbage collection etc. between REPL and function

Niklas Hambüchen

8 May 2013 8 May '13

9:54 p.m.

I have an annoying bug in a C binding that can be triggered like this: handle <- open ... prep <- makePreparedStatement handle "INSERT ..." performGC runStatement prep close handle If I run these steps one by one in ghci, garbage ends up in my handle as expected. However, if I "let main = do ..." this whole block in order to pack it in a test case, it does not happen, neither in ghci nor ghc. What might be the special magic in ghci's REPL that allows me to trigger my bug so easily there? Thanks Niklas

Show replies by date

Jason Dagit

8 May 8 May

11:12 p.m.

New subject: ghci: Difference in garbage collection etc. between REPL and function

On Wed, May 8, 2013 at 6:54 PM, Niklas Hambüchen wrote:

...

I have an annoying bug in a C binding that can be triggered like this:

handle <- open ...

prep <- makePreparedStatement handle "INSERT ..."

performGC

runStatement prep

close handle

If I run these steps one by one in ghci, garbage ends up in my handle as expected.

However, if I "let main = do ..." this whole block in order to pack it in a test case, it does not happen, neither in ghci nor ghc.

What might be the special magic in ghci's REPL that allows me to trigger my bug so easily there?

One thing to investigate: Thread Local Storage By default ghci puts each action you run into a thread and executes it. If the underlying C code stores something (or accesses it) from thread local storage, then you will run into issues like this. Try starting ghci with -fno-ghci-sandbox. If the bad behavior goes away there, try running each step of your test case inside a forkIO from ghc with the threaded RTS. If the problem disappears in ghci but shows up with forkIO, then it's a pretty good indicator that it's related to the way the C code uses thread local storage. I hope that helps, Jason

Jason Dagit

11:21 p.m.

New subject: ghci: Difference in garbage collection etc. between REPL and function

On Wed, May 8, 2013 at 8:12 PM, Jason Dagit wrote:

...

On Wed, May 8, 2013 at 6:54 PM, Niklas Hambüchen wrote:

...
I have an annoying bug in a C binding that can be triggered like this:

handle <- open ...

prep <- makePreparedStatement handle "INSERT ..."

performGC

runStatement prep

close handle

If I run these steps one by one in ghci, garbage ends up in my handle as expected.

However, if I "let main = do ..." this whole block in order to pack it in a test case, it does not happen, neither in ghci nor ghc.

What might be the special magic in ghci's REPL that allows me to trigger my bug so easily there?

One thing to investigate: Thread Local Storage

By default ghci puts each action you run into a thread and executes it. If the underlying C code stores something (or accesses it) from thread local storage, then you will run into issues like this.

My point was meant to be, it's something to try but I have no idea if it's going to be the issue here. The caveat I forgot to add is: I've never seen the thread local storage issue affect GC. In particular, ghc might be running a finalizer in the case of individual actions because it doesn't know you are about to use the handle? Now that I think about it more, I suspect that is more likely to be the case. Jason

Brandon Allbery

9 May 9 May

8:50 a.m.

New subject: ghci: Difference in garbage collection etc. between REPL and function

On Wed, May 8, 2013 at 9:54 PM, Niklas Hambüchen wrote:

...

If I run these steps one by one in ghci, garbage ends up in my handle as expected.

However, if I "let main = do ..." this whole block in order to pack it in a test case, it does not happen, neither in ghci nor ghc.

ghci is in many ways like an endless (or at least until ":l"/":r") do-block. In particular, the handle remains in scope after you run your commands at the prompt, so it is not garbage collected. If you enclose it into its own do block, this introduces local scope and the handle goes out of scope and is garbage collected at the end. -- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

Niklas Hambüchen

10:19 a.m.

New subject: ghci: Difference in garbage collection etc. between REPL and function

On 09/05/13 20:50, Brandon Allbery wrote:

...

ghci is in many ways like an endless (or at least until ":l"/":r") do-block. In particular, the handle remains in scope after you run your commands at the prompt, so it is not garbage collected. If you enclose it into its own do block, this introduces local scope and the handle goes out of scope and is garbage collected at the end.

I am not sure how the handle is relevant - I do not expect it to garbage collected before the close or rely on that, and my problem happens earlier already.

Brandon Allbery

10:26 a.m.

New subject: ghci: Difference in garbage collection etc. between REPL and function

On Thu, May 9, 2013 at 10:19 AM, Niklas Hambüchen wrote:

...

On 09/05/13 20:50, Brandon Allbery wrote:

...
ghci is in many ways like an endless (or at least until ":l"/":r") do-block. In particular, the handle remains in scope after you run your commands at the prompt, so it is not garbage collected. If you enclose it into its own do block, this introduces local scope and the handle goes out of scope and is garbage collected at the end.

I am not sure how the handle is relevant - I do not expect it to garbage collected before the close or rely on that, and my problem happens earlier already.

You said 'garbage in the handle'... but in any case it applies to all the bindings, not just to that one. Generalize. -- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

4395

Age (days ago)

4395

Last active (days ago)

List overview

Download

5 comments

3 participants

participants (3)

Brandon Allbery
Jason Dagit
Niklas Hambüchen