So technically, your example might need to involve using g (and forceful GC at a certain point during execution)
Good observation.
Maybe a stupid question, sorry: The RemoteTable generated using template-haskell in CH without XStaticPointers would keep CAFs alive. So the XStaticPointers extension does not entail using such a table?
That's correct. The extension is a substitute for the remote table. In addition, it has the compiler do what remote tables demanded from the user: * adding functions to the remote table before they are looked up, * collecting the table pieces from the various modules into a global table.
Another question: Would it be sufficient to desugar "static g" to g `seq` StaticPtr(StaticName "" "Main" "g") instead of introducing a stable ptr and all that?
This keeps g alive only while the expression is not evaluated to HNF. The solution I proposed is flawed as well, since it relies on the desugared static form being evaluated to HNF for the CAF to be referenced with a StablePtr. Anyway, after this much time we figured out how to implement the static pointer table.
Finally, there is a flag keepCAFs in the runtime which you can set to secure the CAFs for the entire run. The parallel runtimes for Eden and GUM (as well as my "packman" serialisation) do this.
Good to know about that. Thank you, Facundo
AFter all, g is a CAF, so it is anyway "stable" in some sense, as long as it is alive.
However, I conjecture that this only fixes the one-node test, not the actual use case (sending "static" stuff over the wire).
Finally, there is a flag keepCAFs in the runtime which you can set to secure the CAFs for the entire run. The parallel runtimes for Eden and GUM (as well as my "packman" serialisation) do this.
Facundo
On Tue, Nov 18, 2014 at 1:20 PM, Jost Berthold
Hi Facundo,
You are completely right, the CAF named "g" might be accessed at any time during the program execution. Parallel Haskell systems with distributed heap (and runtime-supported serialisation) need to keep all CAFS alive for this reason.
Some comments inline along your mail:
While working in the StaticPointers language extension [1], we found we have some unusual CAFs which can be accessed after some periods of time where there is no reference to them.
For instance, the following program when compiled contains no reference to `g`. `g` is actually looked up at runtime in symbol tables via the call to `deRefStaticPtr`. g :: String g = "hello"
main = deRefStaticPtr (static g) >>= putStrLn
The bad scenario is certainly one where CAF g (a static thunk) is evaluated during execution (i.e. turned into an indirection into the heap), and then garbage-collected, as it might not be referenced by any (runnable) thread. This GC does not revert the indirection into a thunk. Why should it, there are no references to it, right? ;-)
So technically, your example might need to involve using g (and forceful GC at a certain point during execution):
main = putStrLn g >> performGC >> deRefStaticPtr (static g) >>= putStrLn
Desugars to:
g :: String g = "hello"
main =
putStrLn g >> performGC >>
deRefStaticPtr (StaticPtr (StaticName "" "Main" "g")) >>= putStrLn
During performGC, there would be no reference to g from any thread's stack. I am of course assuming that g is indeed a thunk, and not statically evaluated to a string during compilation (I am unsure whether GHC would do that).
In principle, there is nothing stopping the garbage collector from reclaiming the closure of `g` before it is dynamically looked up.
Maybe a stupid question, sorry: The RemoteTable generated using template-haskell in CH without XStaticPointers would keep CAFs alive. So the XStaticPointers extension does not entail using such a table?
We are considering using StablePtrs to preserve `g`. So the code desugars instead to:
g :: String g = "hello"
main = deRefStaticPtr (let x = StaticPtr (StaticName "" "Main" "g") in unsafePerformIO $ newStablePtr g >> return x ) >>= putStrLn
Another question: Would it be sufficient to desugar "static g" to g `seq` StaticPtr(StaticName "" "Main" "g") instead of introducing a stable ptr and all that? AFter all, g is a CAF, so it is anyway "stable" in some sense, as long as it is alive.
However, I conjecture that this only fixes the one-node test, not the actual use case (sending "static" stuff over the wire).
Finally, there is a flag keepCAFs in the runtime which you can set to secure the CAFs for the entire run. The parallel runtimes for Eden and GUM (as well as my "packman" serialisation) do this.
Yes, obviously, this opens a memory leak. It would be nice to not "keep" but "revert" the CAFs (ghci does that) but on a "by-need" basis when a simple GC cannot reclaim enough space; this would plug the mem.leak. This requires a modification to the GHC runtime system, and it is unclear _which_ CAFs to prefer when starting to revert. But I think it would be a more generally useful feature. However, this discussion (runtime/GC features) leads us straight out of the design goals of "-XStaticPointers", I guess...
Best regards, Jost _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs