Using the ghc-api to run more than one instance of ghc simultaneously
Hi I'm trying to use the GHC API to have several instances of GHC's interpreter loaded simultaneously; each with its own loaded modules, etc. However, this doesn't seem to work well when two instances have loaded modules with the same name. I'm including the code of a small(ish) example of this at the end of the message. The example launches two threads (with forkIO) and fires GHC in interpreted mode on each thread (with GHC.runGhc); then it sequentially loads file TestMain1.hs in the first and TestMain2.hs in the second one and finally tries to evaluate expression test1 defined in the first one followed by test2 defined in the second one. The output is: #./Main 1: Load succeded 2: Load succeded 3: (1,2,3) 4: Main: During interactive linking, GHCi couldn't find the following symbol: Main_test1_closure This may be due to you not asking GHCi to load extra object files, archives or DLLs needed by your current session. Restart GHCi, specifying the missing library using the -L/path/to/object/dir and -lmissinglibname flags, or simply by naming the relevant files on the GHCi command line. Alternatively, this link failure might indicate a bug in GHCi. If you suspect the latter, please send a bug report to: glasgow-haskell-bugs@haskell.org Main: thread blocked indefinitely # The "thread blocked indefinitely" message is not important (comes from simplifying the original example). I tried this both in ghc 6.10.1 and ghc 6.11.20090607 with the same results. Is this a known limitation? Or should I be doing it some other way? Thanks, Daniel {-# LANGUAGE MagicHash #-} module Main where import Prelude hiding ( init ) import Control.Monad ( join, forever ) import Control.Concurrent ( forkIO ) import Control.Concurrent.Chan import GHC ( Ghc ) import qualified GHC import qualified MonadUtils as GHC import qualified GHC.Paths import qualified GHC.Exts main :: IO () main = do let test1 = "TestMain1.hs" let test2 = "TestMain2.hs" writeFile test1 "module Main where test1 = (1,2,3)" writeFile test2 "module Main where test1 = (3,2,1)" -- ghc_1 <- newGhcServer ghc_2 <- newGhcServer line "1" $ runInServer ghc_1 $ load (test1, "Main") line "2" $ runInServer ghc_2 $ load (test2, "Main") line "3" $ runInServer ghc_1 $ eval "test1" line "4" $ runInServer ghc_2 $ eval "test1" where line n a = putStr (n ++ ": ") >> a type ModuleName = String type GhcServerHandle = Chan (Ghc ()) newGhcServer :: IO GhcServerHandle newGhcServer = do pChan <- newChan let be_a_server = forever $ join (GHC.liftIO $ readChan pChan) forkIO $ ghc be_a_server return pChan where ghc action = GHC.runGhc (Just GHC.Paths.libdir) (init >> action) init = do df <- GHC.getSessionDynFlags GHC.setSessionDynFlags df{GHC.ghcMode = GHC.CompManager, GHC.hscTarget = GHC.HscInterpreted, GHC.ghcLink = GHC.LinkInMemory, GHC.verbosity = 0} runInServer :: GhcServerHandle -> Ghc a -> IO a runInServer h action = do me <- newChan writeChan h $ action >>= (GHC.liftIO . writeChan me) readChan me load :: (FilePath,ModuleName) -> Ghc () load (f,mn) = do target <- GHC.guessTarget f Nothing GHC.setTargets [target] res <- GHC.load GHC.LoadAllTargets GHC.liftIO $ putStrLn ("Load " ++ showSuccessFlag res) -- m <- GHC.findModule (GHC.mkModuleName mn) Nothing GHC.setContext [m] [] where showSuccessFlag GHC.Succeeded = "succeded" showSuccessFlag GHC.Failed = "failed" eval :: String -> Ghc () eval e = do show_e <- GHC.compileExpr $ "(show ("++ e ++")) :: String" GHC.liftIO $ putStrLn (GHC.Exts.unsafeCoerce# show_e)
On Sun, Jul 12, 2009 at 05:49:15PM +0200, Daniel Gorín wrote:
Hi
I'm trying to use the GHC API to have several instances of GHC's interpreter loaded simultaneously; [snip]
Is this possible now? 6-12 month ago it was not possible to run multiple ghc threads within the same instance due to some global state IORefs or such. But I'd appreciate hearing that this changed recently. My knowledge can be outdated. Sincerly Marc Weber
On 12/07/2009 16:49, Daniel Gorín wrote:
Hi
I'm trying to use the GHC API to have several instances of GHC's interpreter loaded simultaneously; each with its own loaded modules, etc. However, this doesn't seem to work well when two instances have loaded modules with the same name. I'm including the code of a small(ish) example of this at the end of the message.
The example launches two threads (with forkIO) and fires GHC in interpreted mode on each thread (with GHC.runGhc); then it sequentially loads file TestMain1.hs in the first and TestMain2.hs in the second one and finally tries to evaluate expression test1 defined in the first one followed by test2 defined in the second one. The output is:
#./Main 1: Load succeded 2: Load succeded 3: (1,2,3) 4: Main: During interactive linking, GHCi couldn't find the following symbol: Main_test1_closure This may be due to you not asking GHCi to load extra object files, archives or DLLs needed by your current session. Restart GHCi, specifying the missing library using the -L/path/to/object/dir and -lmissinglibname flags, or simply by naming the relevant files on the GHCi command line. Alternatively, this link failure might indicate a bug in GHCi. If you suspect the latter, please send a bug report to: glasgow-haskell-bugs@haskell.org
Main: thread blocked indefinitely #
The "thread blocked indefinitely" message is not important (comes from simplifying the original example). I tried this both in ghc 6.10.1 and ghc 6.11.20090607 with the same results.
Is this a known limitation? Or should I be doing it some other way?
Yes, it is a known limitation. It ought to be documented somewhere. There are two problems: 1. GHC is not thread-safe. There are some items of global state (the NameCache and the PackageInterfaceTable) that should be protected. It's not a trivial matter to do this - we had a hacky solution for the parallel compilation experiment we did a while back, but it wasn't finished. You could workaround this by using a mutex and only invoking GHC API operations in one thread at a time. 2. There is only one RTS linker with a single symbol table. This is the problem you ran into. There's no workaround that I'm aware of. Cheers, Simon
Yes, it is a known limitation. It ought to be documented somewhere.
There are two problems:
1. GHC is not thread-safe. There are some items of global state (the NameCache and the PackageInterfaceTable) that should be protected. It's not a trivial matter to do this - we had a hacky solution for the parallel compilation experiment we did a while back, but it wasn't finished.
You could workaround this by using a mutex and only invoking GHC API operations in one thread at a time.
2. There is only one RTS linker with a single symbol table. This is the problem you ran into. There's no workaround that I'm aware of.
Are there already bug tracker items for these two problems? I've tried finding them but didn't succeed. This would be a fast way to document this issue even if its unlikely to be fixed soon. Marc Weber
On Jul 13, 2009, at 10:53 PM, Marc Weber wrote:
Yes, it is a known limitation. It ought to be documented somewhere.
There are two problems:
1. GHC is not thread-safe. [...]
2. There is only one RTS linker with a single symbol table. [...]
Are there already bug tracker items for these two problems? I've tried finding them but didn't succeed. This would be a fast way to document this issue even if its unlikely to be fixed soon.
Marc Weber
For the record, now there are: http://hackage.haskell.org/trac/ghc/ticket/3372 http://hackage.haskell.org/trac/ghc/ticket/3373 Daniel
participants (3)
-
Daniel Gorín -
Marc Weber -
Simon Marlow