Re: Remote GHCi

Simon, I'd like to hear how we can support what IHaskell does with remote GHCi. One core functionality that we use dynCompileExpr for (not quite dynCompileExpr, but similar) is getting the standard output of code that is being run. Any time code is run, we 1. Create a unix pipe. 2. Set stdout to point to that pipe using dupTo. 3. Use hscStmt with unsafeCoerce to get the other end of the pipe in the compiled context. 4. Run the statement in the interpreted context in a separate thread; meanwhile, read from the pipe to get the stdout of the code running in the interpreted context. 5. When it is done running, move stdout back to point to the read stdout and close the unix pipe file handle. 6. Send the stdout (both intermediate values and the final value) to the frontend to display to the user. The key here is that we can access directly the file handle created by the interpreted code. If the interpreted code is remote, we clearly cannot read from a pipe it creates. In your remote GHCi, how could we solve this problem? In general, how would stdin and stdout work? Would there be a clean way to feed the remote process its stdin and receive its stdout and stderr? That would effectively mean stdin/stdout/stderr are configurable which would be a godsend for IHaskell. -- Andrew On Wed, Nov 18, 2015 at 1:45 AM, Simon Marlow wrote:

On 18/11/2015 01:41, Manuel M T Chakravarty wrote:

...

Hi Simon,

While this is an interesting proposal, Haskell for Mac strongly relies on running interpreted code in the same process. I’m using ’dynCompileExpr’ as well as ’hscStmtWithLocation’ and some other stuff.

Let me say first of all that I'm not going to remove anything, so there's no need to worry. But I'd like to explore exactly what you need, so that we can see whether there's a way to accommodate it with a separate-process implementation.

hscStmtWithLocation is part of the core GHCi functionality, it is definitely supported. It has a slightly different signature:

hscStmtWithLocation :: HscEnv -> String -- ^ The statement -> String -- ^ The source -> Int -- ^ Starting line -> IO ( Maybe ([Id] , RemoteHValue {- IO [HValue] -} , FixityEnv))

RemoteHValue is a reference to a value in the interpreter's context. These have to be evaluated via an explicit API, rather than just unsafeCoercing HValue as we do now. (this is not strictly speaking part of the GHC API, so a separate but interesting question is: why did you need to use this directly, and what should we add to the GHC API?)

I believe that many uses of dynCompileExpr can be changed so that the code using the resulting value is moved into the interpreter's context, and then there's no problem.

This is quite crucial for some of the interactive

...

functionality. Imagine a game where the game engine is in Swift linked into the main application and the game logic is in *interpreted* Haskell code. The engine calls into the Haskell code multiple times per frame of the animation and for all keyboard/mouse/etc input (using StablePtr and ForeignPtr to construct the scene graph across the Swift and Haskell heap).

So my question is, why wouldn't you run the whole game engine in the interpreter's context? That's what would happen if you were to load the program into GHCi and run it. Directly calling back and forth between the client of the GHC API and the program being interpreted is arguably a strange thing to do, and it's kind of accidental that we allow it.

I actually also might have a use for the architecture that you are

...

proposing. However, I really would like to keep the ability to, at least, optionally run interpreted code in the same process (without profiling etc). Do you think we could have both?

We can certainly have both, it's straightforward to implement, but I don't get to throw away some of the hacks we have to support same-process execution, which would be a shame. We just add more code rather than

Cheers,

...

Manuel

Simon Marlow :

...

Hi folks - I've been thinking about changing the way we run interpreted code so that it would be run in a separate process. It turns out this has quite a few benefits, and would let us kill some of the really awkward hacks we have in GHC to work around problems that arise because we're running interpreted code and the compiler on the same runtime.

I summarised the idea here: https://ghc.haskell.org/trac/ghc/wiki/RemoteGHCi

I'd be interested to hear if anyone has any thoughts around this, particularly if doing this would make your life difficult in some way. Are people relying on dynCompileExpr for anything?

Cheers, Simon _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs