Don Stewart writes:

lazycat.manatee:

...

Hi all,

I'm research to build a hot-swap Haskell program to developing itself in Runtime, like Emacs.

Essentially, Yi/Xmonad/dyre solution is "replace currently executing" technology:

re-compile new code with new binary entry

when re-compile success $ do save state before re-launch new entry replace current entry with new binary entry (executeFile) store state after re-launch new entry

There are some problems with re-compile solution:

1) You can't save *all* state with some FFI code, such as gtk2hs, you can't save state of GTK+ widget. You will lost some state after re-launch new entry.

2) Sometimes re-execute is un-acceptable, example, you running some command in temrinal before you re-compile, you need re-execute command to restore state after re-launch, in this situation re-execute command is un-acceptable.

I wonder have a better way that hot-swapping new code without re-compile/reboot.

Well, the other approach to reloadable modules, using either object code plugins, or bytecode plugins, giving you module-level granularity. Thanks for your reply.

I have read your papers : "Dynamic Application From the Group Up" and "Plugging Haskell In" In "Dynamic Application From the Group Up", you introduction how to use re-compile technology implement source-code level hot-swapping. In "Plugging Haskell In", you introduction to how to buld hot-swapping with object-code level. Yes, Dynamic linking can add new code to a running program, but how to replace existing binding with new ones? Looks you still need some reboot when you do *replace* and not just *add*. Infact, reboot is okay, only problem is *keep state*, some *static state* is easier to re-build, example, if you want restore editor buffer state, you just need save (filepath, cursorPosition), you can re-open file and restore cursor position after reboot process. Difficult is *Stream State*, such as: delete operation in file-manager command running in temrinal network communications in browser It's really difficult to restore those state, and re-execute is un-acceptable sometimes. You can found the screenshot of my project at http://www.flickr.com/photos/48809572@N02/ Currently, the closest library to implement dynamic linking is your plugins package (http://hackage.haskell.org/package/plugins-1.4.1), i really want to write some code to test it, unfortunately, it's broken with Cabal-1.8.0.4 that can't compile with ghc-6.12.x/ghc-6.12.3, can you fix it if you have time? It's so great package... I'm looking for some paper about "Haskell and hot-swapping". Any paper or suggestion are welcome! -- Andy

hi,

if been thinking about an haskell interpreter to, because of erlang's otp. its syntax is a mess, but its scalability is win.

since erlang runs in its vm ("interpreted") is there a need for a real haskell interpreter, or can there be a compiled haskell/otp with hotswapping, scaling and stuff?

now back on topic, i wrote "real" haskell interpreter because there is the hint[1] package, which wrappes the ghc api.

now i dont know what more the plugin package provides, but i thought hint is like is successor (since lambdabot used plugins and now uses mueval, which in turn uses hint ;). please correct me. Yes, "haskell interpreter" maybe is solution, because Emacs is use this solution: "Static C core ++ elisp interpreter", then all elisp hot-swapping is

Martin Hilbig writes: base on elisp interpreter. IMO, "haskell interpreter" is perfect solution for samll script job. But i'm afraid "haskell interpreter" is slow for *large code*, i don't know, i haven't try this way... Thanks for your suggestion, i will consider it. -- Andy

have fun martin

[1]: http://hackage.haskell.org/package/hint

On 16.07.2010 06:06, Andy Stewart wrote:

...

Don Stewart writes:

...

lazycat.manatee:

...

Hi all,

I'm research to build a hot-swap Haskell program to developing itself in Runtime, like Emacs.

Essentially, Yi/Xmonad/dyre solution is "replace currently executing" technology:

re-compile new code with new binary entry

when re-compile success $ do save state before re-launch new entry replace current entry with new binary entry (executeFile) store state after re-launch new entry

There are some problems with re-compile solution:

1) You can't save *all* state with some FFI code, such as gtk2hs, you can't save state of GTK+ widget. You will lost some state after re-launch new entry.

2) Sometimes re-execute is un-acceptable, example, you running some command in temrinal before you re-compile, you need re-execute command to restore state after re-launch, in this situation re-execute command is un-acceptable.

I wonder have a better way that hot-swapping new code without re-compile/reboot.

Well, the other approach to reloadable modules, using either object code plugins, or bytecode plugins, giving you module-level granularity. Thanks for your reply.

I have read your papers : "Dynamic Application From the Group Up" and "Plugging Haskell In"

In "Dynamic Application From the Group Up", you introduction how to use re-compile technology implement source-code level hot-swapping.

In "Plugging Haskell In", you introduction to how to buld hot-swapping with object-code level.

Yes, Dynamic linking can add new code to a running program, but how to replace existing binding with new ones? Looks you still need some reboot when you do *replace* and not just *add*.

Infact, reboot is okay, only problem is *keep state*, some *static state* is easier to re-build, example, if you want restore editor buffer state, you just need save (filepath, cursorPosition), you can re-open file and restore cursor position after reboot process.

Difficult is *Stream State*, such as: delete operation in file-manager command running in temrinal network communications in browser It's really difficult to restore those state, and re-execute is un-acceptable sometimes.

You can found the screenshot of my project at http://www.flickr.com/photos/48809572@N02/

Currently, the closest library to implement dynamic linking is your plugins package (http://hackage.haskell.org/package/plugins-1.4.1), i really want to write some code to test it, unfortunately, it's broken with Cabal-1.8.0.4 that can't compile with ghc-6.12.x/ghc-6.12.3, can you fix it if you have time? It's so great package...

I'm looking for some paper about "Haskell and hot-swapping". Any paper or suggestion are welcome!

-- Andy

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Brandon S Allbery KF8NH wrote:

On 7/16/10 05:21 , Andy Stewart wrote:

...

IMO, "haskell interpreter" is perfect solution for samll script job. But i'm afraid "haskell interpreter" is slow for *large code*, i don't know, i haven't try this way...

Hugs?

Or you can try implementing (or finding) a SASL interpreter[1]. SASL is just the untyped[2] lambda calculus with named functions. ADTs are implemented by function application, greatly simplifying the compiler/interpreter. On their benchmarks, the version presented in the paper is competitive with GHCi 6.4 and outperforms Hugs Jan2005. Interpreting is always slower than compiled code. But how much that matters is a separate issue. Folks seem to like their Perl, Python, Ruby,... even for large projects. [1] Jan Jansen (2005) /Data Types and Pattern Matching by Function Application/, Proc. 17th IFL. [2] The version they present in the paper uses the untyped calculus since their pattern matching doesn't fit in Hindley--Milner. The problem is the need for a fixpoint type operator, and polymorphism under the fixpoint; thus, it fits perfectly fine in System F with iso-recursive types. So just turn on -XRankNTypes and use them directly: newtype List a = List { caseList :: forall r. r -> (a -> List a -> r) -> r } nil = List (\n c -> n) cons x xs = List (\n c -> c x xs) length xs = caseList xs 0 (\x xs' -> 1 + length xs') -- Live well, ~wren

Thomas Schilling writes:

What would be the semantics of hot-swapping? For, example, somewhere in memory you have a thunk of expression e. Now the user wants to upgrade e to e'. Would you require all thunks to be modified? A similar problem occurs with stack frames.

You'd also have to make sure that e and e' have the same (or compatible types). e' most likely has different free variables than e, how can you translate thunk one to the other?

Now assuming you don't try to do this, how would you handle the case when something goes wrong? Good question.

Infact, currently, i just use same technology with Yi/XMonad. Re-compile modules, and save some initial state before re-launch, and do re-execute some command to re-store those state after re-launch. Re-compile will drop old code, and restore state in new code. But re-compile solution not neat way since some state will lost. About your "upgrade expression" question, i haven no answer, that's why i post this topic to looking for a better solution. I know Emacs use some "indirection symbol table" for hot-swapping, but it's not safe since it not do type-check. Maybe we can wrap 'Dynamic' check type for safe re-load in runtime, if type match then upgrade, otherwise stop upgrade and keep old state. I just test re-compile technology, not try dynamic linking/load technology that Dons introduce in "Plugging Haskell In.pdf" since 'plugins' package is broken at the moment. Anyway, i believe this have a neat way to make static typing language can extension dynamic. Now, i'm reading Dons' new paper "Dynamic Extension of Typed Functional Languages" (http://www.cse.unsw.edu.au/~dons/papers/dons-phd-thesis.pdf) I hope i can find answer after read this paper... Any discuss are welcome! -- Andy

...

On 16 July 2010 04:05, Andy Stewart wrote:

...

Hi all,

I'm research to build a hot-swap Haskell program to developing itself in Runtime, like Emacs.

Essentially, Yi/Xmonad/dyre solution is "replace currently executing" technology:

  re-compile new code with new binary entry

  when re-compile success      $ do          save state before re-launch new entry          replace current entry with new binary entry (executeFile)          store state after re-launch new entry

There are some problems with re-compile solution:

1) You can't save *all* state with some FFI code, such as gtk2hs, you can't save state of GTK+ widget. You will lost some state after re-launch new entry.

2) Sometimes re-execute is un-acceptable, example, you running some command in temrinal before you re-compile, you need re-execute command to restore state after re-launch, in this situation re-execute command is un-acceptable.

I wonder have a better way that hot-swapping new code without re-compile/reboot.

Thanks,

 -- Andy

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On Fri, Jul 16, 2010 at 11:21 AM, Don Stewart wrote:

Generally, in Erlang or Haskell, the semantics we use is to keep all the old code in memory, for  the case of closures and thunks that point back into that code.

You can imagine a fine-grained semantics where as each top level function is no longer referenced, the *code* for that is swapped. I believe there have been versions of Erlang bytecode hotswapping that supported that.

In GHC Haskell, we have component-level hotswapping -- only whole-modules may be swapped out at the moment. As we can't unload code based on the GC dropping references, we just keep it all in memory, migrating to new code as *new* (dynamic) bindings are introduced.

The type migration issue is yet another problem. There are several approaches to migrating state types, including serialization (as xmonad et al do), or using open data types.

Is hs-plugins working? I thought it was bit-rotted and needed reworking to function with modern ghc? I'm using the xmonad style static config, but I don't think it's very practical for a reason Andy didn't mention: linking is slow when a program gets big, so every little change takes a long time. xmonad gets away with it because it's tiny. I use hint to interpret code dynamically, but that makes the program even bigger and linking even longer, so it seems like to have my cake and eat it too I would need to be able to recompile a single module, and then dynamically link it into the running application. In my case, I don't think I need to worry about old closures or code migration because I would just be loading new functions to splice into an event loop that can only return data types defined in the main app. So it would be acceptable to swap a module by removing all the functions loaded from it, unloading and reloading the module, and splicing in the new list of functions it exports. I'm not sure how I could prove to a dynamic loading system that I'm holding no references to its code anymore and it can just unload it, but I suppose it would be the same as a C plugin system: if you return pointers to plugin code and stash them in the host, you will be in trouble when the module gets unloaded. I suppose that just means you have to rnf all the data coming out of the plugin before you unload it, right? Is this the sort of thing hs-plugins can do? The other thing I thought of was to use xmonad-style static config, but speed up the link by linking the larger parts dynamically. Isn't ghc supposed to support dynamic linking now? Or would it be just as slow because the time subtracted from the link phase would simply be added to the load phase?

Bartek Ćwikłowski writes:

Hello Andy,

2010/7/16 Andy Stewart :

...

There are some problems with re-compile solution:

1) You can't save *all* state with some FFI code, such as gtk2hs, you can't save state of GTK+ widget. You will lost some state after re-launch new entry.

For my 2008 GSOC application I wrote a demo app that used hot code reloading and it maintained gtk state just fine. The code (and video) is available at http://paczesiowa.dw.pl/ , it worked in ghc 6.8 and needed hs-plugins, so it won't work now, but the approach should work. Wow, thanks for your hint, i will wait dons fix hs-plugins then try this cool feature..

Thanks, -- Andy