claus.reinke:

I think you're providing workarounds for some serious short- comings in Haskell! while a fully integrated design would be nicer (as in the old orthogonally persistent systems - google for "type-safe linguistic reflection"; or more recently in Clean), I look forward to having at least a pragmatically oriented subset of the features available. some questions:

is there any particular reason that your list of compatible platforms seems limited to unix-like systems? or would it work on other GHC platforms as well, such as windows (after all, ghci works there..)?

Only as these are the systems that I've tested on, or that others have. I don't have access to a Windows machine. hs-plugins should only be limited to platforms with a working GHC dynamic loader, which is all the systems I've tested on, plus Windows. Any unixy dependencies should be considered bugs, and will be removed as I find them.

what is the relation of this to -package ghc? at the moment, Simon M. is very busy with all the other things people want, but most of the functionality you provide I would hope to find there when it comes (will hs-plugins be implemented on top of -package ghc, or will its functionality be integrated,

Currently hs-plugins is entirely separate to GHC. This could change as -package ghc becomes more widely used. It would certainly make maintainence across GHC releases easier. The lowest layer of hs-plugins, the dynamic loader, reimplements several utilities of GHC: the .hi file parser, the package.conf parser. And, of course, it contains a binding to the GHC dynamic linker. These interfaces could all be made much simpler by -package ghc. Clean interfaces to -package ghc would make, for example, the load() call just a call into GHC. Also, the makeWith function parses and merges Haskell source. Using GHC's parser for this removes the limitations of the Language.Haskell parser. The other aspect of hs-plugins is the runtime compilation stuff. I think this part of hs-plugins (i.e. Eval.Haskell, Eval.Meta, Printf) should be seen as prototype runtime metaprogramming system. Ideally it would use -package ghc for type checking and compilation. The points where hs-plugins currently invokes GHC externally as a type checker, or as a code generator, should be replaced with calls to the -package ghc interfaces to these components. I'm already playing around with this, but it is still in the alpha phase. So, ultimately, -package ghc may replace some low levels of hs-plugins. Some of the higher levels could turn into nice interfaces to -package ghc components, or into a library along the lines of Template Haskell.

or will there be separate implementations)? you mention it once on page 26, in a way that suggests there should be other references to it in this document, which I can't find.. how bad/good are the startup/load times?

Some timings for the various flavour of load() are in the Haskell Workshop paper. An unchecked plugin load is identical to loading a module in GHCi, and has a similar cost. The interesting 'meta' functions require an external GHC process to compile code, which has a non-trivial cost. For example, compiling a printf string to a new function takes around 0.5s on my laptop. Once that is done, it can be applied to arguments at the same speed as normal native code. I see -package ghc as the best way to reduce this cost, in a similar way as Template Haskell uses GHCi internally for its compile-time eval features.

I could use some more runtime reflection support in Haskell immediately - I'm currently starting ghci as an external process to load&run generated code, which allows me to make it look as if the system just continued running, and to process some error messages, but it doesn't really allow me to combine new and old code, which is what hs-plugins seems to offer;

Yes. This is exactly what it does. You should be able to do some interesting things with the framework as it stands.

however, I don't want to limit my code to unix systems, and I would prefer some convergence on one standardised API.

This would be good. As I see it, the runtime compilation aspects (which is what you are most interested in?) is really staged computation, so I'm currently working on an interface along the lines of Shields, Sheard and SPJ's "Dynamic Typing as Staged Type Inference" paper. That is, a runtime version of TH. A quick sketch of run(), defer() and splice() are in Eval.Meta. This is currently my idea of the final interface, but the entire system is a few months from being really comfortable. There is still more work to be done here, particularly in relation to dynamic typing. Data.Dynamic is too restrictive for a comfortable runtime metaprogramming system. I'm currently looking at a better dynamics, like that used in hs-plugins' pdynload(), and using -package ghc for type checking. Still very preliminary though. -- Don

Don, thanks for your answers - that looks very promising.

Only as these are the systems that I've tested on, or that others have. I don't have access to a Windows machine. hs-plugins should only be limited to platforms with a working GHC dynamic loader, which is all the systems I've tested on, plus Windows. Any unixy dependencies should be considered bugs, and will be removed as I find them.

ah well, you do depend on a unixy build environment;-) I've got cygwin myself, but I can't rely on my users having such things. More seriously, though: a cursory browse of the sources shows up things like "POpen" and "Posix", both no-nos, as far as portable ghc code is concerned (no longer supported in the default ghc for windows, AFAIK). it would be good if ghc-users would keep that in mind when writing portable Haskell code. if you really need pipes to subprocesses, there's an apparently portable implementation in wxhaskell (Graphics.UI.WXCore.Process, no graphics needed - wxhaskell has lots of non-graphics widgets), or the up-and-coming System.Process. don't know about the rest of the Posix modules.. i don't have much time for testing right now, and "configure;make" stalls early on with a rather strange-looking message (see below), so I'll probably have to leave that till next month. but, as i mentioned, i need something like this, and if the idea is to provide a higher-level API on top of -package ghc, that sounds like a good way to go.. provided that the unix-"bugs" don't stop the show. cheers, claus -----------(windows98,cygwin,ghc-6.2.1) cd src && make make[1]: Entering directory `/cygdrive/c/ghc/libraries/hs-plugins-0.9.6/src' make[2]: Entering directory `/cygdrive/c/ghc/libraries/hs-plugins-0.9.6/src/alt ata' =========== building altdata ============= ghc -package-name altdata -O -Wall -fno-warn-missing-signatures $EOPTS --make no-hs-main -pgmltrue AltData.hs Chasing modules from: AltData.hs Compiling AltData.Typeable ( ./AltData/Typeable.hs, ./AltData/Typeable.o ) Compiling AltData.Dynamic ( ./AltData/Dynamic.hs, ./AltData/Dynamic.o ) Compiling AltData ( AltData.hs, AltData.o ) Linking ... rm -f libHSaltdata.a ar cq libHSaltdata.a AltData.o AltData/*.o ranlib libHSaltdata.a rm -f HSaltdata.o cpp -DGHC_VERSION_6_3="0" < altdata.conf.in.cpp | \ sed -e 's/""//g' -e 's/\[ *,/[ /g' -e '/^#/d' > altdata.conf.in lace.in (cd /c/ghc/libraries/hs-plugins-0.9.6 ;\ if [ ! -f plugins.conf.inplace ]; then echo [] > plugins.conf.inplace; fi;\ env PREFIX=`pwd` ghc-pkg -g -f plugins.conf.inplace -u < \ src/altdata/altdata.conf.inplace.in) Reading package info from stdin... done. Expanding embedded variables... done. dependency `base' doesn't exist make[2]: *** [inplace-pkg-conf] Error 1 make[2]: Leaving directory `/cygdrive/c/ghc/libraries/hs-plugins-0.9.6/src/altd ta' make[1]: *** [altdata] Error 2 make[1]: Leaving directory `/cygdrive/c/ghc/libraries/hs-plugins-0.9.6/src' make: *** [build] Error 2

claus.reinke:

Don,

thanks for your answers - that looks very promising.

...

Only as these are the systems that I've tested on, or that others have. I don't have access to a Windows machine. hs-plugins should only be limited to platforms with a working GHC dynamic loader, which is all the systems I've tested on, plus Windows. Any unixy dependencies should be considered bugs, and will be removed as I find them.

ah well, you do depend on a unixy build environment;-) I've got cygwin myself, but I can't rely on my users having such things. More seriously, though:

a cursory browse of the sources shows up things like "POpen" and "Posix", both no-nos, as far as portable ghc code is concerned (no longer supported in the default ghc for windows, AFAIK). it would be good if ghc-users would keep that in mind when writing portable Haskell code.

Ah. Good point. Well, it is portable to everything except windows, right ;)

if you really need pipes to subprocesses, there's an apparently portable implementation in wxhaskell (Graphics.UI.WXCore.Process, no graphics needed - wxhaskell has lots of non-graphics widgets), or the up-and-coming System.Process. don't know about the rest of the Posix modules..

i don't have much time for testing right now, and "configure;make" stalls early on with a rather strange-looking message (see below), so I'll probably

Hmm. Strange. I don't have any ideas about this at the moment.

have to leave that till next month. but, as i mentioned, i need something like this, and if the idea is to provide a higher-level API on top of -package ghc, that sounds like a good way to go.. provided that the unix-"bugs" don't stop the show.

Certainly. I'll look into this. Thanks for pointing it out! -- Don

-----------(windows98,cygwin,ghc-6.2.1)

cd src && make make[1]: Entering directory `/cygdrive/c/ghc/libraries/hs-plugins-0.9.6/src' make[2]: Entering directory `/cygdrive/c/ghc/libraries/hs-plugins-0.9.6/src/alt ata' =========== building altdata ============= ghc -package-name altdata -O -Wall -fno-warn-missing-signatures $EOPTS --make no-hs-main -pgmltrue AltData.hs Chasing modules from: AltData.hs Compiling AltData.Typeable ( ./AltData/Typeable.hs, ./AltData/Typeable.o ) Compiling AltData.Dynamic ( ./AltData/Dynamic.hs, ./AltData/Dynamic.o ) Compiling AltData ( AltData.hs, AltData.o ) Linking ... rm -f libHSaltdata.a ar cq libHSaltdata.a AltData.o AltData/*.o ranlib libHSaltdata.a rm -f HSaltdata.o cpp -DGHC_VERSION_6_3="0" < altdata.conf.in.cpp | \ sed -e 's/""//g' -e 's/\[ *,/[ /g' -e '/^#/d' > altdata.conf.in lace.in (cd /c/ghc/libraries/hs-plugins-0.9.6 ;\ if [ ! -f plugins.conf.inplace ]; then echo [] > plugins.conf.inplace; fi;\ env PREFIX=`pwd` ghc-pkg -g -f plugins.conf.inplace -u < \ src/altdata/altdata.conf.inplace.in) Reading package info from stdin... done. Expanding embedded variables... done. dependency `base' doesn't exist

make[2]: *** [inplace-pkg-conf] Error 1 make[2]: Leaving directory `/cygdrive/c/ghc/libraries/hs-plugins-0.9.6/src/altd ta' make[1]: *** [altdata] Error 2 make[1]: Leaving directory `/cygdrive/c/ghc/libraries/hs-plugins-0.9.6/src' make: *** [build] Error 2