- I'd like to see a standard way to call the GC from C
http://www.mail-archive.com/ffi@haskell.org/msg00565.html
Note that Hugs and GHC have had this for ages except that we call the function 'performGC' and there's no way to control how many generations are collected.
I don't have a strong opinion on this one.
Maybe SimonM and Malcolm do.
I know GHC has an interface already and I suspect that NHC will too. GHC and Hugs both call it performGC but this doesn't match the naming convention used in the ffi and it is expensive to use with generational GC (since there is no way to indicate that a partial GC may be enough).
System.Mem.performGC does a major GC. When would a partial GC be enough? Isn't the property you really want "a full garbage collection has been performed and all finalizers have been run to completion"? Note this would be non-trivial to implement in GHC. We currently don't distinguish between finalizer threads and normal threads, preferring to leave it to the programmer to implement any synchronisation on finalizers using MVars.
- I see the question of Function prototypes as a portability problem waiting to happen. Either Hugs and GHC are right (you should use the user-supplied header file or NHC is right (you should ignore the header file). They can't both be right if we want portable code so the report should be clear about which one is right.
I think the spec should be clarified along these lines: Header files have no impact on the semantics of a foreign call, and whether an implementation uses the header file or not is implementation-defined. Some implementations may require a header file which supplies a correct prototype for the function in order to generate correct code. I'd be equally happy (perhaps happier) if the header file spec was removed altogether. In a sense, this would leave the Haskell part of a foreign binding even more portable, because it doesn't have to specify the names of header files which might change between platforms. There are already "external" parts of a binding such as the names of link libraries, I think I'm arguing that header files also fall into that category.
- Changes to hs_init
I am ok with that. Currently, there is a problem with the version that is in the spec and GHC in that GHC requires an extra argument to initialise modules. So, it all depends a bit on how far SimonM thinks its implementable.
Hugs doesn't have anything resembling hs_init at the moment so I can't say for sure. I think it can be done as a thin layer on top of the the existing spec though so I don't think it's that hard.
Perhaps on GHC you should be required to "register" the top module in your program first, maybe something like registerModule(__stginit_Main); that way you can register multiple modules (which isn't possible at the moment, you have to have another module which imports all the others). Cheers, Simon
System.Mem.performGC does a major GC. When would a partial GC be enough?
I've described the image-processing example a bunch of times. We have an external resource (e.g., memory used to store images) which is somewhat abundant and cheap but not completely free (e.g., eventually you start to swap). It is used up at a different rate than the Haskell heap so Haskell GCs don't occur at the right times to keep the cost low and we want to trigger GCs ourselves. (In the image processing example, images were megabytes and an expression like (x + (y * mask)) would generate 2 intermediate images (several megabytes) while doing just 2 reductions in Haskell.) How often and how hard should we GC? We can't do a full GC too often, or we'll spend a lot of time GCing, destroy our cache and cause premature promotion of Haskell objects into the old generation which will make the GC behave poorly. So if all we can do is a full GC, we'll GC rarely and use a lot of the external resource. Suppose we could collect just the allocation arena. That would be much less expensive (time taken, effect on caches, confusion of object ages) but not always effective. It would start out cheap and effective but more and more objects would slip into older generations and have to wait for a full GC. To achieve any desired tradeoff between GC cost and excess resource usage, we want a number of levels of GC: gc1, gc2, gc3, gc4, ... Each one more effective than the last and each one more expensive than the last. We'll use gc1 most often, gc2 less often, gc3 occasionally, gc4 rarely, ... It seemed important to do this 4-5 years ago when we were doing image processing with Hugs. Now, with memories much, much larger (cf. Hal Daume's 'why does my program segfault with a 4Gb heap' report) but not appreciably faster, it is even more important.
Isn't the property you really want "a full garbage collection has been performed and all finalizers have been run to completion"?
With Hugs, we used feedback between the rate of deallocation/ allocation and the rate of GCing. The feedback was easy because the GC didn't complete until all objects were freed. With GHC, it would be harder because of the delay between GC and release. What I'd do is wander round to the university library, find the section on control theory, find a good book on feedback loops and implement something from there.
I think the spec should be clarified along these lines:
Header files have no impact on the semantics of a foreign call, and whether an implementation uses the header file or not is implementation-defined. Some implementations may require a header file which supplies a correct prototype for the function in order to generate correct code.
I still don't like the fact that compilers are free to ignore header files. Labelling it an error instead of a change in semantics doesn't affect the fact that portability is compromised.
I'd be equally happy (perhaps happier) if the header file spec was removed altogether. In a sense, this would leave the Haskell part of a foreign binding even more portable, because it doesn't have to specify the names of header files which might change between platforms. There are already "external" parts of a binding such as the names of link libraries, I think I'm arguing that header files also fall into that category.
Making header files completely external seems to encourage omission of header files and non-portability. And, annoying as it is to limit ourselves to one header file, it has the nice effect that there is a file out there that Hugs can use - I don't have to go figure out how the GHC Makefiles and configure system contrive to pass the right include files on the command line: it's right there in the file for Hugs to see. (And vice-versa, of course.)
Perhaps on GHC you should be required to "register" the top module in your program first, maybe something like
registerModule(__stginit_Main);
that way you can register multiple modules (which isn't possible at the moment, you have to have another module which imports all the others).
What does that do? Is it for threading, GC, profiling, ...? Do I have to unregister the module later?
On 12-Aug-2002, Simon Marlow <simonmar@microsoft.com> wrote:
I'd be equally happy (perhaps happier) if the header file spec was removed altogether. In a sense, this would leave the Haskell part of a foreign binding even more portable, because it doesn't have to specify the names of header files which might change between platforms.
This is a C interface we're talking about, right? In C, the name of the header file is part of the API. It doesn't change between different platforms unless the API changes. Specifying the header name is essential if Haskell implementations are to ever apply any type-checking to these foreign interfaces. If they don't, then in practice I think Haskell programs using the FFI are likely to be less portable, and certainly more error-prone, since they will contain type errors that may cause problems on one platform but not another.
There are already "external" parts of a binding such as the names of link libraries, I think I'm arguing that header files also fall into that category.
If so, I find this a pretty unconvincing argument. Header search paths (i.e. `-I' options), yes, but the part of the name which goes in between the quotes or angle brackets in a #include line, no. They are part of the API. -- Fergus Henderson <fjh@cs.mu.oz.au> | "I have always known that the pursuit The University of Melbourne | of excellence is a lethal habit" WWW: <http://www.cs.mu.oz.au/~fjh> | -- the last words of T. S. Garp.
"Simon Marlow" <simonmar@microsoft.com> wrote,
- I see the question of Function prototypes as a portability problem waiting to happen. Either Hugs and GHC are right (you should use the user-supplied header file or NHC is right (you should ignore the header file). They can't both be right if we want portable code so the report should be clear about which one is right.
I think the spec should be clarified along these lines:
Header files have no impact on the semantics of a foreign call, and whether an implementation uses the header file or not is implementation-defined. Some implementations may require a header file which supplies a correct prototype for the function in order to generate correct code.
In fact, this is what I think we did originally intend. We definitely said that some implementations (which just generate assmebler, for example) may ignore the header file, but we also recognised that some implementations may make use of them. As Fergus said, it is better if the header files are taken into account, but we didn't want to force that extra complexity onto implementations. I don't see any strong arguments for changing our original intention and there are arguments against changing it. So, let's leave things as we originally intended them. I'll add SimonM's above clarification to the spec. Manuel
"Simon Marlow" <simonmar@microsoft.com> wrote,
- Changes to hs_init
This proposal doesn't work with these types IMHO. The reason is that you won't know which arguments to pass to hs_set_hs_argv (int argc, char *argv[]); Typically, the command line will contain a mix of arguments, where some meant for the Haskell system (in GHC, +RTS stuff) and some for the application. The application has no way to distinguish the two. So, the idea with hs_init (int *argc, char **argv[]); was that the Haskell system grabs all it's arguments and *removes* them from argv (hence, the double indirection). This is sort of set up is essential when there are mulptiple libraries that require to get at some command line arguments. Moreover, although I understand John Meacham concern, I don't see how it is addressed by the proposed change to the interface. hs_set_hs_argv() still may only be called once. Wherever, this is done, we could just have called hs_init(). And do away with all the extra calls to the "new" parameterless hs_init(). Further to John Meacham's concern, the only solution that I see is to allow multiple calls to hs_init() and hs_exit() and place the burden of keeping track of multiple initialisations onto the Haskell system. No change to the function prototypes is needed to implement this, only a change of the specification of the semantics (namely, permitting multiple calls). So, my proposal is to * permit multiple calls to hs_init() and hs_exit() (the number of calls must match of course), * command line arguments specified in the second and following invocations to hs_init() are ignored, and * both arguments to hs_init() may be NULL (to indicate the absence of arguments). The second point is necessary to keep the implementation on the Haskell side reasonably straight forward. I think, we also need to say something about the timing of hs_set_argv() and calling getProgName and getArgs. I'd say, if hs_set_argv() is used at all, it must be before the first call to getProgName and getArgs (otherwise, the system behaviour is undefined). Objections?
I am ok with that. Currently, there is a problem with the version that is in the spec and GHC in that GHC requires an extra argument to initialise modules. So, it all depends a bit on how far SimonM thinks its implementable.
Hugs doesn't have anything resembling hs_init at the moment so I can't say for sure. I think it can be done as a thin layer on top of the the existing spec though so I don't think it's that hard.
Perhaps on GHC you should be required to "register" the top module in your program first, maybe something like
registerModule(__stginit_Main);
that way you can register multiple modules (which isn't possible at the moment, you have to have another module which imports all the others).
So, registerModule() would be a GHC-specific extension to the FFI API? Manuel
So, my proposal is to: [...]
Another possibility is to say that we don't really understand the issues well enough to put in the standard yet and just comment out that section. I think only GHC implements anything like this (correct me if wrong, Malcolm) and they haven't used it in the way John Meacham is interested in. Dunno... if you think we understand it well enough, go ahead. A
Alastair Reid <alastair@reid-consulting-uk.ltd.uk> writes:
So, my proposal is to: [...]
I think only GHC implements anything like this (correct me if wrong, Malcolm) and they haven't used it in the way John Meacham is interested in.
At the moment, nhc98 provides a routine void haskellInit (int argc, char **argv) which collects the command-line args intended for (1) the nhc98 runtime system, and (2) the response to System.getArgs. It does not strip any arguments from the given set, on the basis that the calling C routine may be interested in the same arguments that nhc98 is interested in. Also, if the controlling C routine wants to alter the argument set before Haskell sees it, that is perfectly catered for by this interface. Regards, Malcolm P.S. The nhc98 internal names will change shortly to match those defined by the FFI spec.
Malcolm Wallace <Malcolm.Wallace@cs.york.ac.uk> wrote,
Alastair Reid <alastair@reid-consulting-uk.ltd.uk> writes:
So, my proposal is to: [...]
I think only GHC implements anything like this (correct me if wrong, Malcolm) and they haven't used it in the way John Meacham is interested in.
At the moment, nhc98 provides a routine
void haskellInit (int argc, char **argv)
which collects the command-line args intended for (1) the nhc98 runtime system, and (2) the response to System.getArgs. It does not strip any arguments from the given set, on the basis that the calling C routine may be interested in the same arguments that nhc98 is interested in.
Also, if the controlling C routine wants to alter the argument set before Haskell sees it, that is perfectly catered for by this interface.
I am sorry, but I really think that this interface is too restrictive. My opinion here stems less from using Haskell for implementing C libraries and more from using and implementing C libraries (in C) that need to get at some library-specific arguments. Here some requirements: * In general, hs_init() must be able to remove arguments. Some Haskell systems - eg, GHC - will allow arguments to the runtime that just don't make any sense to the user program and/or other libraries. * We must design the interface such that it does not place any restriction on the order in which libraries are initialised. The reason is that the libraries themselves may put constraints in the order of initialisation, which may conflict with any artificial ordering that we require by a less flexible interface. * For mostly transparent Haskell libraries, it is not sufficient that the C program can alter the arguments before Haskell sees it, but the C program may expect that the library (incidentally) implemented by a Haskell program already strips all arguments that are present because the library is implemented in Haskell (eg, again arguments to the RTS). Cheers, Manuel
participants (5)
-
Alastair Reid -
Fergus Henderson -
Malcolm Wallace -
Manuel M T Chakravarty -
Simon Marlow