John Meacham wrote:
However, in order to achieve that we would have to annotate the foreign functions with whether they use thread local state.
I am not opposed to that; however, you might not like that here again, there would be the safe, possibly inefficient default choice, which means "might access thread local data", and the possibly more efficient annotation that comes with a proof obligation, which says "guaranteed not to access thread local data". The main counterargument is that some libraries, like OpenGL require many *fast* nonconcurrent, nonreentrant but tls-using calls (and, nost likely, one reentrant and possibly concurrent call for the GLUT main event loop). Using OpenGL would probably be infeasible from an implementation which requires a "notls" annotation to make foreign imports fast.
it would pretty much be vital for implementing them efficiently on a non OS-threaded implemenation of the language.
True, with the implementation plan you've outlined so far. Have you considered hybrid models where most threads are state threads (all running in one OS thread) and a few threads (=the bound threads) are OS threads which are prevented from actually executing in parallel by a few well-placed locks and condition variables? You could basically write an wrapper around the state threads and pthreads libraries, and you'd get the best of both worlds. I feel it wouldn't be that hard to implement, either.
Oddly enough, depending on the implementation it might actually be easier to just make every 'threadlocal' function fully concurrent. you have already paid the cost of dealing with OS threads.
Depending on the implementation, yes. This is the case for the inefficient implementation we recommended for interpreters like Hugs in our bound threads paper; there, the implementation might be constrained by the fact that Hugs implements cooperative threading in Haskell using continuation passing in the IO monad; the interpreter itself doesn't even really know about threads. For jhc, I fell that a hybrid implementation would be better.
they are a bonus in that you can't run concurrent computing haskell threads at the same time. you get "free" concurrent threads in other languages that you would not get if the libraries just happened to be implemented in haskell. However, if the libraries were implemented in haskell, you would still get concurrency on OS blocking events because the progress guarentee says so.
Hmm... it sounds like you've been assuming cooperative scheduling, while I've been assuming preemptive scheduling (at least GHC-style preemption, which only checks after x bytes of allocation). Maybe, in a cooperative system, it is a little bit of a bonus, although I'd still want it for practical reasons. I can make my Haskell computations call yield, but how do I make a foreign library (whose author will just say "Let them use threads") cooperate? In a preemtive system, the ability to run a C computation in the background remains a normal use case, not a bonus.
The question "can I provide a certain guarantee or not" could be answered with "no" by default to flatten the learning curve a bit. My objection against having "no default" is not very strong, but I do object to specifying this "in neutral language". This situation does not call for neutral language; rather, it has to be made clear that one of the options comes with a proof obligation and the other only with a performance penalty.
you seem to be contradicting yourself, above you say a performance penalty is vitally important in the GUI case if a call takes too long, [...]
I am not. What I was talking about above was not performance, but responsiveness; it's somewhat related to fairness in scheduling. If a foreign call takes 10 microseconds instead of 10 nanoseconds, that is a performance penalty that will matter in some circumstances, and not in others (after all, people are writing real programs in Python...). If a GUI does not respond to events for more than two seconds, it is badly written. If the computer or the programming language implementation are just too slow (performance) to achieve a certain task in that time, the Right Thing To Do is to put up a progress bar and keep processing screen update events while doing it, or even do it entirely "in the background". Of course, responsiveness is not an issue for non-interactive processes, but for GUIs it is very important.
Who is to say whether a app that muddles along is better or worse than one that is generally snappy but has an occasional delay.
I am ;-). Apart from that, I feel that is a false dichotomy, as even a factor 1000 slowdown in foreign calls is no excuse to make a GUI "generally muddle along".
Though, I am a fan of neutral language in general. you can't crash the system like you can with unsafePerformIO, FFI calls that take a while and arn't already wrapped by the standard libraries are relatively rare. no need for strong language.
To the end user, all unusable programs are equivalent :-). Also, one decision can make a library unusable for interactive programs while the other can't; the language should be strong enough to make that clear to library writers who have never heard the words "state threads" and who don't care much about concurrency in general. As for your claim about the relative rarity of such calls, I see that your bias is very different from mine. My world consists mostly of console-based programs that compute something (compilers, etc.) and don't need any FFI or concurrency to speak of, and of interactive graphical applications (GUIs + games) for which the standard libraries are only a tiny fragment of the FFI world. In your world, network servers (or applications with a similar structure) seem to figure more prominently. Cheers, Wolfgang