- Haskell-prime - Haskell.org

RE: FFI, safe vs unsafe
by Simon Marlow 29 Mar '06

29 Mar '06

I agree with what you say, but let me summarise it if I may, because there's an assumption in what you're saying that's easy to miss. IF the combination of 'blockable' and 'reentrant' is not required by the standard, THEN we should allow foreign calls to be annotated with one or the other, rather than requiring both. I agree with this statement, but I don't necessarily agree that the predicate should be true. Indeed, given that it requires us to complicate the language and puts a greater burden on FFI library writers, there's a good argument not to. Nevertheless, we're filling out the design space, and that's a good thing. I'll try to digest the stuff that has gone past recently on to the wiki. Cheers, Simon On 29 March 2006 11:36, John Meacham wrote: > On Wed, Mar 29, 2006 at 11:15:27AM +0100, Simon Marlow wrote: >> On 29 March 2006 09:11, John Meacham wrote: >> >>> It would be nice if we can deprecate the not very informative >>> 'safe' and 'unsafe' names and use more descriptive ones that tell >>> you what is actually allowed. >>> >>> 'reentrant' - routine might call back into the haskell run-time >>> 'blockable' - routine might block indefinitly >> >> I've been meaning to bring this up. First, I don't think >> 'blockable' is the right term here. This relates to Malcolm's point >> too: > > yeah, I am not happy with that term either. 'blocking'? 'canblock'? > >> >>> Another piece of terminology to clear up. By "non-blocking foreign >>> call", you actually mean a foreign call that *can* block. As a >>> consequence of the fairness policy, you wish to place the >>> requirement on implementations that such a blocking foreign call >>> _should_not_ block progress of other Haskell threads. The >>> thread-nature of the foreign call is "blocking". The Haskell-API >>> nature is desired to be "non-blocking". >> >> Malcolm correctly notes that when I say "non-blocking" I'm referring >> to the behaviour from Haskell's point of view, not a property of the >> foreign code being invoked. >> >> In fact, whether the foreign code being invoked blocks or not is >> largely immaterial. The property we want to capture is just this: >> >> During execution of the foreign call, other Haskell threads >> should make progress as usual. >> >> It doesn't matter whether the foreign call "blocks" or not (although >> that is a common use for this feature). I'd rather call it >> 'concurrent', to indicate that the foreign call runs concurrently >> with other Haskell threads. > > 'concurrent' sounds fine to me, I have little preference. other than > please not 'threadsafe', a word so overloaded as to be meaningless :) > >> >> Back to 'reentrant' vs. 'blockable'. I'm not convinced that >> 'blockable unsafe' is that useful. The reason is that if other >> Haskell threads continue running during the call, at some point a GC >> will be required, at which point the runtime needs to traverse the >> stack of the thread involved in the foreign call, which means the >> call is subject to the same requirements as a 'reentrant' call >> anyway. I don't think it's necessary to add this finer distinction. >> Unless perhaps you have in mind an implementation that doesn't do GC >> in the traditional way... but then I'm concerned that this is >> requiring programmers to make a distinction in their code to improve >> performance for a minority implementation technique, and that's not >> good language design. > > it has nothing to do with performance, they are just fundamentally > different concepts that just happen by coincidence to have the same > solution in ghc. there is no fundamental relation between the two. > This is one of those things that I said was "GHC-centric even though > no one realizes it" :) > > in any cooperative/event loop based system, 'blockable unsafe' can be > implemented by > > 1 spawning a new system thread, calling the routine in it, having the > routine write a value to a pipe when done. the pipe is integrated into > the standard event loop of the run-time. > > however, 'blockable safe' or 'blockable reentrant' now implies that a > call may come back into the haskell run-time _on another OS level > thread_ which implys we have to set up pthread_mutexes everywhere, > perhaps switch to a completely different run-time or at least switch > to a different incoming foreign calling boilerplate. > > note that none of this has anything to do with the GC (though, likely > implementations will have to do something special with their GC stack > too) and there are a lot of other possible models of concurrency that > we have not even thought of yet. > > >> If 'reentrant' in its full glory is too hard to implement, then by >> all means don't implement it, and emit a run-time error if someone >> tries to use it. > > but reentrant is perfectly fine, blocking is perfectly fine, the > combination is not. giving up the ability to have haskell callbacks > from C code is not so good. > > > besides, for a language standard we should avoid any implementation > details so specifying _exactly_ what we mean is a good thing. the fact > that reentrant and blocking produce the same code in GHC is _very > much_ an implementation detail. > > John

2 1

RE: bringing discussions to a close
by Simon Peyton-Jones 29 Mar '06

29 Mar '06

| As mentioned in my email from Tuesday March 21 [1], I'd like to bring | most threads to a close very soon, and to document your discussions on | the wiki. The only topics that should remain open are concurrency and Just before we do I'd like to mention one point that John Hughes and I discussed: Proposal: make all pattern bindings completely monomorphic (regardless of type signatures) At the moment you can say let (f,g) = e in ... and get *polymorphic* values f,g. This causes a good deal of trouble in the implementation (think of what the System F translation looks like), and it's an odd thing to do. Why odd? Well here it's fine: data T = C (forall a. a->a) h x = let C f = e in ... Constructor C has a polymorphic argument, so we can match e against (C f) and get a polymorphic function. But in the first example, (,) only takes monomorphic arguments, so we have to generalise after selection. To put it another way, even if the "..." is definitely strict in f, we cannot transform to case e of (f,g) -> ... I've been meaning to give GHC a flag to give it this behaviour, so that I can see if any Haskell programs break. My bet is that this is a feature that is tricky to implement, but which is virtually never used. Removing it would simplify the language. I have made a ticket for this http://haskell.galois.com/trac/haskell-prime/ticket/103 Simon

2 1

RE: Pre-emptive or co-operative concurrency (was: Concurrency)
by Simon Marlow 29 Mar '06

29 Mar '06

On 29 March 2006 12:25, Ross Paterson wrote: > On Wed, Mar 29, 2006 at 11:44:12AM +0100, Simon Marlow wrote: >> It's much more useful to reserve the term "cooperative" for >> when the burden is on the programmer to insert context-switch >> points, as is the case in Hugs. This is a significant difference >> from the programmer's point of view, whereas the difference in >> imlementation between GHC and YHC is to a very close approximation >> invisible. > > The dividing line isn't "cooperative", though. Even if an > implementation did a context switch after each IO action (i.e. in > >>=), it would fail your first fairness requirement, as an IO action > looping in pure code will starve other threads (as does one that > doesn't do allocation in GHC). Good point - I didn't mean to imply that (not cooperative => fair). Cheers, Simon

1 0

FFI, safe vs unsafe
by John Meacham 29 Mar '06

29 Mar '06

It would be nice if we can deprecate the not very informative 'safe' and 'unsafe' names and use more descriptive ones that tell you what is actually allowed. 'reentrant' - routine might call back into the haskell run-time 'blockable' - routine might block indefinitly and it is not to hard to imagine others (though, perhaps pragmas might be better), not that I am suggesting we standardize these.. unless others want to. 'reentrant_tail' - will tail call a haskell routine 'reentrant_nonglobal' - will only call arguments passed to it. 'fatal' - routine always aborts or performs a non-local return 'cheap' - routine is cheap to call, may be duplicated freely by optimizer 'speculatable' - routine may be reordered and called speculatively, even if the optimizer can't prove it will eventually be used John -- John Meacham - ⑆repetae.net⑆john⑈

2 1

RE: Re[2]: important news: refocusing discussion
by Simon Marlow 29 Mar '06

29 Mar '06

On 29 March 2006 11:00, Ross Paterson wrote: > On Tue, Mar 28, 2006 at 10:25:04AM +0100, Simon Marlow wrote: >> On 28 March 2006 00:24, Ross Paterson wrote: >>> How about STM (minus retry/orElse) and TVars as the portable >>> interface? They're trivial for a single-threaded implementation, and >>> provide a comfortable interface for everyone. >> >> It just occurred to me that STM isn't completely trivial in a >> single-threaded implementation, because exceptions have to abort a >> transaction in progress. > > Almost trivial, though: > > import Prelude hiding (catch) > import Control.Exception > import Data.IORef > > -- The reference contains a rollback action to be executed on > newtype STM a = STM (IORef (IO ()) -> IO a) very clever ;-) Simon

1 0

RE: Re[2]: important news: refocusing discussion
by Simon Marlow 29 Mar '06

29 Mar '06

On 28 March 2006 00:24, Ross Paterson wrote: > On Mon, Mar 27, 2006 at 09:36:28AM +0100, Simon Marlow wrote: > >> The portable interface could be Control.Concurrent.MVar, perhaps. > > As Malcolm pointed out, using MVars requires some care, even if you > were just aiming to be thread-safe. I don't really understand the problem, maybe I'm missing something. I thought the idea would be that a thread-safe library would simply use MVar instead of IORef. So instead of this: do x <- readIORef r ... writeIORef r x' you would write do modifyMVar_ r $ \x -> ... return x' actually the second version is not only thread-safe, but exception-safe too. Malcolm's objections: > But Q2 explicitly raises the issue of whether a non-concurrent > implementation must still follow a minimum API. That could be a > reasonable requirement, if we fleshed out the detail a bit more. > The specific suggestion of requiring MVars makes me a tiny bit > worried though. After all, MVars capture the idea of > synchronisation between threads, and I assume that since a > non-concurrent implementation has only one thread, that thread will > be trying to MVar-synchronise with something that does not exist, > and hence be blocked for ever. I can imagine that there are > situations where synchronisation is entirely safe > and free of blocking, but how to specify when it would be unsafe? There's no synchronisation, because we're not writing multi-threaded code here. Just code that doesn't have any race conditions on its mutable state when run in a multi-threaded setting. Maybe you could elaborate on what problems you envisage? Back to Ross: > How about STM (minus retry/orElse) and TVars as the portable > interface? They're trivial for a single-threaded implementation, and > provide a comfortable interface for everyone. Now that's a rather good idea. It does raise the bar for the concurrent implementations, though, and STM is not nearly as mature and well-understood as MVars. There do exist implementations of STM in terms of MVars (at least two I know of). Cheers, Simon

5 4

Concurrency (was: RE: Re[2]: important news: refocusing discussion)
by Simon Marlow 29 Mar '06

29 Mar '06

On 26 March 2006 02:31, isaac jones wrote: > Possible Interests: > 1. I can write tools like filesystems, web servers, and GUIs in > Haskell' > 2. Libraries that I use are thread-safe > 3. I can compile my code with any Haskell' compiler > 4. Tools such as debuggers and tracers that claim to support Haskell' > actually work on my code. > 5. That there not be "too many Haskell's" > 6. That there be a diversity of Haskell' implementations > 7. That concurrency be reasonable to implement for existing > compilers/interpreters. > 8. That it be reasonable to implement for new compilers/interpreters. > 9. Show off how effective Haskell can be in this area (possibly > attracting new users). > > By 5 I mean that it might be nice to have a "core" Haskell and a bunch > of addenda. But this could cause no two Haskell' implementations to > be the same. (My Haskell' might have concurrency and FFI, but no class > system, or something.) The more optional addenda we have, the more we > actually fracture the language. We could be in the same situation > we're in today. > > Isaac's Interests > * 1-6, 9 > > Simon's Interests: > * He's mentioned 9, I'm sure that there are others. I'd count all of 1-9 as interests - they're all desirable. But we haven't found a design that satisfies 1-9, and in the absence of that we have to compromise somewhere. But before we get carried away figuring out all the pros and cons of various options, let me point out once again that This is just a marketing decision Because (a) we're going to standardise concurrency anyway (b) it is unlikely that Hugs or JHC will implement concurrency even if it goes into the standard (c) the question is just whether the "brand" Haskell' encompasses concurrency or not (I thought I'd use that word because I know JL likes it so much :-) Yes there are several ramifications of this decision, but none of them are technical. As I see it, we either specify Concurrency as an addendum, or NoConcurrency as an addendum, and both options are about the same amount of work. So on that note, I'll restate my preference that Haskell' should include concurrency, and leave it at that. We can start the standardisation process without arriving at a conclusion on this issue anyway. Cheers, Simon

6 5

bringing discussions to a close
by isaac jones 29 Mar '06

29 Mar '06

Greetings, As mentioned in my email from Tuesday March 21 [1], I'd like to bring most threads to a close very soon, and to document your discussions on the wiki. The only topics that should remain open are concurrency and the class system. Instructions for modifying or creating proposals is here [2]. peace, isaac [1] http://www.haskell.org//pipermail/haskell-prime/2006-March/001008.html [2] http://haskell.galois.com/cgi-bin/haskell-prime/trac.cgi/wiki/CreateProposal -- isaac jones <ijones(a)galois.com>

3 3

the following program can't be compiled with ghc 6.4.1
by Bulat Ziganshin 28 Mar '06

28 Mar '06

to be fair, it also don't work with Hugs 03 and Hugs 05 data UnboxedMutableArray i e = UMA !i !i type IOUArray i e = UnboxedMutableArray i e data Dynamic a i e = Dynamic (a i e) type DynamicIOUArray s = Dynamic IOUArray if second line substituted with the following type IOUArray = UnboxedMutableArray then all compilers are happy. i don't know, may be i should appeal to Haskell' committee instead of compiler developers (but you are the same persons, anyway :) btw, the kinds are trickiest language part, especially in Hugs what don't supports kind annotations. in one case compiler don't accept partial type (as here), in other case compiler don't accept "extraneous" parameters. as net result, making type definition that will work everywhere is a kind of shamanism -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

1 0

unicode/internalization issues
by Bulat Ziganshin 28 Mar '06

28 Mar '06

Hello haskell-prime, i've planned some time ago to open unicode/internalization wiki page, what reflects current state of the art in this area. here is the information i have, please add/correct me if i don't know something or wrong. 1. Char supports full Unicode range (about million of chars) instead of just 8-bit ASCII: implemented in GHC 6.0 and Hugs 2005 (i'm not sure about exact versions) 2. Character classification/convertion routines in Data.Char: all Unicode chars managed properly starting from GHC 6.4 and Hugs 2005. Author of this update, Dmitry Golubovsky, also provides it as additional lib for ghc 6.2.2, and i think it is possible to extend his work to work with any compiler supporting "wide" Chars. 3. Unicode support in I/O routines, i.e. ability to read/write UTF-8 encoded files and files what use other Unicode byte encodings: not implemented in any compiler, afaik, but there are 3rd-party libs: Streams library, New I/O library, and even CharIO module from jhc sources 4. Support for UTF-8 encoded source files: implemented in ghc 6.5 and jhc. afaik, ghc's support is more advanced because it uses abovementioned routines to classify Chars, so you can use any national characters in identifiers according to their case, and all other symbols in operators. because ghc 6.5 supports ONLY utf-8 encoded source files, these creates some problems when compiling files created for previous versions of ghc (or for other compilers) and using ASCII encoding with national (>chr 127) chars in comments and especially string literals. GHC team asked their users for best solution of this problem if i don't mentioned here any issues regarding unicode/internalization, please add this -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

3 2