On Fri, Feb 03, 2006 at 10:03:08AM -0600, John Goerzen wrote:

Maybe this is just me being dense, but how is poll or select concurrency? There is no multiprocessing involved; it is simply a more efficient way to find which file descriptors are ready for some I/O action.

I know, of course, that Java green threads and Haskell forkIO threads are called "threads", but I personally believe its misleading to call it concurrency -- they're not doing more than one thing at a time.

Aren't you thinking about Parallellism? http://en.wikipedia.org/wiki/Concurrency_%28computer_science%29 In computer science, concurrency is a property of systems which consist of computations that execute overlapped in time http://en.wikipedia.org/wiki/Parallel_programming Parallel computing is the simultaneous execution of the same task (split up and specially adapted) on multiple processors in order to obtain results faster. This agrees with what I have read in many texts on the subjects. Best regards Tomasz -- I am searching for programmers who are good at least in (Haskell || ML) && (Linux || FreeBSD || math) for work in Warsaw, Poland

On 2006-02-03, John Goerzen wrote:

On Fri, Feb 03, 2006 at 05:56:41PM +0100, Tomasz Zielonka wrote:

...

On Fri, Feb 03, 2006 at 10:03:08AM -0600, John Goerzen wrote:

...

I know, of course, that Java green threads and Haskell forkIO threads are called "threads", but I personally believe its misleading to call it concurrency -- they're not doing more than one thing at a time.

Aren't you thinking about Parallellism?

No.

...

http://en.wikipedia.org/wiki/Concurrency_%28computer_science%29 In computer science, concurrency is a property of systems which consist of computations that execute overlapped in time

You're not doing anything simultaneously ("overlapped in time") when you're using poll and select (only). To do something simultaneously in Unix, you'd have to either use fork() or start a thread.

That was his point. Threading is a way of structuring a program. Parallelism is a strategy for exploiting that structuring (and others). -- Aaron Denney -><-

On Fri, Feb 03, 2006 at 10:03:08AM -0600, John Goerzen wrote:

On Fri, Feb 03, 2006 at 01:00:32AM -0800, John Meacham wrote:

...

On Fri, Feb 03, 2006 at 08:40:27AM -0000, Simon Peyton-Jones wrote:

...

The interface can be a library, but (a) what libraries are available is part of the language definition and (b) it's hard to build a good implementation without runtime support. And the nature of the runtime support depends on what the library interface is.

If we had a good standard poll/select interface in System.IO then we actually could implement a lot of concurrency as a library with no (required) run-time overhead. I'd really like to see such a thing get

Maybe this is just me being dense, but how is poll or select concurrency? There is no multiprocessing involved; it is simply a more efficient way to find which file descriptors are ready for some I/O action.

Yeah, it doesn't. however I thought I'd bring it up becauese it is related and is a hole in the current haskell set up. even on implementations with concurrency such a thing is useful as many tasks actually are easier to implement by hand this way when you need fine control over scheduling and whatnot. It's not so much that it's "a more efficient" way as its the only way for any serious application. GHCs concurrency is a nice interface to it, but it is quite high-level and access to the functionality at a medium level in a standardized way would be quite beneficial and allow all the state threaded style programs without necesarisy needing full blown concurrency. Also, providing hPoll is as simple as any FFI wrapper with no implemenation consequences othen than additions to the library so it is quite a bargain indeed for what you get.

I know, of course, that Java green threads and Haskell forkIO threads are called "threads", but I personally believe its misleading to call it concurrency -- they're not doing more than one thing at a time.

this whole field is rife with ambiguous terminology. it has already been a source of confusion several times. I think 'state-threads' are the accepted term for this sort of thread, but am unsure.

...

the ability to write thread-safe (but not thread using) libraries portably. which means MVars and foreign annotations but nothing more.

Yes. Plus, I'd say, the presence of threading primitives that return certain well-defined exceptions or something along those lines, so that it's not necessary to know whether multithreading is supported at compile time.

It would be odd to have routines in the standard that are only standardized to fail :). We couldn't include those in the standard without saying what their correct behavior is when they worked, which is exactly the task I don't think we can acomplish. Actually, I think it would be difficult to even specify what those primitives are, let alone their exact semantics. Also, I can't think of any reason you would ever want to defer such a decision to run time. either your program needs concurrency and thus should fail at compile time if it isn't available or it just needs to be concurrent-safe in which case it will succeed and work portably because we have included the primitives needed to allow that.

Right now, we have forkIO, which seems, to me, like a fancy wrapper around select or poll. It's very nice, really.

I'm not clear, though, on how to integrate other C libraries that have their own async I/O systems into all of this. (For instance, LDAP or SQL libraries)

this is a well known issue even outside of haskell land. various solutions have evolved, the glib main loop, liboop, libevent, if ghc were to switch to one then that would allow some sort of interoperability but none are perfect, and each is mutually exclusive in general. this is another reason I feel a hPoll is important, because its low level control is often needed for interacting with other libraries in tricky situations like this.

The exact interaction between FFI, forkIO, forkOS, etc. is, to me, extremely vague right now. It also seems much more complex than in other languages, and perhaps varies from one Haskell implementation to the next.

I am positive it varies, even if there were a (somewhat odd) concerted effort to emulate ghcs behavior, I doubt others would get it right any time soon. It is just such a tricky field! All languages have issues here, some are better at hand waving them away though or just ignoring them. John -- John Meacham - ⑆repetae.net⑆john⑈

On Sat, Feb 04, 2006 at 01:40:26PM +0300, Bulat Ziganshin wrote:

GHC's libs (including handling of Handles) check "threaded" at run-time just to have one common compiled library instead of two ones

Yeah, but I don't expect a common compiled library between different implementations. John -- John Meacham - ⑆repetae.net⑆john⑈

On Wed, Feb 08, 2006 at 12:03:54PM +0300, Bulat Ziganshin wrote:

JM> If we had a good standard poll/select interface in System.IO then we JM> actually could implement a lot of concurrency as a library with no JM> (required) run-time overhead. I'd really like to see such a thing get JM> into the standard. Well, mainly it would just be a really useful thing JM> to have in general. If others think it is a good idea I can try to come JM> up with a suitable API and submit it to the repo.

i have delayed answering to this letter until i announced my Streams library. now i can say that such API already exists - in terms of my library you need just to write an transformer that intercepts vGetBuf/vPutBuf calls and pass them to the select/poll machinery. so you can write such transformer just now and every program that uses Streams will benefit from its usage. Converting programs that use Handles to using Streams should be also an easy task.

I was actually asking for something much more modest, which was the routine needed to pass them to the select/poll machinery. but yeah, what you say is one of my expected uses of such a routine. Once a standard IO library settles down, then I can start working on the exact API such a routine would have.

of course, Streams library is not some standard just now, and moreover - it is not compatible with JHC. the greatest problem is what i using type classes extensions available in GHC/Hugs what is not in H98 standard. so, i'm interested in pushing Haskell' to accept most advanced possible extensions in this area and, of course, in actual implementing these extensions in the Haskell compilers. alternative way to make Streams available to wider range of Haskell compilers is to strip support of streams working in monads other that IO.

Don't take the absence of a feature in jhc to mean I don't like or want that feature. There are a lot of things I don't have but that I'd definitly want to see in the language simply because I was only shooting for H98 to begin with and was more interested in a lot of the back end stuff. You should figure out the nicest design that uses just the extensions needed for the design you want. it could help us decide what goes into haskell-prime to know what is absolutely needed for good design and what is just nice to have.

if you can make select/poll transformer, at least for testing purposes, that will be really great.

Yeah, I will look into this. the basic select/poll call will have to be pretty low level, but hopefully it will allow interesting higher level constructs based on your streams or an evolution of them. John -- John Meacham - ⑆repetae.net⑆john⑈

On Fri, Feb 03, 2006 at 10:20:01AM +0100, Tomasz Zielonka wrote:

Even if concurrency is part of Haskell', it should still be clear that it doesn't affect the definition of non-concurrent Haskell' subset at all (is that true?). For example, all pure functions will be entirely thread-safe.

As another example, Ben Rudiak-Gould recently pointed out that the inclusion of stToIO breaks threaded state reasoning for ST, e.g. readSTRef won't necessarily get what your last writeSTRef wrote (because the region might be RealWorld, with other threads modifying it).