November 2002 - Glasgow-haskell-users

RE: incremental linking?
by Simon Marlow 25 Jan '03

25 Jan '03

> I too am getting link times in the several minutes range for > my modestly > sized project, I am on a standalone dual-cpu redhat linux box with > 5.04.1 (no nfs, no nuttin') > > the project is available at > http://repetae.net/john/computer/ginsu/ > > I think there is definatly something fishy going on. I don't remember > linking always taking this long, I just assumed I added a > bunch of code > to my project or something but linking takes longer than all the > other compilation stages combined. That's bizarre. I just compiled your program on my laptop (Gentoo Linux, gcc 3.2) and linking took 6-7 seconds. John, what version of gcc/binutils is on your RedHat box? I haven't been able to discern any pattern among those experiencing long link times so far, except that -export-dynamic flag used by the dynamic loader stuff seems to cause the linker to go off into space for a while. Cheers, Simon

3 4

RE: comparison of execution speed of array types
by Zdenek Dvorak 20 Jan '03

20 Jan '03

Hello, >DiffArray seems to be broken :). Either that or I'm using it >incorrectly. I've attached the relevant code, but when I don't reverse >the array everything works fine; when I reverse it the program doesn't >(seem to) halt. I've tried to use DiffArray recently and it is terribly slow. I was forced to write my own version (it is a bit faster, but still nothing great). My suspicion about the result of your measuring is that it does the updates in reversing the array one by one, thus leading to quadratic behavior. Zdenek Dvorak _________________________________________________________________ Chat with friends online, try MSN Messenger: http://messenger.msn.com

2 1

RE: Record of STRefs better than STRef to a Record?
by Simon Marlow 27 Dec '02

27 Dec '02

> If I use an STRef to a record, will a new record be created > each time I want > to update a single field? Or can I expect GHC to optimize it > and have the field of the record updated in place? You'll get a new record for each update. This might not be so bad though, depending on the number of fields in your record. > Right now I'm using a record of STRefs, like: > data E s = E{ > refi :: STRef s Int, > refc :: STRef s Char > } > > but it can get a little messy, since thread s propagates to > every datatype that uses the record type in it's definition. Here's another trick if you use this route: data E s = E{ refi :: !STRef s Int, refc :: !STRef s Char } and compile with -funbox-strict-fields. This will eliminate the boxing of the STRefs. Cheers, Simon

4 4

RE: Native Threads in the RTS
by Simon Peyton-Jones 01 Dec '02

01 Dec '02

Improving. Want to put it in CVS? Simon M can suggest where. Simon | Definitions | ~~~~~~~~~~~ | A native thread (aka OS thread) is a thread as defined by the operating | system. | A Haskell thread is [*** FIXME - How shall I put this? ***] the thing | you see from Haskell land. A "Haskell thread" encapsulates the execution of a Haskell I/O action. A Haskell thread is created by forkIO, and dies when the I/O action completes. A Haskell thread is always executed by a native thread. The Haskell RTS creates one or more "worker native threads" to execute Haskell threads. | Haskell threads may be associated at thread creation time with either | zero or one native threads. Each Native thread is associated with zero | or more native threads. zero or more *Haskell* threads. Can you give an example of when a native thread is associated with more than one Haskell thread? To avoid "A Haskell thread associated with a native thread" I'd prefer to define the term "a bound Haskell thread". I would also like to describe a "bound native thread" as one that has associated Haskell thread(s). | If a native thread is associated with one or more Haskell threads, | exactly one of the following must be true: | *) Exactly one Haskell thread associated with the native thread is | executing. | *) The native thread is executing foreign code. | *) The native thread and all Haskell threads associated with it are | blocked. You don't say (but you do mean) A bound Haskell thread can be executed only by its associated native thread You don't say (and I'm not sure if you mean) If a bound native thread blocks, all of its associated Haskell threads are blocked too If a bound Haskell thread blocks, its associate native thread and all its associated Haskell threads also block. | The thread that main runs in, threads created using forkIO and threads | created for running finalizers or signal handlers are not necessarily | associated with a native thread. However, an implementation might | choose to do so. But the impl may *not* choose a bound native thread. These must be kept inviolate. | When a "free" foreign exported function is invoked, the implementation | may freely choose what kind of Haskell thread the function is executed | in. It is not specified whether this thread is associated with a | particular OS thread or not. Again, it must not be a bound native thread. | When a foreign imported function is invoked [by Haskell code], the | foreign code is executed in the native thread associated with the | current Haskell thread, if an association exists. If the current | Haskell thread is not associated to a native thread, the implementation | may freely decide which thread to run the foreign function in. | The existing distinction between unsafe, safe and threadsafe calls | remains unchanged. If a bound Haskell thread calls a foreign import that is not labelled 'threadsafe' which calls a bound foreign export does that work? What if the foreign export was not bound? Similarly, if the foreign import was labelled 'threadsafe', would it work? It's not obvious to me. Some kind of semantics would be good.

4 3

RE: Request: suppress specific warnings at specific places
by Simon Marlow 01 Dec '02

01 Dec '02

> GHC's excellent warnings are very helpful. They would be somewhat > more so if it were possible to suppress a warning about a specific bit > of code. One possible syntax (to which I gave no commitment) would be > > {-# WOFF "non-exhaustive pattern matches" #-} > <offending code> > {-# WON "non-exhaustive pattern matches" #-} > > . Another would be > > {-# WOFF 523 #-} > <offending code> > {-# WON 523 #-} > > where 523 is a warning number emitted with the warning message. > > This would be particularly useful with the recently granted wish for > -Werror. This would be nice, but it would be tricky to implement: declarations have to be tagged with a list of "exceptions" to the prevailing warning settings. I doubt we'll do this any time soon, but patches are welcome as usual... (of course, the workaround is to put the offending code into a module of its own, and use OPTIONS to turn off the appropriate warnings). Cheers, SImon

2 2

RE: Native Threads in the RTS
by Simon Marlow 29 Nov '02

29 Nov '02

> Or, we can adopt a much weaker semantics than Wolfgang > intended and have: > > $0 <= bindees(f) - uses(f)$ > > This would allow several currently running, active Haskell threads to > all be bound to the same foreign thread. When any of these threads > makes a foreign call, the other threads could all keep running and > they would only block if they too tried to make foreign calls. From > an implementation point of view, this requires: > > 1) That foreign threads are _not_ used to execute Haskell code. > > 2) That we maintain a lock on foreign threads so that only one > Haskell thread tries to use it at a time. > > I see two potential problems with this (but would like to hear which, > if either, dominates your thoughts): > > 1) If foreign threads cannot be used to execute Haskell code, foreign > calls require (OS-level) context switches which are expensive. > > 2) Adding an implicit lock to foreign calls might surprise > programmers. Another problem, from an implementation point of view, is that we would have to surround "unsafe" foreign calls with a lot of context-switching gumph, in case the calling Haskell thread is bound to a native thread. I really think we don't want to do this. The "thread groups" idea is similar, but only allows one of the Haskell threads in a group to be executing at any one time. This means you can run the Haskell thread using its native thread, and you don't have to context switch on every C call. However, I believe it would be tricky to implement this correctly in the scheduler (not impossible, just tricky and hard to test). Cheers, Simon

3 2

RE: Native Threads in the RTS
by Simon Marlow 29 Nov '02

29 Nov '02

> > This is all getting highly confusing, as it seems we're working with > > different ideas of what's on the table. Alastair: you seem to be > > working on your own proposal - could you write it down either as a > > complete proposal or diffs to Wolfgangs? > > I did. You sent comments on it and I sent back a cleaned up version. > Here it is again. My apologies - I didn't notice you'd already proposed 'foreign import bound' so I thought you were working with a definition that we hadn't seen. Never mind. Cheers, Simon

1 0

RE: Native Threads in the RTS
by Simon Marlow 29 Nov '02

29 Nov '02

> Simon Marlow: > > > Another problem, from an implementation point of view, is that we > > would have to surround "unsafe" foreign calls with a lot of > > context-switching gumph, in case the calling Haskell thread is bound > > to a native thread. I really think we don't want to do this. > > Note that you only have to do this to foreign calls of the form: > > foreign import bound foo :: ... > > since any other calls are free to use whatever thread they feel like. Umm, this is new... the proposal doesn't have "foreign import bound", although it does have a parenthetical comment that indicates that it might be considered: There are now two kinds of foreign exported [and foreign import wrapped] functions: bound and free. and later on: When a foreign imported function is invoked [by Haskell code], the foreign code is executed in the native thread associated with the current Haskell thread, ... So currently the idea is that every foreign import is executed by the native thread bound to the current Haskell thread, if any. This is all getting highly confusing, as it seems we're working with different ideas of what's on the table. Alastair: you seem to be working on your own proposal - could you write it down either as a complete proposal or diffs to Wolfgangs? Cheers, Simon

2 1

RE: incremental linking?
by Simon Marlow 28 Nov '02

28 Nov '02

> > > More fun with Haskell-in-the-large: linking time has become the > > > main bottleneck in our development cycle. The standard solution > > > would be to use an incremental linker, but it seems that gnu does > > > not yet support this:-| > > > > Hmm, I've never heard of linking being a bottleneck. Even > GHC itself > > links in about 3-4 seconds here. One common problem is > that linking on > > a network filesystem takes a *lot* longer than linking > objects from a > > local disk. It's always a good idea to keep the build tree > on the local > > disk, even if the sources are NFS-mounted. > > I also have this problem, and while being on a local disk > rather than NFS > helps, it doesn't help all that much. For large projects, I > usually have > time to get a cup of coffee while linking (admittedly only four doors > away, but...). When on NFS, I have time to go to the local > coffeehouse... Ok, it looks like we need to investigate this. NFS isn't the problem in itself: I realised that our GHC installation is NFS-mounted on the machine I tried the experiment on, and it makes very little difference (although Linux's NFS implementation is a bit fast & loose when it comes to caching, I seem to recall). Those who experience long link times (longer than a few seconds), please reply with your - platform / OS version - versions of relevent things (GHC, GCC, binutils). - time to link 'main = print "hello"'. Does starting up GHCi take a long time? Would someone like to strace (or truss, or ktrace or whatever) the ld process and see what it is doing for all that time. Is it CPU or I/O bound? Cheers, Simon

2 1

RE: Native Threads in the RTS
by Simon Marlow 28 Nov '02

28 Nov '02

Simon P.J. writes: > Can you give an example of when a native thread is associated > with more than one Haskell thread? I gave an example in my previous message. It's when a bound Haskell thread makes a foreign call which re-enters Haskell via a bound foreign export. > You don't say (and I'm not sure if you mean) > > If a bound native thread blocks, all of its associated Haskell > threads are blocked too "When Bagpuss goes to sleep, all his friends go to sleep too." :-) (apologies to those who never watched British childrens TV in the 70s). Actually, for any given native thread, only one of its bound Haskell threads can be runnable, the others must all be blocked waiting on the result of a foreign call. So you don't really have to worry about a native thread being bound to multiple Haskell threads; this would be quite tricky to implement in the scheduler anyway. Cheers, Simon

1 0