February 2003 - Glasgow-haskell-users

RE: Using large amounts of system time
by Simon Marlow 12 Feb '03

12 Feb '03

> I appear to be using huge amounts of CPU time, and I am trying to work > out why. The commands I ran and their output are in systime_log at > http://urchin.earth.li/~ian/systime/, and all the source and other > output is there too. Essentially the program uses more system > time than > user time according to time (bash built-in, Linux x86), and > I'd like to > know why (or more precisely I'd like to reduce it, but > knowing why seems > like a good first move :-) ). > > I'm somewhat confused as to why the numbers given by time and > strace -c > don't match, but nevertheless it looks to me like the time must be due > to one or more of: > > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > rt_sigprocmask(SIG_BLOCK, [], [], 8) = 0 > --- SIGVTALRM (Virtual timer expired) --- > sigreturn() = ? (mask now []) > > I'm not sure where to go from here. Any suggestions? >From the strace -c ouput, I'd say the system time isn't being eaten up by system calls and signals. The only other system time sink is page faults and swapping activity (I believe), but from the look of it your program is only using a few MB. What does the output from +RTS -Sstderr look like? I'm stumped too, but I'm intrigued to know what's going on. Cheers, Simon

2 1

RE: GHC 5.05 daily snapshots
by Simon Marlow 12 Feb '03

12 Feb '03

> The last source snapshot is dated from the 24th of january. > Is there any reason for that ? Sorry about that, a small hiccup in some recent changes I'd been making to the nightly build system. Should be fixed now, hopefully a source tarball will pop out of tonight's build. Cheers, Simon

1 0

GHC 5.05 daily snapshots
by Alain Cremieux 11 Feb '03

11 Feb '03

Hi, The last source snapshot is dated from the 24th of january. Is there any reason for that ? Thanks, Alain

1 0

Using large amounts of system time
by Ian Lynagh 10 Feb '03

10 Feb '03

Hi all, I appear to be using huge amounts of CPU time, and I am trying to work out why. The commands I ran and their output are in systime_log at http://urchin.earth.li/~ian/systime/, and all the source and other output is there too. Essentially the program uses more system time than user time according to time (bash built-in, Linux x86), and I'd like to know why (or more precisely I'd like to reduce it, but knowing why seems like a good first move :-) ). I'm somewhat confused as to why the numbers given by time and strace -c don't match, but nevertheless it looks to me like the time must be due to one or more of: rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 rt_sigprocmask(SIG_BLOCK, [], [], 8) = 0 --- SIGVTALRM (Virtual timer expired) --- sigreturn() = ? (mask now []) I'm not sure where to go from here. Any suggestions? Thanks Ian

1 0

irg.ssu.ac.kr
by Jane Brooks 09 Feb '03

09 Feb '03

Hi, I thought you might be interested in getting in-depth knowledge about your web audience and web traffic patterns in a reliable and cost-effective way. Stop Guessing - Start Knowing! - CoolStats measures web site traffic and online behavior of your visitors. - CoolStats will help you understand how to optimize your site to meet the needs of your visitors. - You get access to detailed, real-time statistical analysis of your web pages - 24 hours a day. Click at http://www.coolstats.com/viewdemo/index.html to view Online Demo. - CoolStats is the ultimate real-time tracking solution for small and mid-sized businesses. - 100% accuracy by measuring activity at the client, not via server based log files. - The fee of $19.95 is minimal compared to what it would cost you to run a tracking service yourself! Why CoolStats? - no programming to do - no servers to maintain - no software applications to install Special Offer! Now Only $19.95/month (Usual Price/$29.95). Click at http://p1j2m3a4.pdhost.com/pdsvr/www/r?1000014767.1435.15.myCEw395Bf4OFn to Sign Up now! "We needed to make business sense out of our web visitor behavior - CoolStats delivers first-class graphical reports that help us continuously improve and optimize our website to match the requirements of our target audience." BRYAN KASHILIN, BOSTON Click at http://www.coolstats.com/product/customerref.html to check what other customers say about us! For more information about our website tracking services, please visit our website or contact me directly at the below email. I look forward to hearing from you soon. Best regards, Jane Brooks CoolStats Support Email: jane_brooks(a)coolstats.com http://www.coolstats.com Don't be the last one to know! ----------------------------------------------------------------------- This message has been brought to glasgow-haskell-users(a)haskell.org. If you do not wish to receive anymore emails, please follow the opt-out instruction below. We apologize for any inconvenience. http://p1j2m3a4.pdhost.com/pdsvr/www/r?1000014767.1435.3.Xyk7dtHTzbyJuG

1 0

RE: openFile and threads
by Simon Marlow 07 Feb '03

07 Feb '03

> Now I'm wondering about ways to cut that down: > 1. How can I avoid the allocations inside of openFile? > 2. Would it help to call f6 in different threads? That is, does a > thread yield when it calls an IO function? openFile is quite expensive because it normally allocates a buffer, which I suspect accounts for most of the allocation you're seeing. It also sets up a finalizer, allocates the Handle, and does a couple of extra system calls. If the openFile isn't on the critical path, then it can help to do it in another thread, but only if the openFile is blocking (not usually the case if you're opening a normal file). You might consider bypassing the Handle interface and going to the bare metal using the Posix library, which will cut down on the overhead in openFile. The best solution is to avoid the open altogether if possible (eg. with caching), but I don't know about your particular application so this might not be a possibility. Cheers, Simon

4 4

RE: Build error
by Simon Marlow 07 Feb '03

07 Feb '03

> ../../ghc/compiler/ghc-inplace -optc-O -optc-Wall -optc-W > -optc-Wstrict-prototypes -optc-Wmissing-prototypes > -optc-Wmissing-declarations -optc-Winline -optc-Waggregate-return > -optc-Wbad-function-cast -optc-Wcast-align -optc-I../includes > -optc-I. > -optc-Iparallel -optc-DCOMPILING_RTS > -optc-fomit-frame-pointer -ldl -O2 > -static -package-name rts -O -Rghc-timing -c Adjustor.c -o Adjustor.o > ghc: 598636 bytes, 2 GCs, 32532/32532 avg/max bytes residency > (1 samples), 5M > in use, 0.02 INIT (0.01 elapsed), 0.02 MUT (0.04 elapsed), > 0.06 GC (0.16 elapsed) :ghc>> > make[2]: *** [Adjustor.o] Error 1 > make[1]: *** [all] Error 1 > make[1]: Leaving directory `/usr/local/src/ghc-5.04.2/ghc' > make: *** [all] Error 1 It looks like GHC just plain crashed, without an error message, although it did manage to print out GC stats. Very strange. Could you try with EXTRA_HC_OPTS=-v, and send us the output? Thanks. Also, what compiler are you building GHC with? Installed from where? Cheers, Simon

1 0

Build error
by Garner, Robin 07 Feb '03

07 Feb '03

I'm trying to build ghc 5.04.2 from the source distribution (on Linux, RH 7.2), and get the following error ------------------------ ...==fptools== make all -wr; in /usr/local/src/ghc-5.04.2/ghc/rts ------------------------------------------------------------------------ ..../utils/ghc-pkg/ghc-pkg-inplace --update-package <rts.conf.inplace Reading package info from stdin... done. Expanding embedded variables...done. warning: can't find GHCi lib `HSrts.o' Saving old package config file... done. Writing new package config file... done. ..../utils/ghc-pkg/ghc-pkg-inplace -f ../driver/package.conf --update-package <rts.conf.installed Reading package info from stdin... done. Expanding embedded variables...done. warning: can't find GHCi lib `HSrts.o' Saving old package config file... done. Writing new package config file... done. .../../ghc/compiler/ghc-inplace -optc-O -optc-Wall -optc-W -optc-Wstrict-prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc-Waggregate-return -optc-Wbad-function-cast -optc-Wcast-align -optc-I../includes -optc-I. -optc-Iparallel -optc-DCOMPILING_RTS -optc-fomit-frame-pointer -ldl -O2 -static -package-name rts -O -Rghc-timing -c Adjustor.c -o Adjustor.o ghc: 598636 bytes, 2 GCs, 32532/32532 avg/max bytes residency (1 samples), 5M in use, 0.02 INIT (0.01 elapsed), 0.02 MUT (0.04 elapsed), 0.06 GC (0.16 elapsed) :ghc>> make[2]: *** [Adjustor.o] Error 1 make[1]: *** [all] Error 1 make[1]: Leaving directory `/usr/local/src/ghc-5.04.2/ghc' make: *** [all] Error 1 ---------------------------------------- If I use the analogous gcc command, everything works. How can I find out what's going wrong ? Thanks, Robin

1 0

RE: awaitEval in Concurrent Haskell
by Simon Peyton-Jones 06 Feb '03

06 Feb '03

Colin [ccing GHC users in case there are other enthusiasts out there] | we briefly discussed the lack in | Concurrent Haskell of any way to set up a data-driven | thread -- one that works on a data structure D without | ever forcing its evaluation, only proceeding with the | computation over D as and when the needed parts get | evaluated by some other thread. | | An example application is assertion-checking in a program that depends | on laziness -- and indeed I am currently supervising a student | project on this topic. Concurrent Haskell looked like | an obvious basis for implementation, but we are stuck | without data-driven threads. | | You did say that it would not take too much work for someone | familiar with the Concurrent Haskell implementation, such as yourself, | to add a suitable primitive for the expression of data-driven threads. | Is there any possibility that you could indeed do this in the near future? It's not entirely obvious what you want here. Ideally you'd be able to say f xs = assert (length xs > 10) (...) and have the predicate (length xs > 10) evaluated only when xs itself was evaluated. But that means that the evaluations performed by length would have to be suspended somehow. That seems hard, if we are sharing a single copy of length. (Not *all* evaluations performed by these assertion-checking threads must be suspended... they may create thunks of their own.) Next idea is to have a function await :: a -> b -> b which is rather like 'seq' except that it waits for its first argument to be evaluated, whereas 'seq' evaluates it. Then you could write a new version of 'length' which used await before every case expression. Is that what you had in mind? A bit fragile, isn't it? How to implement 'await'. We thought of three ways: 1. Implement a primitive checkEval :: a -> Bool, which tells whether something is evaluated; if not, it yields to other threads, as well as returning False. So now you can have a busy-wait version of await thus: await x y = case checkEval x of True -> y False -> await x y 2. Implement await directly, by putting the thread in a pool of threads that the scheduler checks at regular intervals. They each just "watch" a thunk. Still a busy-waiting solution. 3. Do the "right thing": implement 'await' by copying the thunk, replacing it by a "smart indirection" which also points to the non-suspended thread. When the indirection is evaluated, it just puts the suspended thread in the runnable pool before entering the thunk. Actually, this is probably no harder than (2). Now, it's true that it would take Simon or I less time to implement one of these than it would take your student to do, but we have so *many* things to implement, and each takes time. And you will probably change your mind several times about what the right design should be (that's what research is like). So there's a lot to be said for the wading in yourself. What I can promise is that we'd turn around questions rapidly. Simon

2 1

RE: GHC and C++ (was RE: Creating COM objects and passing out pointers to them via a COM interface)
by Simon Peyton-Jones 06 Feb '03

06 Feb '03

| 1. A facility to replace the default uncaught exception behaviour for COM | objects and similar shared library code I'm out of my depth here. I think that if those interested in COMish things get a consensus on what would be the Right Thing, and have some idea about implementation, we'd be willing to help fix GHC to do the Right Thing, if it wasn't too hard. | 2. HDirect rolled into the main GHC tree | | 3. A facility within HDirect to create COM objects based on existing | (Haskell) data structures, returning an interface pointer as a return | parameter on a call to an existing COM object. (This was the question Simon | asked if I'd had an answer to - I've attached a copy below for reference). Sounds great -- but Sigbjorn is HDirect supreme so we'll have to see what he says. Simon | -----Original Message----- | From: Sarah Thompson [mailto:sarah@telergy.com] | Sent: 04 February 2003 10:34 | To: Simon Peyton-Jones; Sigbjorn Finne | Cc: Daan Leijen; Kevin S. Millikin; Mike Thomas; glasgow-haskell-users(a)haskell.org; | Haskell-Cafe@Haskell. Org | Subject: Re: GHC and C++ (was RE: Creating COM objects and passing out pointers | to them via a COM interface) | | I'm replying to two threads at the same time and cross posting my reply, | because they are very relevant to each other. I'm sorry if anyone here ends | up seeing this more than once as a consequence. | | [s] | | ------ | > Sarah | > | > Did you get this problem sorted out? | | Not directly. I ended up building an 'outer' COM component in C++ that | mapped an MSXML-like interface onto a simpler, internal COM component that | wraps up the Haskell code. This approach seems to work well - I might have | been able to build the necessary wrapper directly in Haskell using HDirect, | but I was becoming aware that I was probably starting to take up an | unreasonable amount of Sigbjorn's time with increasingly detailed questions | and I didn't have time to figure out the necessary intricacies from scratch. | | HDirect seems very good, but I don't think it's currently in quite as stable | a state as the main GHC release. The problems I had in building a version | that works with the current GHC are an example of this. I'm not | complaining - it worked well in the end, but I do think it might be a 'good | thing' to consider rolling HDirect into the main GHC release at some point. | | The other thing that bit me was exception handling, or rather the default | behaviour when an uncaught exception 'falls out' and reaches the top level | of the runtime system. Currently, the functionality appears to be to abort | the current process. Whilst this is probably desirable for a typical | application executable (be that console or GUI based), it is a problem if it | happens inside a COM component. My thinking is quite straightforward on | this. Preferred behaviour should really be for a call to a COM component to | fail returning an error code, rather than abort the calling process. | 'Server-style' applications are usually expected to keep running when minor | faults occur, rather than terminate. HDirect/GHC's current behaviour makes | this difficult to achieve - I managed to successfully catch exceptions, but | the price was needing to use some code based on DeepSeq to force deep strict | evaluation for all state updates. This (seemingly) was the only way I could | stop exceptions from happening in the context of the automatically generated | COM wrapper - passing it a fully evaluated data structure was apparently the | only solution available. However, this does break the advantages of laziness | as applied to the current state of the object - it would be pretty neat, for | example, to be able to define a view of a database as the result of an | expression involving relational algebra combinators, with the query | evaluated one record at a time as the data is pulled back by the client | application. The need for DeepSeq forces strict evaluation, so this benefit | is lost. This is actually more of an issue than it might immediately seem - | many queries don't actually need to retrieve all of a record set before | useful things can be done. A particular example is updating a GUI, where a | list box reflects the results of a query. Lazy evaluation will allow the | screen to start updating more or less immediately even on a query that | returns thousands of records - strict evaluation might incur a delay of a | few seconds (for a very complex query or a large database) before updates | can start happening. | | In an ideal world, what I'd dearly love to see would be an HDirect option | that allows a choice between existing functionality and having calls return | a failure code, or better still allowing a user-supplied handler function to | determine the correct course of action. | | So, to sum up my wish list: | | 1. A facility to replace the default uncaught exception behaviour for COM | objects and similar shared library code | | 2. HDirect rolled into the main GHC tree | | 3. A facility within HDirect to create COM objects based on existing | (Haskell) data structures, returning an interface pointer as a return | parameter on a call to an existing COM object. (This was the question Simon | asked if I'd had an answer to - I've attached a copy below for reference). | | Sarah | | PS: I've not yet looked closely at the .NET integration in GHC yet, but I | can imagine myself having a very similar wish list in that case too. | | -------- | Hi again, | | A slightly harder problem this time. I want to implement something that | looks something like the Microsoft XML parser. This is a very COM heavy | contraption that provides an interface that looks to client applications | like a tree of COM objects that reflects the internal structure of an XML | document. | | The idea is, you start with the document object, make a method call | requesting the root node, and have a COM object that wraps that node | returned to you. You can then request the children of that node, and their | children and siblings, all which involves being passed back interface | pointers to COM objects. Effectively, there is one COM object per syntactic | item in the XML document. | | I already have my document object working nicely. This is where the actual | database resides. I want to be able to spawn views of this database as COM | objects. When I've done this in the past (using ATL mostly, euuuugh), I've | used an IDL definition something like | | HRESULT CreateView([in,string] BSTR query, [out,retval] IDbView **); | | What's the correct way to implement the necessary Haskell code to create the | coclass and return a pointer to it? I fiddled around for ages, but either | couldn't get GHC to like the types I was using, or when I finally got it to | compile, I found that my test harness was being passed an invalid COM | object. This should be simple stuff, so I'm sure someone will have done it! | | Secondly, it is a nasty, though fairly common trick, if you know your | objects are going to be created in the same process space, to allow them | some 'behind the scenes' communication. In my case, I'd like to avoid | needing to serialise and deserialise the data in a view during creation of | these child objects (of which there may be a great many, and which may be | created on the fly rather often). Ideally, I'd like to be able to do | something like: | | newView :: String -> State -> IO State | newView db (State st) = do | db <- readIORef st | r <- newIORef db | -- Insert some kind of COM magic here | return (State r) | | so the database content gets copied within the Haskell environment (lazily, | I'd hope), without needing to be serialised and deserialised. Is this | feasible? One (messy) approach might be to create a bunch of extra | interfaces on the main database object, then create 'thin wrapper' child | objects that simply make calls back into the parent object. I'm not so keen | on that for various reasons, however. | | Sarah

1 0