December 2002 - Libraries

RE: getting a Binary module into the standard libs
by Simon Marlow 20 Dec '02

20 Dec '02

> In response to the two suggestions, I've linked my directory to: > > http://www.isi.edu/~hdaume/haskell/NewBinary I've made one or two small changes, the new version is attached. Summary: according to my rough measurements, the byte version is 2.5 times faster than the bit version (using your SpeedTest{Bits,Bytes}, with BinMem rather than BinIO). Compile it with -O -funbox-strict-fields. The library was designed with -funbox-strict-fields in mind (sorry I should have put a prominent comment to that effect somewhere in the source). It makes a *big* difference, especially to BinMem. > In there you will find FastMutInt, which is (almost) > unchanged, Binary.hs > which contains the new bit-based stuff, and a few test > programs. You will > also find profiles for bits and bytes (only 100000 elements > this time, but > there's still a huge discrepancy between bits and bytes) in: > > bits_io_p.prof > bytes_io_p.prof > > One thing that immediately surprises me looking at the > profile information > is that the bits version allocates 200m more heap than the > bytes version. The discrepancy is still about 200M with my version. I haven't delved into the code that the compiler generates for putBits yet, so I imagine we can probably cut this down by quite a bit. The idea is that putBits should allocate nothing (I believe putWord8 has that property). > I have a function in FastMutInt for or-ing them. The comment > next to it > says "we should optimize this: ask SimonM :)", so I am asking SimonM > now for a version which looks like incFastMutIntBy and hopefully uses > unboxed or. Nevertheless, this only accounts for 1% of the > time, so it's probably not a big deal. These functions are fine, the compiler does a good job here. I'll take a closer look if I get a chance later. Cheers, Simon

2 1

RE: getting a Binary module into the standard libs
by Simon Marlow 19 Dec '02

19 Dec '02

> I tend to agree with Alastair. I just reran the experiments > with BinIOs > instead of BinMems to see if there was a greater difference > (I expect a > much smaller difference, in fact, due to the high disk > overhead and the > fact that putChar is slow). I didn't both with -O0, but with > -O2, we get: > > bits read 2156 > write 2639 > > bytes read 1617 > write 2078 > > frankly, this shocks me. reading using bits is 33% slower; writing is > about 27% slower. i actually expected the bits version to be *faster* > (relatively, of course) here, due to the fact that we have > much smaller > files (the byte-based file is 4200000 bytes while the > bit-based file is > 3325000 bytes). With a 33% difference, I think I might want separate > instances again :). Try profiling. It's hard to tell where the speed is going without seeing the code - would you like to put it up somewhere so we can take a look? Remember that putByte and friends have been carefully tuned, I suspect we'll need to take a close look at the compiler output for your bit versions and tweak a few things. Cheers, Simon

2 1

RE: getting a Binary module into the standard libs
by Simon Marlow 18 Dec '02

18 Dec '02

> (I'll try to > do some perf measurements on BinIO vs. BinMem later today, that should > give us a rough idea). I did a rough test using GHC's current Binary library hacked to be standalone. The test was writing 100000 Ints, followed by reading back 100000 Ints. Results: using BinMem took about 0.25 secs, BinIO took about 1.1 secs. That is, BinMem is current 4-5 times faster than BinIO. I should stress that this isn't a realistic benchmark. I conjecture the difference is mostly due to the overhead of going through the IO library each time a character is written. When I wrote the Binary library I was pretty careful to make sure that putWord8/getWord8 on a BinMem was just straight-line code (that's what the FastMutInt stuff is for). The upshot is that if you want really good performance for BinIO, there will have to be a cache in the BinHandle. Cheers, Simon

3 5

Re: showUnique
by George Russell 11 Dec '02

11 Dec '02

I support the suggestion of providing instance Show Unique I would also support having instance Read Unique As John says, the representation of a Unique value as a String should be completely undefined except that u == u' iff showUnique u == showUnique u' In UniForM we have a similar type to Unique which also instances (Eq,Ord,Show,Read) One advantage of instancing Show is that it makes it easier to debug programs which use Unique values. Read is more debatable, but I'm sure we use it somewhere.

1 0

showUnique
by John Meacham 10 Dec '02

10 Dec '02

how about adding a show instance for Unique in Data.Unique? oftentimes arbitrary unique labels need to be generated, but 'hashUnique' doesnt guarentee the returned Int is unique. there is no way to add this functionality outside of the Data.Unique module since Unique is opaque. John -- --------------------------------------------------------------------------- John Meacham - California Institute of Technology, Alum. - john(a)foo.net ---------------------------------------------------------------------------

2 2

Re: Posix.installHandler issues.
by Volker Stolz 05 Dec '02

05 Dec '02

In local.libraries, you wrote: > solution: add another argument to Posix.installHandler which takes a > list of flags as described in sigaction(3) (including SA_ONESHOT). I > apologize if i am misinterpreting the behavior of my program. I'm not sure that the other flags are (generally) useful or apply at all to Haskell (e.g. SA_NODEFER: is a Haskell-handler equivalent to a C signal handler?). Furthermore, it's probably not necessary to break the API. How about a new function with the parameter indicating ONESHOT or not and then expressing installHandler through this? I don't like primInstallHandler or installHandler', so if somebody has a suggestion... BTW: One might argue that you could simply write your own in-Haskell signalstack by simply setting MVars/Semaphores in the signal handler and then have one Concurrent Haskell thread which decides what to do with those 'signals'. -- http://www-i2.informatik.rwth-aachen.de/stolz/ *** PGP *** S/MIME

1 0

Posix.installHandler issues.
by John Meacham 05 Dec '02

05 Dec '02

I cant seem to figure out how to do one-shot signals in haskell, the problem is that signals are queued to be run on a haskell thread so I cannot do a 'installHandler _ Default _' without the possiblility of more signals coming in before the thread gets scheduled. Proposed solution: add another argument to Posix.installHandler which takes a list of flags as described in sigaction(3) (including SA_ONESHOT). I apologize if i am misinterpreting the behavior of my program. John -- --------------------------------------------------------------------------- John Meacham - California Institute of Technology, Alum. - john(a)foo.net ---------------------------------------------------------------------------

1 0

RE: getting a Binary module into the standard libs
by Simon Marlow 03 Dec '02

03 Dec '02

> > BinMem and BinIO differ quite a bit here: for BinMem you can write > > straight into the array, whereas for BinIO we need a cache > - a single > > byte at the least, but ideally more. BinMem is the most > important case > > to optimise (for us in GHC anyhow), since BinIO is already > significantly > > slower due to the overhead of the Handle interface. > > Currently the BinIO implementation doesn't do any caching, though, > right? I actually rarely use the BinMem implementation, so I would be > more interested in improving the speed of BinIO. Presumably, > however, the > Handle actually does buffering, which should take care of a > large part of this, right? The problem is the overhead of the Handle interface. hPutChar is quite expensive, because it has to lock/unlock the Handle. Using a cache in the BinIO implementation would help a lot, but it also means that you have to keep exclusive access to the Handle while doing binary operations. There is already a similar problem, because the library caches the file pointer outside the Handle, so someone doing an hSeek on the same Handle will cause the cached version to be out of sync with the real one. Should multiple threads be allowed to access the same BinHandle simultaneously? Probably not in write mode, but it might be useful when reading. Maybe we could provide dupBin :: BinHandle -> IO BinHandle this is easy enough to implement, and the separate BinHandles could be used from different threads. > > There should really be a closeBin function too; it's quite simple to > > add. > > On files this would flush the current byte then close the file handle, > right? What would it do on BinMems (other than flush the byte)? closeBin can throw away a BinMem. If it needs to be written to a file, then we provide writeBinMem. Cheers, Simon

3 3