July 2004 - Glasgow-haskell-users

RE: major missing piece to arrays?
by Simon Marlow 20 Jul '04

20 Jul '04

On 20 July 2004 02:23, John Meacham wrote: > Cool, I can build more efficient versions on these now that I know the > ByteArray# as Addr# trick. > > I am curious what the best way to go about writing specialized > versions is, > placing the copying functions in a class, with (slow) default methods > for everything and special instances for (IO)UArrays? or relying on > RULES pragmas to do the appropriate specialization? RULES is the way to go if you're just providing optimised versions. If you anticipate that there might be a semantic difference (I presume not), then a class is more appropriate. Cheers, Simon

1 0

major missing piece to arrays?
by John Meacham 19 Jul '04

19 Jul '04

Perhaps I am just missing something, but a major piece of efficient array functionality seems to be missing. Namely the ability to efficiently copy spans of arrays into one another and/or compare spans of memory. (basically memcpy and memcmp from C). any particular reason these basic building blocks were left out? I am trying to write an efficient packed string library and having to go through intermediate lists kills the garbage collector as well as the overall speed of my program. Ideally concatination should be a simple alloc + pair of memcpys. (yeah, I know they are both of linear order, but the constant factor on a system tuned assembler memcpy vs. chugging of the STG-machine is absurdly different when cache is taken into account) So, my question is also, as a workaround for the moment, how might I implement these rountines (without resorting to rewriting all the array functionality with Foreign)? Looking at the implementations of arrays, they all seem to be built on ByteArray#'s, if I could get a (Ptr a) out of them, that would be ideal, but I imagine that might be problematic without some mechanism to temporarily pin the address in place so the GC doesn't move it. I am thinking a family of routines. (with psuedosignatures) copySpan: range -> MArray -> whereto -> MArray -> m () extractSpan : range -> IArray -> IArray extractSpanM : range -> MArray -> m IArray saveSpan : range -> IArray -> whereto -> MArray -> m () hopefully the meaning of all is clear... perhaps better names can be come up with though. A similar set of routines which perform 'memcmp' would also be useful, but less vital at the moment. Sorry if I am just blind and missing these rountines... John -- John Meacham - ⑆repetae.net⑆john⑈

4 6

Release plans
by Simon Marlow 19 Jul '04

19 Jul '04

This message is to keep everyone updated on our future release plans. We are currently planning two new releases around the end of August: one from the STABLE branch and one from the HEAD. STABLE: 6.2.2 ------------- This will be the final release from the 6.2 branch, containing bugfixes only relative to 6.2.1. Please if you have any outstanding bugs in 6.2.1, drop us a note. We'll be doing a sweep through the SourceForge bug list and any other outstanding bugs that we're aware of before the release, but we'd be grateful if you could remind us if you still have an outstanding bug in 6.2.1 so that we can be sure not to miss anything. HEAD: 6.4 --------- This release will have some significant rewrites, and is likely to be less stable than 6.2.2. Highlights: - completely new back-end (post-STG) based on a C-- intermediate language, including a largely rewritten native code generator. - Source locations inside the compiler are now much more comprehensive: pretty much every syntactic entity has an exact source span, so error messages should be a lot more accurate. - generalised algebraic data types (currently in development, might not make it into the release). The following features are also slated for inclusion: - integration of some of the DData libraries - the new System.Process library - Data.Version library and System.Info.compilerVersion - various Template Haskell changes/fixes - changes to support Cabal - FFI updates (CString locale translation) - threaded RTS: I/O multiplexing simplification & speedup - new X11 & Win32 libraries(?) Many of these features aren't in the tree yet, and/or haven't been finished. The end-Aug release date is variable, depending on how much progress we make before then, and the feature list might change too. Feedback welcome as usual - I've probably forgotten lots of stuff on these lists. Cheers, Simon

3 2

RE: Release plans
by Simon Marlow 19 Jul '04

19 Jul '04

On 19 July 2004 14:20, Shae Matijs Erisson wrote: >> - generalised algebraic data types (currently in development, might >> not make it into the release). > > What does this mean exactly? http://research.microsoft.com/Users/simonpj/papers/gadt/index.htm > Ian Lynagh has built ghc-cvs debs, is anything in particular ghc-cvs > users should try to test? The new backend stuff isn't on the mainline yet, it's on the backend-hacking-branch (and there's a *lot* of uncommitted stuff in my local tree). Large changes are coming on the HEAD before the release, so hold on tight. Cheers, Simon

1 0

RE: ForeignPtr performance.
by Simon Marlow 19 Jul '04

19 Jul '04

On 17 July 2004 05:31, John Meacham wrote: > So, I was looking at the implementation of ForeignPtr's in an attempt > to > determine why they were slow, and have an idea to speed them up.. > > right now we have: > > ForeignPtr a > = ForeignPtr ForeignObj# !(IORef [IO ()]) > | MallocPtr (MutableByteArray# RealWorld) !(IORef [IO ()]) > > and I think the inderection caused by the disjunction is what is > messing things up, not allowing ForeignPtrs to be inlined even in > strict > contexts as the discriminator must still be examined. so how bout > something like > > > data ForeignPtr a = ForeignPtr Addr# !FP -- note FP should be strict > but BOXED [2] data FP = ForeignPtrObj ForeignObj# {-# UNPACK #-} > !(IORef [IO ()]) | MallocPtr (MutableByteArray# RealWorld) > {-# UNPACK #-} !(IORef [IO ()]) At first I thought this wasn't safe, but having thought about it some more I now think it might be. The reason it is safe is because all ForeignPtr access happens inside withForeignPtr, which is ended by touch#. The implementation of touch# is invisible to the compiler, so GHC's optimiser *must* ensure that the FP stays alive until the touch#, and hence the ForeignObj# also stays alive. Interestingly, unless I'm being stupid (entirely possible) this implies that we don't actually need ForeignObj# at all! This would do: data FP = ForeignPtrObj {-# UNPACK #-} !(IORef [IO ()]) | MallocPtr (MutableByteArray# RealWorld) {-# UNPACK #-} !(IORef [IO ()]) And in the ForeignPtrObj case, we place the finalizer on the IORef (we need an atomic object with well-defined allocation semantics on which to place the finalizer - just placing it on the FP wouldn't do). Getting rid of ForeignObj# is a serious win. Deleting code and improving performance at the same time rarely happens these days! Would you like to try this out? Cheers, Simon

1 0

RE: major missing piece to arrays?
by Simon Marlow 19 Jul '04

19 Jul '04

On 17 July 2004 01:47, John Meacham wrote: > so, ByteArray# seems to be equivalant to a raw pointer in speed, with > the advantage that it is garbage collected. > > however foreignptrs are twice as slow! and even slower than an IORef. Were you using mallocForeignPtr here? Or newForeignPtr? > as a tangent.. > > I have been using the > > counter :: Ptr Int > counter = unsafePerformIO (new 0) > > trick to create fast global counters in performance critical stuff, it > seems to work quite well. it would be nice if there were a way to > allocate the memory staticaly though, because then counter could be a > constant and should be much faster. > > perhaps something like the > > "foo"# :: Addr# trick? like > > foreign data counter 4 :: Ptr Int > > to reserve 4 bytes in the bss... hmm.. You could allocate the variable statically in C, and use foreign import "&foo" :: Ptr Int That should be nice & fast to access. Cheers, Simon

1 0

RE: type synonyms in instances...
by MR K P SCHUPKE 18 Jul '04

18 Jul '04

>Actually I lied. You need the -fglasgow-exts flag. Yes, I should have tried it before I posted, I just remembered reading in the manual that type synonyms were excluded... But on re reading it actually said partial applications of type synonyms were excluded, which does not affect what I wanted to do... It just tidies up a few long instances. Keean.

2 1

RE: major missing piece to arrays?
by Simon Marlow 17 Jul '04

17 Jul '04

On 15 July 2004 22:08, John Meacham wrote: > On Thu, Jul 15, 2004 at 04:20:38PM +0200, Jérémy Bobbio wrote: >> memcpy is available in Foreign.Marshal.Utils: >> >> copyBytes :: Ptr a -> Ptr a -> Int -> IO () >> >> Copies the given number of bytes from the second area (source) >> into the first (destination);the copied areas may not overlap >> >> Here is the result of a quick try to implement fast copy using it and >> Data.Array.Storable: > > Yeah, > I know I can copy areas of memory allocated via the foreign library > or C code around. What I am looking for is an efficient way to work > with > the standard Arrays as provided by Data.Array. The idea is that if you copy areas of an array using intermediate lists, GHC should do the appropriate deforestation to remove the lists. Whether this actually happens in practice or not is another matter - but it would help greatly if we had examples that we can investigate where the deforestation isn't happening properly. However, this still isn't going to be as fast as using memcpy. A general array-copying operating using memcpy() is entirely possible - you're allowed to pass a ByteArray# to a foreign function as long as it is marked 'unsafe'. Only a *pinned* ByteArray# can be passed to 'safe' FFI calls. I've wondered in the past whether we should pin all IOUArrays, which would make StorableArray almost obsolete (except for making arrays from Ptrs returned from FFI calls). However, having lots of small pinned IOUArrays could lead to bad memory performance, because each pinned object holds onto the page it resides in, wasting up to 4k of memory. > Also, in my tests, arrays implemented via ByteArray# or Ptr a seem to > be signifigantly faster than those implemented via ForeignPtr. Is this > expected? Yes, StorableArray suffers from this problem. Specialising the array operations on StorableArray might help. Cheers, Simon

2 2

Using GHC and FFI
by Dominik Oesterreich 16 Jul '04

16 Jul '04

Hello, I try to use the FFI, but it doesn't work. I write: int test (void) { return 2; } and compiled this. So I created a file named project.obj. Then I wrote a Lhs File named test.lhs >module Blah where >import Foreign.C.Types(CInt) >foreign import ccall test :: CInt >blah = test After that I wrote ghc -fffi -c test.lhs. But when I call blah from ghci I get the error message: "test.o unknown symbol '_test' I think ghc doen't link test.o to projekt.obj. What do I have to do? Thx Dominik

3 3

RE: type synonyms in instances...
by Simon Peyton-Jones 16 Jul '04

16 Jul '04

Actually I lied. You need the -fglasgow-exts flag. Simon | -----Original Message----- | From: glasgow-haskell-users-bounces(a)haskell.org [mailto:glasgow-haskell-users- | bounces(a)haskell.org] On Behalf Of Simon Peyton-Jones | Sent: 16 July 2004 15:28 | To: MR K P SCHUPKE; glasgow-haskell-users(a)haskell.org | Subject: RE: type synonyms in instances... | | It seems that GHC does this by default (which is technically wrong, | perhaps). I've just tried it. | | Simon | | | -----Original Message----- | | From: glasgow-haskell-users-bounces(a)haskell.org | [mailto:glasgow-haskell-users- | | bounces(a)haskell.org] On Behalf Of MR K P SCHUPKE | | Sent: 07 July 2004 11:34 | | To: glasgow-haskell-users(a)haskell.org | | Subject: type synonyms in instances... | | | | | | Is there any chance of enabling type synonyms in instance | declarations. | | I remember a posting where is was stated that there was no technical | | reason for not being able to do this. | | | | Could we perhaps have a command line switch to enable this... | | | | For example, I think the following:- | | | | instance (HList e HNil) where ... | | | | is more readable as: | | | | type HSingleton e = HCons e HNil | | | | instance (HSingleton e) where ... | | | | Keean. | | _______________________________________________ | | Glasgow-haskell-users mailing list | | Glasgow-haskell-users(a)haskell.org | | http://www.haskell.org/mailman/listinfo/glasgow-haskell-users | _______________________________________________ | Glasgow-haskell-users mailing list | Glasgow-haskell-users(a)haskell.org | http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

1 0