March 2011 - Libraries

Proposal: add forkOnIO and friends to Control.Concurrent:
by Simon Marlow 30 Mar '11

30 Mar '11

Ticket: http://hackage.haskell.org/trac/ghc/ticket/4859 I think these functions are implementation-independent enough to add to the main `Control.Concurrent` API: {{{ {- | Like 'forkIO', but lets you specify on which CPU the thread is created. Unlike a `forkIO` thread, a thread created by `forkOnIO` will stay on the same CPU for its entire lifetime (`forkIO` threads can migrate between CPUs according to the scheduling policy). `forkOnIO` is useful for overriding the scheduling policy when you know in advance how best to distribute the threads. The `Int` argument specifies the CPU number; it is interpreted modulo the value returned by 'getNumCapabilities'. -} forkOnIO :: Int -> IO () -> IO ThreadId -- | Like 'forkIOWithUnmask', but the child thread is pinned to the -- given CPU, as with 'forkOnIO'. forkOnIOWithUnmask :: Int -> ((forall a . IO a -> IO a) -> IO ()) -> IO ThreadId {- | Returns the number of Haskell threads that can run truly simultaneously (on separate physical processors) at any given time. The CPU number passed to `forkOnIO` is interpreted modulo this value. An implementation in which Haskell threads are mapped directly to OS threads might return the number of physical processor cores in the machine, and 'forkOnIO' would be implemented using the OS's affinity facilities. An implementation that schedules Haskell threads onto a smaller number of OS threads (like GHC) would return the number of such OS threads that can be running simultaneously. GHC notes: this returns the number passed as the argument to the @+RTS -N@ flag. In current implementations, the value is fixed when the program starts and never changes, but it is possible that in the future the number of capabilities might vary at runtime. -} getNumCapabilities :: IO Int -- | returns @Just x@ if the given thread was created with either -- @forkOnIO x@ or @forkOnIOWithUnmask@, or @Nothing@ otherwise. threadIsPinned :: ThreadId -> IO (Maybe Int) }}} In base 4.3.0.0 (GHC 7.0.1) we currently have `forkOnIO` and `forkOnIOUnmasked`, available from `GHC.Conc`. I am about to add the other functions to `GHC.Conc`, and deprecate `forkOnIOUnmasked` (see http://hackage.haskell.org/trac/ghc/ticket/4457) This proposal is to export the above functions from`Control.Concurrent` too. A feature request for `threadIsPinned` was submitted as http://hackage.haskell.org/trac/ghc/ticket/4457. Naming is up for grabs: I'm not at all sure that "capabilities" is a good word here, but I can't think of any better ideas. "processors" or "CPUs" don't seem quite right. Discussion period: 4 weeks (until 19 Jan 2011) Cheers, Simon

12 18

System.IO.latin1 docs
by Wolfgang Jeltsch 30 Mar '11

30 Mar '11

Hello, on <http://www.haskell.org/ghc/docs/6.12.2/html/libraries/base-4.2.0.1/System-I…> it says: > An attempt to write a character greater than '\255' to a Handle using > the latin1 encoding will result in an error. However, according to my experience (with GHC 6.12.1 and base 4.2.0.0), latin1 just outputs a character '\cccc' as cccc `mod` 256. Should the docs be changed? Best wishes, Wolfgang

8 14

ANNOUNCE: enumerator 0.4.8
by John Millikin 29 Mar '11

29 Mar '11

----------------------------------------------------------------------------------------------------------------------------------------------------- Enumerators are an efficient, predictable, and safe alternative to lazy I/O. Discovered by Oleg Kiselyov, they allow large datasets to be processed in near–constant space by pure code. Although somewhat more complex to write, using enumerators instead of lazy I/O produces more correct programs. http://hackage.haskell.org/package/enumerator http://john-millikin.com/software/enumerator/ ----------------------------------------------------------------------------------------------------------------------------------------------------- Hello -cafe, It's been a while since the last point release of enumerator. This one is sufficiently large that I think folks might want to know about it, and since I try not to spam too many announcements, I'll give a quick rundown on major changes in other 0.4.x versions as well. First, most of what I call "list analogues" -- enumerator-based versions of 'head', 'take', 'map', etc -- have been separated into three modules (Data.Enumerator.List, .Binary, and .Text) depending on what sorts of data they operate on. This separation has been an ongoing process throughout 0.4.x releases, and I think it's now complete. The old names in Data.Enumerator will continue to exist in 0.4.x versions, but will be removed in 0.5. Second, Gregory Collins and Ertugrul Soeylemez found a space leak in Iteratee's (>>=), which could cause eventual space exhaustion in some circumstances. If you use enumerators to process very large or infinite streams, you probably want to upgrade to version 0.4.7 or higher. Third, the source code PDF has seen some substantial improvement -- if you're interested in how the library is implemented, or have insomnia, read it at < http://john-millikin.com/software/enumerator/enumerator_0.4.8.pdf > Finally, there is a known issue in the current encoding of iteratees -- if an iteratee yields extra data but never consumed anything, that iteratee will violate the monad law of associativity. Oleg has updated his implementations to fix this problem, but since it would break a *lot* of dependent libraries, I'm holding off until the vague future of version 0.5. Since iteratees that yield extra data they didn't consume are invalid anyway, I hope this problem will not cause too much inconvenience. New features ----------------- * Range-limited binary file enumeration (requested + initial patch by Bardur Arantsson). * splitWhen , based on the "split" package < http://hackage.haskell.org/package/split > * 0.4.6: Typeable instances for most types (requested by Michael Snoyman) * 0.4.5: joinE , which simplifies enumerator/enumeratee composition (requested by Michael Snoyman)

3 6

Fwd: How to best deal with nullPtr from alloca and friends?
by Jason Dagit 29 Mar '11

29 Mar '11

Sending this question to Haskell-Cafe may have been a mistake on my part. Is the library list the right place for this? Since posting this, I'm now convinced that the documentation in the haddocks are wrong. It seems that alloca is designed to never return nullPtr whether that means crashing or throwing an exception. Do I just update the haddocks and submit a patch some where? ---------- Forwarded message ---------- From: Jason Dagit <dagitj(a)gmail.com> Date: Mon, Mar 28, 2011 at 9:43 PM Subject: How to best deal with nullPtr from alloca and friends? To: Haskell Cafe <haskell-cafe(a)haskell.org> I was reading up on the documentation for alloca and friends[1], which says, "If any of the allocation functions fails, a value of nullPtr<http://www.haskell.org/ghc/docs/7.0.2/html/libraries/base-4.3.1.0/Foreign-P…> is produced." It seems like every example of FFI code that I find which uses alloca ignores the possibility of a nullPtr[2, 3, 4]. So then I started trying out examples with the help of dmwit and others from #haskell. It seems that actually, alloca and friends throw exceptions: dmwit> main = allocaArray (2^30) (\ptr -> print ((nullPtr :: Ptr Double) == ptr)) <dmwit> lispy: alloca also throws an exception. <dmwit> lispy: Or rather, allocaBytes throws an exception, and alloca is implemented in terms of allocaBytes, so I'm *guessing* that alloca throws an exception. I'm on a 64bit version of windows here with more than 4GB of memory to spare for the GHC process. Unfortunately, allocaBytes takes an Int so I can't test it with a request larger than the amount of physical ram I have. Just playing around with different arguments to allocaBytes I can get different behavior: <= 2^30 bytes, works perfectly == 2^30 + 2^20 bytes, I get an "<interactive>: out of memory" message and ghci terminates == 2^31-1, I get a crash where windows pops up a little dialog saying my program (ghci) has crashed. The behavior seems to be inconsistent with the documentation. What is the correct behavior for alloca and friends and should I be checking for nullPtr? Thanks, Jason [1] http://www.haskell.org/ghc/docs/7.0.2/html/libraries/base-4.3.1.0/Foreign-M… [2] http://en.wikibooks.org/wiki/Haskell/FFI [3] http://www.haskell.org/haskellwiki/HSFFIG/Examples [4] http://book.realworldhaskell.org/read/interfacing-with-c-the-ffi.html

1 1

Assertion in IO monad
by Mitar 29 Mar '11

29 Mar '11

Hi! In Haskell there is difference between throw and throwIO. Because of that I searched for a way to assure ordering in a similar manner of my assertions. But I have not found any. Wouldn't it be useful to have also assertIO counterpart of assert? I would propose "Bool -> IO ()" type. Mitar

4 7

ANN: unix-bytestring 0.3.4
by wren ng thornton 27 Mar '11

27 Mar '11

-------------------------------------------- -- unix-bytestring 0.3.4 -------------------------------------------- The unix-bytestring package offers a full selection of Unix/Posix-specific functions for reading and writing ByteStrings to Fds. -------------------------------------------- -- Changes (since 0.3.2) -------------------------------------------- * For all the low-level functions (fd*Buf) there are now safer versions (tryFd*Buf) which return errors via Either Errno, instead of throwing IOError exceptions. These will be of particular interest to the maintainers of iteratee, iteratee-mtl, and liboleg. * All the low-level functions (and the appropriate higher-level analogues) now check that the ByteCount or Int arguments are positive. If they're negative or zero then they return immediately (saying that zero bytes were read/written, returning an empty ByteString, etc). Note that this behavior is different than the functions in @unix@ which only check if they're equal to zero, or don't check at all. -------------------------------------------- -- Future work -------------------------------------------- * The readv(2) and writev(2) functions have a number of corner cases that throw errors (e.g., if you have too many iovecs where "too many" could mean >16) but these can be detected beforehand, or worked around by making multiple calls. Unfortunately, finding out what the limit is (or whether the linked version of libc does its own workarounds) is pretty tricky and platform-dependent. So I'll work on figure that out and making some safer wrappers. -------------------------------------------- -- Links -------------------------------------------- Homepage: http://code.haskell.org/~wren/ Hackage: http://hackage.haskell.org/package/unix-bytestring Darcs: http://community.haskell.org/~wren/unix-bytestring Haddock (Darcs version): http://community.haskell.org/~wren/unix-bytestring/dist/doc/html/unix-bytes… -- Live well, ~wren

1 0

Faster timeout but is it correct?
by Bas van Dijk 25 Mar '11

25 Mar '11

Dear all, I wrote a faster implementation for System.Timeout.timeout but wonder whether it's correct. It would be great if someone can review the code. The implementation comes with a tiny Criterion benchmark: darcs get http://bifunctor.homelinux.net/~bas/bench_timeouts/ On ghc-7.0.1 with -O2 the following benchmark, which always times out, doesn't show a difference with the current timeout: timeout 1 (threadDelay 1000000) However the following benchmark, which never times out, shows nice speedups for both a non-threaded and a threaded version: timeout 1000000 (return ()) non-threaded: 3.6 faster threaded: 10.8 faster I suspect the reason why my timeout is faster is that I use only one 'catch' while the original uses two: handleJust and bracket. I have to admit that my implementation is more complex and more fragile than the original. That's why I'm not sure of it's correctness yet: newtype Timeout = Timeout Unique deriving (Eq, Typeable) instance Show Timeout where show _ = "<<timeout>>" instance Exception Timeout timeout :: Int -> IO a -> IO (Maybe a) timeout n f | n < 0 = fmap Just f | n == 0 = return Nothing | otherwise = do myTid <- myThreadId timeoutEx <- fmap Timeout newUnique uninterruptibleMask $ \restore -> do tid <- restore $ forkIO $ threadDelay n >> throwTo myTid timeoutEx let handle e = case fromException (e :: SomeException) of Just timeoutEx' | timeoutEx' == timeoutEx -> return Nothing _ -> killThread tid >> throwIO e mb <- restore (fmap Just f) `catch` handle killThread tid return mb If nobody proves it incorrect I will make a patch for the base library. Regards, Bas

6 35

System.Mem.Weak semantics question
by Andrei Paskevich 25 Mar '11

25 Mar '11

Hello, The documentation page on System.Mem.Weak [1] contains the following sentence: "The finalizer, and the relationship between the key and the value, exist regardless of whether the program keeps a reference to the Weak object or not." Our reading of this statement implies, in particular, that, - given a key k, a value v, and a weak pointer w <- mkWeak k v Nothing, - if we lose all references to v and w, - then the value v will not be garbage-collected as long as we keep a reference to k, even though we cannot possibly retrieve v from k only. Is our understanding correct? Thanks in advance and best regards, Andrei Paskevich [1] http://www.haskell.org/ghc/docs/latest/html/libraries/base/System-Mem-Weak.…

2 1

Re: efficient Bytestring mapM_ for IO/ST?
by John Lato 22 Mar '11

22 Mar '11

> > From: wren ng thornton <wren(a)freegeek.org> > On 3/21/11 4:40 PM, Brandon Moore wrote: > > Is there an efficient way to iterate over the bytes of a ByteString? > > The code I've been using (rather similar to your unsafe map) is: > > import qualified Data.ByteString.Internal as BSI > import qualified Foreign.ForeignPtr as FFI > > foldIO :: (a -> Word8 -> IO a) -> a -> ByteString -> IO a > foldIO f z0 (BSI.PS fp off len) = > FFI.withForeignPtr fp $ \p0 -> do > let q = p0 `plusPtr` (off+len) > let go z p > | z `seq` p `seq` False = undefined > | p == q = return z > | otherwise = do > w <- peek p > z' <- f z w > go z' (p `plusPtr` 1) > go z0 (p0 `plusPtr` off) > {-# INLINE foldIO #-} > > Some things to note: > > * It's a left fold rather than a right fold, just like foldM, except > that we can't generalize it to work for all monads. (We could do a right > fold just as easily by starting with p0`plusPtr`(off+len) and counting > down to p0`plusPtr`off if desired.) > > * Because we're just keeping the head pointer, we can increment it as we > go instead of using peekElemOff. This improves the performance by only > performing one addition per loop (the p++) instead of two (ix++ and > *(p+ix)), and by requiring one less register (for ix). > Out of curiosity, do you have measurements that demonstrate improved performance from this? When I did some tests with a similar problem, there was no noticeable difference between the two approaches. In my case I also needed the element index though, so it was a slightly different problem. For the OP, note that 'plusPtr' doesn't do pointer arithmetic, it increments a ptr by n bytes. This works for ByteStrings, but if you're generalizing to arbitrary storables you may prefer to use 'advancePtr', from Foreign.Marshal.Array. John L.

2 1

Re: cabal removeLink denied on Mac
by Ulrich Vollert 22 Mar '11

22 Mar '11

Hi, I had an older cabal in /Library/Frameworks/HaskellPlatform.framework/bin (?) and in ~/.cabal/bin: cabal --version cabal-install version 0.8.2 using version 1.8.0.6 of the Cabal library Calling cabal started the old cabal because I had an ~/.cabal/bin on my path. I deleted the directory /Library/Frameworks/HaskellPlatform.framework/ and ~/.cabal/bin from my path. Now...everything is fine: calling cabal starts the actual version from /Library/Haskell/bin and I am able to install any package. I dont understand the reason...but anyway...thank you very much. Regards, Ulrich Am 22.03.2011 um 15:47 schrieb Mark Lentczner: > Something is very wrong -- an executable should never have a problem reading/writing to the allocated per-process temp directory. Does this happen with all packages, or only sqllte? Also, can you verify your cabal version? After 2011.2.0.0 it should be: > > > cabal --version > cabal-install version 0.10.2 > using version 1.10.1.0 of the Cabal library > > - Mark >

1 0