March 2012 - Haskell-Cafe

ANNOUNCE: pipes-core 0.0.1
by Paolo Capriotti 12 Mar '12

12 Mar '12

I'm pleased to announce the release of version 0.0.1 of pipes-core, a library for efficient, safe and compositional IO, similar in scope to iteratees and conduits. http://hackage.haskell.org/package/pipes-core This is a reimplementation of the original 'Pipe' concept by Gabriel Gonzales. The package documentation contains an introduction to Pipes and a detailed tutorial, so here I'll just outline some of the differences from the original version: - A single composition operator. "Strict" composition has been removed, since its role wasn't actually that clear, and its purpose for resource finalization has been replaced by better constructs (see below). - A new primitive 'tryAwait', which makes it possible to recover from termination of an upstream pipe. Using 'tryAwait' you can write pipes that are able to consume all their input and return a value (like iteratees or Conduit's sinks can), and stateful pipes that can make use of their final state before terminating. - Full exception safety and exception-handling primitives. Pipes have been augmented with 'catch' and 'finally' primitives, which allow you to recover from exceptions and ensure finalization of resources within the Pipe monad. There's no more need to use resource-simple or monadic regions together with Pipes. - Multi-channel pipes. This is a generalization of Arrows using sums instead of pairs for contexts. We provide a number of combinators that allow you to combine Pipes in much the same way as Arrows, although there is not unfortunately an alternative for the Arrow syntax. Together with pipes-core, I also released a number of accessory packages with various utilities. Here is a complete list: * pipes-extra: File readers and writers, chunk pipes. http://hackage.haskell.org/package/pipes-extra * pipes-attoparsec: Utilities to convert a parser into a pipe. http://hackage.haskell.org/package/pipes-attoparsec * pipes-conduit: Conduit adapters. http://hackage.haskell.org/package/pipes-conduit * pipes-network: Utilities to deal with sockets. Ported from conduit. http://hackage.haskell.org/package/pipes-network * pipes-zlib: Pipes to deal with zipped data. Ported from conduit. http://hackage.haskell.org/package/pipes-zlib This is an experimental release, but it should be equipped with all the functionality needed to write serious applications. I encourage people to try it out and send feedback if anything looks too simplistic or too limiting. BR, Paolo Capriotti

4 25

Global Arrays
by Clark Gaebel 12 Mar '12

12 Mar '12

In Haskell, what's the canonical way of declaring a top-level array (Data.Vector of a huge list of doubles, in my case)? Performance is key in my case. The straightforward way would just be something like: globalArray :: V.Vector Double globalArray = V.fromList [ huge list of doubles ] {-# NOINLINE globalArray #-} However, I don't want to have to run the fromList at runtime. Not only would this mean a bigger executable (having to store a linked list, instead of an array), it would be quite inefficient since we don't even use the source list! Therefore, I was thinking of storing the array in a C file: static const double globalArray[] = { huge list of doubles }; double* getGlobalArray() { return globalArray; } int getGlobalArraySize() { return sizeof(globalArray)/sizeof(globalArray[0]); } And importing it in haskell witht he FFI, followed with an unsafeCast: foreign import ccall unsafe "getGlobalArray" c_globalArray :: Ptr CDouble foreign import ccall unsafe "getGlobalArraySize" c_globalArraySize :: CInt globalArray :: V.Vector Double globalArray = V.unsafeCast $ unsafeFromForeignPtr0 (unsafePerformIO $ newForeignPtr_ c_globalArray) (fromIntegral c_globalArraySize) {-# NOINLINE globalArray #-} But this version (clearly) is full of "unsafe"ty. Is there a better way that I haven't thought of? Regards, - clark

5 8

Question about concurrency, threads and GC
by Paul Graphov 12 Mar '12

12 Mar '12

Hello Cafe! I am trying to implement networked application in Haskell. It should accept many client connections and support bidirectional conversation, that is not just loop with Request -> Response function but also sending notifications to clients etc. NB: I came from C++ background and used to explicit asynchonous operations and pools of OS threads. (Boost.Asio + Boost.Thread) There seem to be lots of goodies around (green threads, STM). The problem is that I cannot figure out how to design such applications. First of all now I have to start some threads for each client. Client state and behaviour may change depending on data that comes from it or other clients. So it seems that I have to start at least three threads: one that reads data from socket, one that sends queued messages to the socket and one that represents clients main behaviour loop. The last one will supposedly read from some STM sources, one of them being TChan/TMVar filled by thread that reads socket. Here I get stuck: how to manage that threads? I tried to search STM examples at hackage and the most interesting packages were: 1) combinatorrent - quite complex application, but it seem to be highly Erlang-inspired, so I am not sure that it is the simplest way of doing thing. 2) network-connection package - it is a library that wraps socket and uses STM internally but exposes almost the same interface with blocking operations. Thread management is quite explicit there. But this package is marked obsolete. I wondered if one can rely upon garbage collector to reap dead threads when using STM? Or is it too error prone (accidental references to STM object that prevent it from being collected) and unpredictable? I have read this post: http://blog.ezyang.com/2011/07/blockedindefinitelyonmvar/ It shows that relying upon handling BlockedIndefinitelyOn* exceptions is a bad idea but is it OK just to rely upon that blocked forever thread will die and not to kill it explicitly? And also if it is really reliable, what are exact rules when some object goes out of scope and becomes collectable? For example in blog post mentioned above in "main1" example "lock" variable is accessible until the end of do-block, i.e. if I add "takeMVar lock" line to its end, it will compile. But it is not used actually. Is it guaranteed that explicit performGC will reap it? Possibly the best answer would be just pointing me to some recent examples of concurrent network applications. Thanks!

5 9

Understanding GC time
by Thiago Negri 12 Mar '12

12 Mar '12

Hi all. I wrote a very simple program to try out parallel Haskel and check how it would look like to make use of more than one core in this language. When I tried the program with RTS option -N1, total time shows it took 2.48 seconds to complete and around 65% of that time was taken by GC. Then I tried the same program with RTS options -N2 and total time decreased to 1.15 seconds as I expected a gain here. But what I didn't expect is the GC time to drop to 0%. I guess I'm having trouble to understand the output of the RTS option -s. Can you enlighten me? The source for the testing program: > module Main where > > import Data.List (foldl1') > import Control.Parallel (par, pseq) > import Control.Arrow ((&&&)) > > f `parApp` (a, b) = a `par` (b `pseq` (f a b)) > seqApp = uncurry > > main = print result > where result = (+) `parApp` minMax list > minMax = minlist &&& maxlist > minlist = foldl1' min > maxlist = foldl1' max > list = [1..19999999] The results on a Windows 7 64bits with an Intel Core 2 Duo, compiled with GHC from Haskell Platform: c:\tmp\hs>par +RTS -s -N1 par +RTS -s -N1 20000000 803,186,152 bytes allocated in the heap 859,916,960 bytes copied during GC 233,465,740 bytes maximum residency (10 sample(s)) 30,065,860 bytes maximum slop 483 MB total memory in use (0 MB lost due to fragmentation) Generation 0: 1523 collections, 0 parallel, 0.80s, 0.75s elapsed Generation 1: 10 collections, 0 parallel, 0.83s, 0.99s elapsed Parallel GC work balance: nan (0 / 0, ideal 1) MUT time (elapsed) GC time (elapsed) Task 0 (worker) : 0.00s ( 0.90s) 0.00s ( 0.06s) Task 1 (worker) : 0.00s ( 0.90s) 0.00s ( 0.00s) Task 2 (bound) : 0.86s ( 0.90s) 1.62s ( 1.69s) SPARKS: 1 (0 converted, 0 pruned) INIT time 0.00s ( 0.00s elapsed) MUT time 0.86s ( 0.90s elapsed) GC time 1.62s ( 1.74s elapsed) EXIT time 0.00s ( 0.00s elapsed) Total time 2.48s ( 2.65s elapsed) %GC time 65.4% (65.9% elapsed) Alloc rate 936,110,032 bytes per MUT second Productivity 34.6% of total user, 32.4% of total elapsed gc_alloc_block_sync: 0 whitehole_spin: 0 gen[0].sync_large_objects: 0 gen[1].sync_large_objects: 0 c:\tmp\hs>par +RTS -s -N2 par +RTS -s -N2 20000000 1,606,279,644 bytes allocated in the heap 74,924 bytes copied during GC 28,340 bytes maximum residency (1 sample(s)) 29,004 bytes maximum slop 2 MB total memory in use (0 MB lost due to fragmentation) Generation 0: 1566 collections, 1565 parallel, 0.00s, 0.01s elapsed Generation 1: 1 collections, 1 parallel, 0.00s, 0.00s elapsed Parallel GC work balance: 1.78 (15495 / 8703, ideal 2) MUT time (elapsed) GC time (elapsed) Task 0 (worker) : 0.00s ( 0.59s) 0.00s ( 0.00s) Task 1 (worker) : 0.58s ( 0.59s) 0.00s ( 0.01s) Task 2 (bound) : 0.58s ( 0.59s) 0.00s ( 0.00s) Task 3 (worker) : 0.00s ( 0.59s) 0.00s ( 0.00s) SPARKS: 1 (1 converted, 0 pruned) INIT time 0.00s ( 0.00s elapsed) MUT time 1.15s ( 0.59s elapsed) GC time 0.00s ( 0.01s elapsed) EXIT time 0.00s ( 0.00s elapsed) Total time 1.15s ( 0.61s elapsed) %GC time 0.0% (2.4% elapsed) Alloc rate 1,391,432,695 bytes per MUT second Productivity 100.0% of total user, 190.3% of total elapsed gc_alloc_block_sync: 90 whitehole_spin: 0 gen[0].sync_large_objects: 0 gen[1].sync_large_objects: 0

6 7

Associative prefix operators in Parsec
by Troels Henriksen 12 Mar '12

12 Mar '12

Consider a simple language of logical expressions: > import Control.Applicative > import Text.Parsec hiding ((<|>), many) > import Text.Parsec.String > import Text.Parsec.Expr > > data Expr = Truth > | Falsity > | And Expr Expr > | Or Expr Expr > | Not Expr > deriving (Show, Eq) I define a simple expression parser using Parsec: > expr :: Parser Expr > expr = buildExpressionParser table (lexeme term) > <?> "expression" > > term :: Parser Expr > term = between (lexeme (char '(')) (lexeme (char ')')) expr > <|> bool > <?> "simple expression" > > bool :: Parser Expr > bool = lexeme (string "true" *> pure Truth) > <|> lexeme (string "false" *> pure Falsity) > > lexeme :: Parser a -> Parser a > lexeme p = p <* spaces > > table = [ [prefix "!" Not ] > , [binary "&" And AssocLeft ] > , [binary "|" Or AssocLeft ] > ] > > binary name fun assoc = Infix (do{ lexeme (string name); return fun }) assoc > prefix name fun = Prefix (do{ lexeme (string name); return fun }) Now this doesn't work: > test1 = parseTest expr "!!true" But this does: > test2 = parseTest expr "!(!true)" I have studied the code for buildExpressionParser, and I know why this happens (prefix operators are treated as nonassociative), but it seems like one would often want right-associative prefix operators (so test1 would work). Is there a common workaround or solution for this problem? I assume the nonassociativity in Parsec is by design and not a bug. -- \ Troels /\ Henriksen

2 3

Re: [Haskell-cafe] Why so many strings in Network.URI, System.Posix and similar libraries?
by Jason Dusek 12 Mar '12

12 Mar '12

2012/3/11 Thedward Blevins <thedward(a)barsoom.net>: > On Sun, Mar 11, 2012 at 13:33, Jason Dusek <jason.dusek(a)gmail.com> wrote: > > The syntax of URIs is a mechanism for describing data octets, > > not Unicode code points. It is at variance to describe URIs in > > terms of Unicode code points. > > This claim is at odds with the RFC you quoted: > > 2. Characters > > The URI syntax provides a method of encoding data, presumably for the sake > of identifying a resource, as a sequence of characters. The URI characters > are, in turn, frequently encoded as octets for transport or presentation. > This specification does not mandate any particular character encoding for > mapping between URI characters and the octets used to store or transmit > those characters. > > (Emphasis is mine) > > The RFC is specifically agnostic about serialization. I generally agree that > there are a lot of places where ByteString should be used, but I'm not > convinced this is one of them. Hi Thedward, I am CC'ing the list since you raise a good point that, I think, reflects on the discussion broadly. It is true that intent of the spec is to allow encoding of characters and not of bytes: I misread its intent, attending only to the productions. But due to the way URIs interact with character encoding, a general URI parser is constrained to work with ByteStrings, just the same. The RFC "...does not mandate any particular character encoding for mapping between URI characters and the octets used to store or transmit those characters..." and in Section 1.2.1 it is allowed that the encoding of may depend on the scheme: In local or regional contexts and with improving technology, users might benefit from being able to use a wider range of characters; such use is not defined by this specification. Percent-encoded octets (Section 2.1) may be used within a URI to represent characters outside the range of the US-ASCII coded character set if this representation is allowed by the scheme or by the protocol element in which the URI is referenced. It seems possible for any octet, 0x00..0xFF, to show up in a URI, and it is only after parsing the scheme that we can say whether the octet belongs there are not. Thus a general URI parser can only go as far as splitting into components and percent decoding before handing off to scheme specific validation rules (but that's a big help already!). I've implemented a parser under these principles that handles specifically URLs: http://hackage.haskell.org/package/URLb Although the intent of the spec is to represent characters, I contend it does not succeed in doing so. Is it wise to assume more semantics than are actually there? The Internet and UNIX are full of broken junk; but faithful representation would seem to be better than idealization for those occasions where we must deal with them. I'm not sure the assumption of "textiness" really helps much in practice since the Universal Character Set contains control codes and bidi characters -- data that isn't really text. -- Jason Dusek pgp // solidsnack // C1EBC57DC55144F35460C8DF1FD4C6C1FED18A2B

2 1

ANNOUNCE: freesect-0.0.5
by Ras Far 11 Mar '12

11 Mar '12

Hi, I'm pleased to announce my first package, freesect, which is a Haskell syntax extension to generalise sections. Please visit the homepage http://www.fremissant.net/freesect for more information. The package is also available on Hackage. Some previous discussion occurred on the haskell-cafe thread http://www.mail-archive.com/haskell-cafe@haskell.org/msg97643.html It's been an exhilarating experience! Kind Regards, Andrew Seniuk (rasfar on #haskell)

1 0

ANNOUNCE : Leksah 0.12
by Hamish Mackenzie 11 Mar '12

11 Mar '12

Source is in Hackage and https://github.com/leksah OS X Binary Installers http://leksah.org/packages/leksah-0.12.0.3-ghc-7.0.4.dmg http://leksah.org/packages/leksah-0.12.0.3-ghc-7.4.1.dmg Windows Binary Installers http://leksah.org/packages/leksah-0.12.0.3-ghc-6.12.3.exe http://leksah.org/packages/leksah-0.12.0.3-ghc-7.0.3.exe http://leksah.org/packages/leksah-0.12.0.3-ghc-7.0.4.exe http://leksah.org/packages/leksah-0.12.0.3-ghc-7.4.1.exe Changes to the last release include: * Support for GHC 7.2 and 7.4 * Better handling of navigation from text panes * Addition of a Files pane * Support for "cabal test" * Better printer for cabal files (when Leksah is built with Cabal >1.10) * Better candy support for literal Haskell * Syntax highlighting for quasi-quoted hamlet, julius and cassius (DSLs used in Yesod) * Several bug fixes and optimizations Thanks to Sanny Sanoff for improving some of the keyboard navigation and symbol searching among other things. Thanks to Harald Jagenteufel and Stephan Fortelny we have basic source control integration (git,svn) in a repo branch, but it is not part of this release. We hope to merge this in soon. Work has been progressing on a major restructuring of Leksah to make it a modular plugin based system from the ground up. It is a big task and there is still more to do. Unfortunately this has meant we have spent less time adding features.

3 5

Haskell-platform and ghc 7.4
by G 11 Mar '12

11 Mar '12

Am new to haskell, but one thing I wanted to try out was the performance of an app in haskel. Having issues getting it to compile under ghc 7.0.4. The owner recommend trying to upgrade to 7.2/7.4 I see a warning that the haskell-platform isn't compatible with with 7.4.1, or so my homebrew install says. Can it be made to compile ?? Can I remove the haskell-platform and then install the dev tools from the haskell-platform via cabal etc once I have moved to 7.4.x etc ? Thanks

2 2

network-conduit proxy
by grant 11 Mar '12

11 Mar '12

I am trying to get a proxy working using the network-conduit package on windows. So I send a request to port 5002 and that gets forwarded to another port 5000 where I have a simple echo server running. I made a stab at it, but get intermittent send errors after the first connection Here is the code: {-# OPTIONS -Wall #-} import Data.Conduit import Data.Conduit.Network import Network (withSocketsDo) import Control.Monad.IO.Class (liftIO) import Control.Monad.Trans.Resource main::IO () main = withSocketsDo $ runTCPClient (ClientSettings 5000 "localhost") $ \src1 sink1 -> do liftIO $ print "in tcpclient section" liftIO $ withSocketsDo $ runTCPServer (ServerSettings 5002 Nothing) $ \src sink -> do liftIO $ print "in tcpserver section" _ <- liftIO $ runResourceT $ resourceForkIO $ do src1 $$ sink return () src $$ sink1 Thanks for any help, Grant

3 14