December 2011 - Haskell-Cafe

Using QuickCheck to test against a database
by Oliver Charles 01 Dec '11

01 Dec '11

Hi! I've just spent my evening trying to use QuickCheck properties to test the database accessing functions in a project I'm working on at the moment. I've got some stuff that's working, and I wanted to see what the cafe thought about it, and if anyone had any suggestions or critiscm that can help me make it even better! I got here by trying to figure out how I want to test my application. At first I saw 2 options: use a test database and assume it exists, or have each test setup the data it needs. I like the latter more, because it minimises dependence on the outside world. So I got down to writing some HUnit tests and then had a wave of inspiration... what if I didn't even define the data, but let QuickCheck do that for me? Here's what I came up with - note the code here is somewhat paraphrased for brevity: data DBState a = DBState { initDb :: IO () , entity :: a } This type stores an action that initialises the database with enough data such that it contains whatever 'entity' is. For example, 'DBState Person' knows how to insert a specific person into the database, and carries along an in-memory log. Continuing with the Person idea, I then went and made an Arbitrary instance for DBState Person: instance Arbitrary (DBState Person) where arbitrary = do l <- Person <$> arbitrary `suchThat` (not . null) return $ DBState (void $ insertPerson l) p where insertPerson p = doSql "INSERT INTO person (name) VALUES (?)" [ personName p l ] Then it's just a case of defining some properties: prop_allPersons = monadicIO $ do dbState <- pick arbitrary :: Gen [Person] fetched <- run $ initDb `mapM` dbState >> findAllPersons assert $ fetched `eqSet` (entity `map` dbState) where eqSet a b = all (`elem` a) b && all (`elem` b) a Voila! For any database, 'findAllPersons' returns all persons. Now... onto my own critiscisms with this. Firstly, way too much boiler plate. You have to remember to apply all of the states of your arbitrary instances, which is a pain, and guaranteed to be missed. At first I thought DBState looks like it's basically a writer/IO monad, but then I couldn't actually figure out the monoid to write to. In this case it's just [Person], but in more complex property we might need 2 types of entities (for example, to ensure a join worked correctly) which makes this list heterogenous. Secondly, the initDb action is sensitive to the order actions are sequenced. Presumably, I want to test against a database that's as accurate to production as possible, which means I want foreign keys enabled. If things are initialized in the wrong order, it violates the foreign key constraint, and it's a burden if people have to determine the correct order. One solution here is to make use of deferrable constraints, but support for these is flakey at best (and it doesn't work for bracketting tests with BEGIN; ROLLBACK). These problems just make the properties harder, but they don't seem to impede the quality or usefulness of the tests. I really want to make this work, because the idea of it is quite beautiful to me. Though that might be partly because this is my first time really using QuickCheck, so I'm getting the "oh wow this is sweeeet" effect at the same time ;) Would love to hear peoples thoughts! - Ollie

5 4

List Fusion of concatMap
by Joachim Breitner 01 Dec '11

01 Dec '11

Hi, in further attempts to get a better understanding of list fusion, I am investigating under what conditions, calls to concat and concatMap succesfully fusion away. Here is my test code: import System.Exit func0 :: Int -> Bool func0 n = any (>5) [1..n] {-# NOINLINE func0 #-} func1 :: Int -> Bool func1 n = any (>5) (concatMap (\m -> [1..m]) [1..n]) {-# NOINLINE func1 #-} func2 :: Int -> Bool func2 n = any (>5) (concat (map (\m -> [1..m]) [1..n])) {-# NOINLINE func2 #-} func3 :: Int -> Bool func3 n = any (>5) [ i | m <- [1..n] , i <- [1..m]] {-# NOINLINE func3 #-} func4 :: Int -> Bool func4 n = any (>5) ( let ok m = [ i | i <- [1..m] ] in concatMap ok [1..n] ) {-# NOINLINE func4 #-} main = if func0 10 && func1 10 && func2 10 && func3 10 && func4 10 then exitSuccess else exitFailure I deliberately did not use any putStr statements in main, so that I can easily spot list types in the output of ghc -fext-core. The use of "any" and "[...]" represent any fusion-enabled producers or consumers; these are just very well-behaving and easy to spot in the core output. Here are my obeservations: * The code generated for func0 does not use any lists at all, but rather one recursive (even tail-recursive) function, as expected. This was basically a test whether I am indeed able to observe list fusion. * func1 and func2 are not fused successfully; both core outputs still contain mentions of [] (search for ZMZN; is there a tool that does this unescaping for me? ghc-core does not, it seems.) * Now the surprising fact: func3 does without any lists! The code generated contains two nested recursive functions, the inner one even tail recursive (and the outer one could have been made tail-recursive, it seems, but that would be a different issue). * func4, which is a direct one-step translation of func3 according to the semantic rules of list comprehension in the haskell report, again is not fused completely. Now I wonder: How do I need to phrase the function with concatMap such that fusion works? Does ghc treat list comprehensions differently here? Do the rewrite rules need work so that func1 and func2 do fuse correctly? Thanks for your attention, Joachim -- Joachim "nomeata" Breitner mail(a)joachim-breitner.de | nomeata(a)debian.org | GPG: 0x4743206C xmpp: nomeata(a)joachim-breitner.de | http://www.joachim-breitner.de/

1 2

Re: [Haskell-cafe] How to get a file path to the program invoked?
by dokondr 01 Dec '11

01 Dec '11

Balazs, thanks! It's great that these packages exist! On Thu, Dec 1, 2011 at 11:17 PM, Balazs Komuves <bkomuves(a)gmail.com> wrote: > > Hello, > > I'm not subscribed to haskell cafe, but I browse the archives sometimes. > > As Simon Hengel wrote there, there are two packages on Hackage > trying to solve this exact problem (since GHC and the standard library > does not provide the necessary information): > > http://hackage.haskell.org/package/executable-path > http://hackage.haskell.org/package/FindBin > > (Hackage is down at the moment, so I'm not completely sure about the > second link). > I'm the author of the first one. > > Balazs > >

1 0

Re: [Haskell-cafe] How to get a file path to the program invoked?
by dokondr 01 Dec '11

01 Dec '11

To be precise, $0 always contains the path to the program called. You are right, this path will change depending on location from which the program was called. So $0 is OK for my case, while current directory is unrelated. Try this: #!/bin/sh echo "Arg 0: $0" echo "All Parameters: [$@]" Again, any way to get the same functionality in GHC? On Thu, Dec 1, 2011 at 10:32 PM, Giovanni Tirloni <gtirloni(a)sysdroid.com>wrote: > On Thu, Dec 1, 2011 at 5:26 PM, dokondr <dokondr(a)gmail.com> wrote: > >> On the contrary, standard shell variable $0 - contains a full path to the >> program location in the directory structure, no matter from what directory >> the program was called. >> >> > Are you sure? > > $ zero.sh > ./zero.sh > > $ ./zero.sh > ./zero.sh > > $ /home/gtirloni/zero.sh > /home/gtirloni/zero.sh > > > > -- > Giovanni > > >

2 1

Hackage down!
by Edgar Gomes Araujo 01 Dec '11

01 Dec '11

Hi everybody, Is hackage.haskell.org down? I'm trying to access it but no answer, no ping response (timeout) , nothing. Is somebody else facing the same problem? Cheers, Edgar Gomes

6 5

Poll: Do you want a mascot?
by heathmatlock 01 Dec '11

01 Dec '11

Question: Do you want a mascot? Answers: Yes No -- This is an attempt to figure out if this idea is going anywhere.

26 30

Ann: Netwire 3.0.0
by Ertugrul Söylemez 01 Dec '11

01 Dec '11

Hello there, Netwire 3.0.0 is out: <http://hackage.haskell.org/package/netwire> Overview -------- Netwire is a library for high performance functional reactive programming, which calls its signal functions 'wires'. It solves the classic problems of FRP through a number of simple concepts: * It is based on an automaton arrow (similar to Yampa), so performance problems like time leaks become local. Getting high FRP performance is the art of writing good primitive automata, which is simple enough. * Wire inhibition (ArrowZero), combination (ArrowPlus) and choice (ArrowChoice) make switching combinators entirely obsolete. * Events are based on inhibition, so non-happened events take no CPU time. * Very simple internal representation: Writing custom primitive wires is easy. Since Netwire is developed mainly with libraries in mind, its internals are exposed entirely. If you need a wire that is not predefined, just write an addon. * Finally Netwire comes with a large set of predefined wires. This new version comes along with a major API change (hence the new major version number) -- hopefully the last one. The main change is that the Wire type is now a data family. This gives the flexibility of Netwire 2 with the high performance of Netwire 1. We are back to 600k FPS now for moderately complicated wires. =) Changes ------- Major changes and new features include: * Wire is now a data family. Most wire features are now addons through type classes, and every arrow can provide its own version of them, which is optimized for their respective internal representation. Mainly you get back the high performance monad-based wires from Netwire 1. * Nondeterministic wires (see below). * New wire transformers for supplying a clock, concurrent wires, and memoization/caching. * Improved ArrowLoop instance: Inhibition is now allowed inside of 'rec', as long as it doesn't break data dependencies. If it does, you get a useful error message now instead of a fatal pattern match failure. Minor changes: * Measuring FPS without disturbing performance: avgFpsInt. * The 'execute' wire is back. * More convenient wire stepping and testing functions. * Often you don't care about an event's value: gotEvent. Nondeterministic wires ---------------------- Now that Wire is a data family, you can have efficient nondeterministic wires by using a Kleisli-wrapped logic monad as the base arrow: import Control.Monad.Logic type MyWire = Wire e (Kleisli (LogicT IO)) Now you can use branching in a MyWire computation: x <- swallow branch -< [1..100] This will split the wire session into 101 branches at the first instant (it will always branch into a single inhibiting wire). Suppose you have an HTTP bot. You can now read URLs from a file and branch into an own wire for each URL: url <- swallow (branch <<< execute) -< fmap lines (readFile "urls.txt") quitWith <<< myBot -< url Performance ----------- Netwire 3 easily outperforms Netwire 2 by a factor of 10-30. Usually you will work with wire arrows on top of a Kleisli arrow. Now that Wire is a data family you can have an efficient representation of that, which gets along entirely without the Kleisli constructor wrapping. Now the main bottleneck is the monadic actions performed. This is very satisfactory. Greets, Ertugrul -- nightmare = unsafePerformIO (getWrongWife >>= sex) http://ertes.de/

1 0

List fusion of nested lists
by Joachim Breitner 01 Dec '11

01 Dec '11

Dear Cafe, I’m trying to exploit list fusion as provided by GHC (build/foldr). One function that I want to get fusable is this, it splits a list of integeres into maximal monotonous subsequences: streaks :: [Integer] -> [[Integer]] streaks [] = [] streaks (x:xs) = let (this,rest) = oneStreak (x:xs) in this:streaks rest oneStreak :: [Integer] -> ([Integer], [Integer]) oneStreak [x] = ([x],[]) oneStreak l@(x:y:_) = splitWhile2 (\a b -> a `compare` b == x `compare` y) l splitWhile2 :: (Integer -> Integer -> Bool) -> [Integer] -> ([Integer], [Integer]) splitWhile2 p [x] = ([x],[]) splitWhile2 p (x:y:xs) | p x y = let (s,r) = splitWhile2 p (y:xs) in (x:s,r) | otherwise = ([x],y:xs) Now I’d like to implement streaks in terms of build and foldr such that it is subject to list fusion. Especially, when used in concatMap (streaks . func) :: [X] -> [[Integer]] where func :: X -> [Integer] is implemented with buildr, this should ideally remove all intermediate lists. Can this be done with list fusion at all? How would I go about it? If the above example is too complicated, known how it would work for Data.List.group would help me already a lot. Greetings, Joachim -- Joachim "nomeata" Breitner mail(a)joachim-breitner.de | nomeata(a)debian.org | GPG: 0x4743206C xmpp: nomeata(a)joachim-breitner.de | http://www.joachim-breitner.de/

1 2

ANNOUNCE: data-timeout - 64-bit timeouts of nanosecond precision
by Mikhail Vorozhtsov 01 Dec '11

01 Dec '11

Hi. I grew up tired of counting milliseconds, so I wrote a small library[1] that allows me to specify time units for timeouts and convert between them. The library also provides wrapped versions of 'timeout' and 'threadDelay' functions: > threadDelay $ 1 # Minute + 30 # Second Nanosecond precision seems to be enough for RTS and POSIX calls and it also provides a good range for 64-bit representation: > maxBound :: Timeout 30500 w 3 d 23 h 34 m 33 s 709 ms 551 us 615 ns One thing that might disappoint some people is that I chose unsigned underlying type (Word64), which means that "infinite" timeouts cannot be represented. I think "negative" timeouts essentially are performance warts (a way of packing `Maybe Word63` into Word64) that muddle equality and I recommend using `Maybe Timeout` whenever timeout is optional. [1] http://hackage.haskell.org/package/data-timeout

1 0

Munich December Haskell Meeting
by Heinrich Hördegen 01 Dec '11

01 Dec '11

Dear all, on our last meeting, we decided to set up a doodle poll to determine the date of our December's get-together: http://www.doodle.com/x6x4sgnemc35myq3 If you want to join, vote! There are also new dates available for next year's meetings (with rotating week-days): http://www.haskell-munich.de/dates Have a nice December, Heinrich -- -- hoerdegen(a)funktional.info www.funktional.info --

1 0