- Haskell-Cafe - Haskell.org

I killed performance of my code with Eval and Strategies
by Janek S. 18 Nov '12

18 Nov '12

Dear Haskellers, I am reading Simon Marlow's tutorial on parallelism and I have problems with correctly using Eval monad and Strategies. I *thought* I understand them but after writing some code it turns out that obviously I don't because parallelized code is about 20 times slower. Here's a short example (code + criterion benchmarks): {-# LANGUAGE BangPatterns #-} module Main where import Control.Parallel.Strategies import Criterion.Main main :: IO () main = defaultMain [ bench "Seq" $ nf calculateSeq xs , bench "Par" $ nf calculatePar xs ] where xs = [1..16384] calculateSeq :: [Double] -> [Double] calculateSeq [] = [] calculateSeq (x:xs) = (sin . sqrt $ x) : xs calculatePar :: [Double] -> [Double] calculatePar xss = runEval $ go xss where go :: Strategy [Double] go [] = return [] go xs = do lsh <- (rpar `dot` rdeepseq) $ calculateSeq as lst <- go bs return (lsh ++ lst) where !(as, bs) = splitAt 8192 xs Compiling and running with: ghc -O2 -Wall -threaded -rtsopts -fforce-recomp -eventlog evalleak.hs ./evalleak -oreport.html -g +RTS -N2 -ls -s I get: benchmarking Seq mean: 100.5990 us, lb 100.1937 us, ub 101.1521 us, ci 0.950 std dev: 2.395003 us, lb 1.860923 us, ub 3.169562 us, ci 0.950 benchmarking Par mean: 2.233127 ms, lb 2.169669 ms, ub 2.296155 ms, ci 0.950 std dev: 323.5201 us, lb 310.2844 us, ub 344.8252 us, ci 0.950 That's a hopeless result. Looking at the spark allocation everything looks fine: SPARKS: 202 (202 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled) But analyzing eventlog with ThreadScope I see that parallel function spends most of the time doing garbage collection, which suggests that I have a memory leak somewhere. I suspected that problem might be caused by appending two lists together in the parallel implementation, but replacing this with difference lists doesn't help. Changing granularity (e.g. splitAt 512) also brings no improvement. Can anyone point me to what am I doing wrong? Janek PS. This is of course not a real world code - I know that I'd be better of using unboxed data structures for doing computations on Doubles.

3 2

How to determine correct dependency versions for a library?
by Janek S. 18 Nov '12

18 Nov '12

Recently I started developing a Haskell library and I have a question about package dependencies. Right now when I need my project to depend on some other package I only specify the package name in cabal file and don't bother with providing the package version. This works because I am the only user of my library but I am aware that if the library were to be released on Hackage I would have to supply version numbers in the dependencies. The question is how to determine proper version numbers? I can be conservative and assume that version of libraries in my system are the minimum required ones. This is of course not a good solution, because my library might work with earlier versions but I don't know a way to check that. What is the best way to determine a minimal version of a package required by my library? I also don't see any sane way of determining maximum allowed versions for the dependencies, but looking at other packages I see that this is mostly ignored and package maintainers only supply lower versions. Is this correct approach? Janek

16 36

List all multiply/add combinations
by Rune Harder Bak 17 Nov '12

17 Nov '12

Given a list of numbers of fixed length I need to list all possible values (and the associated computation) you get by inserting +,-,*,/ between the numbers, and also set parentheses where you please. It shouldn't list computations with unnecessary parentheses. Example list of length 3 [a,b,c] and only with + and *: a*b+c, a*(b+c),a*b*c,a+b*c,(a+b)*c,a+b+c What would be a good way to do this, and a good representation in Haskell? Best, Rune

3 6

Call for Participation: Programming Languages Mentoring Workshop - a POPL workshop.
by Gareth Smith 17 Nov '12

17 Nov '12

Apologies for any duplicates: CALL FOR PARTICIPATION SIGPLAN Programming Languages Mentoring Workshop, Rome Tuesday January 22, 2013 Co-located with POPL 2013 PLMW web page: http://www.doc.ic.ac.uk/~gds/PLMW/index.html After the resounding success of the first Programming Languages Mentoring Workshop at POPL 2012, we proudly announce the 2nd SIGPLAN Programming Languages Mentoring Workshop (PLMW), co-located with POPL 2013 and organised by Nate Foster, Philippa Gardner, Alan Schmitt, Gareth Smith, Peter Thieman and Tobias Wrigstad. The purpose of this mentoring workshop is to encourage graduate students and senior undergraduate students to pursue careers in programming language research. This workshop will provide technical sessions on cutting-edge research in programming languages, and mentoring sessions on how to prepare for a research career. We will bring together leaders in programming language research from academia and industry to give talks on their research areas. The workshop will engage students in a process of imagining how they might contribute to our research community. We especially encourage women and underrepresented minority students to attend PLMW. Since PLMW will be in Rome this year, we particularly look forward to seeing Eastern European students at the workshop. This workshop is part of the activities surrounding POPL, the Symposium on Principles of Programming Languages, and takes place the day before the main conference. One goal of the workshop is to make the POPL conference more accessible to newcomers. We hope that participants will stay through the entire conference, and will also attend the POPL tutorials on Monday 21st January which are free to PLMW registered attendees. Through the generous donation of our sponsors, we are able to provide scholarships to fund student participation. These scholarships will cover reasonable expenses (airfare, hotel and registration fees) for attendance at both the workshop and the POPL conference. Students attending this year will get one year free student membership of SIGPLAN The workshop registration is open to all. Students with alternative sources of funding are welcome. APPLICATION for PLMW scholarship: The scholarship application can be accessed from the workshop web site (http://www.doc.ic.ac.uk/~gds/PLMW/index.html) The deadline for full consideration of funding is 9th December, 2012. Selected participants will be notified from Friday 14th December, and will need to register for the workshop by December 24th. SPONSORS: Imperial College London Jane Street Monoidics NSF Resource Reasoning SIGPLAN vmware

1 0

Custom Enum instance with non-consecutive indices
by Nicolas Trangez 17 Nov '12

17 Nov '12

All, I've been working on a server implementation of an existing networking protocol. The protocol uses "magic constants" in some places (e.g. to tag message types), as well as bitfields, or a combination of both packed in a single value. I created data types for both the identifiers as well as the bitfield masks, e.g. > import Data.Bits > data MessageType = Message1 > | Message2 > | Message5 > data MessageFlag = Flag1 > | Flag2 Since I need to be able to get a numeric representation of them, I thought making a custom Enum instance would make sense: > instance Enum MessageType where > fromEnum a = case a of > Message1 -> 1 > Message2 -> 2 > Message5 -> 5 > toEnum n > | n == 1 = Message1 > | n == 2 = Message2 > | n == 5 = Message5 > instance Enum MessageFlag where > fromEnum a = case a of > Flag1 -> 1 `shiftL` 0 > Flag2 -> 1 `shiftL` 1 > toEnum n > | n == 1 `shiftL` 0 = Flag1 > | n == 1 `shiftL` 1 = Flag2 This is not a complete definition (not to mention non-exhaustive pattern matches), so I was wondering what the best practices are to extend this so these instances are 'correct'? I've been trying several approaches, but all of them failed on code like > getFlags :: Int -> [MessageFlag] > getFlags i = filter (\v -> (i .&. fromEnum v) /= 0) [Flag1 ..] Unless I hard-code all options in the last list, of course, but this is obviously not the intention. Any help or pointers (including "This is not how Enum is intended to be used") would be appreciated! Thanks, Nicolas

3 5

Re: [Haskell-cafe] I killed performance of my code with Eval and Strategies
by Janek S. 17 Nov '12

17 Nov '12

> Do you really mean to calculate the 'sin . sqrt' of just the head of the list, or do you mean: > calculateSeq = map (sin . sqrt) ? Argh.. of course not! That's what you get when you code in the middle of a night. But in my code I will not be able to use map because elements will be processed in pairs, so let's say that my sequential function looks like this: calculateSeq :: [Double] -> [Double] calculateSeq [] = [] calculateSeq [x] = [sin . sqrt $ x] calculateSeq (x:y:xs) = (sin . sqrt $ x) : (cos . sqrt $ y) : calculateSeq xs > I don't think there's a memory leak. It looks more like you're just > allocating much more than is sane for such a simple function. > On a recent processor, sin . sqrt is two instructions. Meanwhile, you have > a list of (boxed?) integers being split up, then recombined. That's bound > to hurt the GC. I am not entirely convinced that my idea of using eval+strategies is bound to be slow, because there are functions like parListChunk that do exactly this: split the list into chunks, process them in parallel and then concatenate the result. Functions in Control.Parallel.Strategies were designed to deal with list so I assume it is possible to process lists in parallel without GC problems. However I do not see a way to apply these functions in my setting where elements of lists are processed in pairs, not one at a time (parList and parMap will not do). Also, working on a list of tuples will not do. > Also, you might want to configure criterion to GC between > runs. That might help. The -g flag passed to criterion executable does that. > What I'd suggest doing instead, is breaking the input into chucks of, say, > 1024, and representing it with a [Vector]. Then, run your sin.sqrt's on > each vector in parallel. Finally, use Data.Vector.concat to combine your > result. As stated in my post scriptum I am aware of that solution :) Here I'm trying to figure what am I doing wrong with Eval. Thanks! Janek > > Hope that helps, > - Clark > > On Wed, Nov 14, 2012 at 4:43 PM, Janek S. <fremenzone(a)poczta.onet.pl> wrote: > > Dear Haskellers, > > > > I am reading Simon Marlow's tutorial on parallelism and I have problems > > with correctly using Eval > > monad and Strategies. I *thought* I understand them but after writing > > some code it turns out that > > obviously I don't because parallelized code is about 20 times slower. > > Here's a short example > > (code + criterion benchmarks): > > > > {-# LANGUAGE BangPatterns #-} > > module Main where > > > > import Control.Parallel.Strategies > > import Criterion.Main > > > > main :: IO () > > main = defaultMain [ > > bench "Seq" $ nf calculateSeq xs > > , bench "Par" $ nf calculatePar xs ] > > where xs = [1..16384] > > > > calculateSeq :: [Double] -> [Double] > > calculateSeq [] = [] > > calculateSeq (x:xs) = (sin . sqrt $ x) : xs > > > > calculatePar :: [Double] -> [Double] > > calculatePar xss = runEval $ go xss > > where > > go :: Strategy [Double] > > go [] = return [] > > go xs = do > > lsh <- (rpar `dot` rdeepseq) $ calculateSeq as > > lst <- go bs > > return (lsh ++ lst) > > where > > !(as, bs) = splitAt 8192 xs > > > > Compiling and running with: > > > > ghc -O2 -Wall -threaded -rtsopts -fforce-recomp -eventlog evalleak.hs > > ./evalleak -oreport.html -g +RTS -N2 -ls -s > > > > I get: > > > > benchmarking Seq > > mean: 100.5990 us, lb 100.1937 us, ub 101.1521 us, ci 0.950 > > std dev: 2.395003 us, lb 1.860923 us, ub 3.169562 us, ci 0.950 > > > > benchmarking Par > > mean: 2.233127 ms, lb 2.169669 ms, ub 2.296155 ms, ci 0.950 > > std dev: 323.5201 us, lb 310.2844 us, ub 344.8252 us, ci 0.950 > > > > That's a hopeless result. Looking at the spark allocation everything > > looks fine: > > > > SPARKS: 202 (202 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled) > > > > But analyzing eventlog with ThreadScope I see that parallel function > > spends most of the time doing > > garbage collection, which suggests that I have a memory leak somewhere. I > > suspected that problem > > might be caused by appending two lists together in the parallel > > implementation, but replacing > > this with difference lists doesn't help. Changing granularity (e.g. > > splitAt 512) also brings no > > improvement. Can anyone point me to what am I doing wrong? > > > > Janek > > > > PS. This is of course not a real world code - I know that I'd be better > > of using unboxed data > > structures for doing computations on Doubles. > > > > _______________________________________________ > > Haskell-Cafe mailing list > > Haskell-Cafe(a)haskell.org > > http://www.haskell.org/mailman/listinfo/haskell-cafe

2 2

ANN: hledger-web 0.19
by Simon Michael 17 Nov '12

17 Nov '12

I'm pleased to announce hledger-web 0.19, compatible with the latest hledger and Yesod. Release notes: * web: builds with yesod 1.1.3 * web: obeys command-line query options at startup again * the autogenerated session file is now a dot file (.hledger-web_client_session.aes) hledger is a command-line tool and haskell library for tracking financial transactions, which are stored in a human-readable plain text format. In addition to reporting, it can also help you record new transactions, or convert CSV data from your bank. Add-on packages include hledger-web, a web interface. hledger is inspired by and compatible with John Wiegley's Ledger. For more, see http://hledger.org . Install it: cabal update; cabal install hledger-web For more installation help, see http://hledger.org/MANUAL.html#installing . Or, sponsor a ready-to-run binary for your platform: http://hledger.org/DOWNLOAD.html .

1 0

Polymorphic Type Safe URL building -- I need advice on my project
by Kyle Hanson 17 Nov '12

17 Nov '12

Hello, So I started working on a project involving polymorphic type safe urls. I know there are other type-safe url's out there, but I want some advice before I start reorganizing it. It is located here: https://github.com/dracule/web-scatter One might ask why you would want polymorphic types in your URL. Current implementation of type-safe URLs use a single type and then have it pattern match on the constructors. However, It is a pretty common occurrence that you might want to use a type from other parts of code. Where this becomes prevalent is in IxSet and AcidState. For IxSet your Index keys all have to have separate types . So with this, you can convert your url into your index key's type and then just make your handler like this: myHandler :: BlogId -> SomeWebHandler I know my code probably has some terrible inefficiencies or something, so I am looking for some feedback on some jarring problems. Thank You

1 0

Taking over ghc-core
by Shachaf Ben-Kiki 16 Nov '12

16 Nov '12

With Don Stewart's blessing (<https://twitter.com/donsbot/status/267060717843279872>), I'll be taking over maintainership of ghc-core, which hasn't been updated since 2010. I'll release a version with support for GHC 7.6 later today. Shachaf

8 8

using createProcess and waitForProcess, but mpg123 plays for only 6 seconds
by rnons 15 Nov '12

15 Nov '12

Hi guys, I'm building an application that needs to wrap an external mp3 player, in this case mpg123. Here is my code: import System.IO import System.Process import Control.Concurrent mpgLoop = do let sh = "mpg123 -R" (Just hin, Just hout, _, hdl) <- createProcess (shell sh){ std_in = CreatePipe, std_out=CreatePipe } --hPutStrLn hin "SILENCE" hPutStrLn hin "LOAD /home/rnons/Music/test.mp3" hFlush hin waitForProcess hdl return () main = do forkIO mpgLoop -- threadDelay 20000000 getChar I expected with "waitForProcess", this program will be able to run till the song ends. However, mpg123 plays for only 6 seconds. Maybe I missed something? rnons

3 3