February 2012 - Haskell-Cafe

Optimizations and parallel execution in the IO for a small spellchecker
by Paul Sujkov 16 Feb '12

16 Feb '12

Hi everyone, http://hpaste.org/63732 that's a very simple spellchecker application: it consumes standard Linux dictionary, reads a file, and prints out words from that file that are not on the dictionary. I have taken it from a little cross-language benchmark and used to benchmark existing hash table implementations available on Hackage (with one used in code being the best according to timings). The question is: can I go further and effectively execute lookup and (maybe) output (line 20 or line 26 in the snippet) in parallel? I'm not good at all with Haskell parallel packages; I've tried monad-parallel (naive implementation eats away all the memory available) and parallel (haven't tried Par monad yet), but the last dosn't seem to fit well with the IO operations. If anyone can give me any suggestions on how can I implement parallel execution, or any thoughts on further sequential optimizations, it would be deeply appreciated. -- Regards, Paul Sujkov

1 1

Concurrency strategy for 2 threads and rare events
by JP Moresmau 16 Feb '12

16 Feb '12

Hello, I'm wondering what's the best strategy to use in the following scenario: - 2 threads - One perform some work that will take time, possibly go on forever - Another waits for user input (like commands from the keyboard) that affects thread 1 (causing it to stop, in the simplest case) I've read a bit on MVar and channels, but they seem to be a lot for cases where the reading thread block for input. In my case, I expect to have "something" that thread 2 updates when an event occur, and thread 1 checks it regularly. So thread 1 should not block, but should check "is there something" and there is, act on it, otherwise continue doing what it was currently doing. I suppose I could just tryTakeMVar on a MVar, but is there something more adapted to my needs? Thanks! -- JP Moresmau http://jpmoresmau.blogspot.com/

4 5

Re: [Haskell-cafe] Hackage 2 maintainership
by Duncan Coutts 16 Feb '12

16 Feb '12

On 14 February 2012 10:43, Kirill Zaborsky <qrilka(a)gmail.com> wrote: > I apologize, > But does hackage.haskell.org being down for some hours already has something > with the process of bringing up Hackage 2? No, completely independent. The server was down for a few hours (and I think restored in the morning by the sysadmin at Galois). It was while I was mirroring to the new test server, but I don't think that was the cause of the outage. Some people were joking that the old server didn't want to be retired and so was shutting down to prevent the mirroring :-) Duncan

1 0

[haskell-cafe] Some reflections on Haskell
by Doug McIlroy 15 Feb '12

15 Feb '12

Markus: "What about hoogle/hayoo and hackage?" Antoine: "Do you have any links to examples that we should imitate?" Hackage is notionally similar to the Java API documentation at http://www.oracle.com/technetwork/java/javase/documentation/ But Hackage "Documentation" pages typically only give syntax, while Java pages invariably summarize semantics. This makes a world of difference. The quality of the summaries bespeaks a lot of editorial attention above and beyond culling annotations from source code. Considerable care has been taken in describing the GHC library at http://www.haskell.org/ghc/docs/ but even there one can find absolute mystery entries like http://www.haskell.org/ghc/docs/latest/html/libraries/haskell98-2.0.0.1/Loc… Doug > It is hard to find one's way in this ecosystm. It needn't be, > as Java illustrates. To my mind Java's great contribution to the > world is its library index--light years ahead of typical > "documentation" one finds at haskell.org, which lacks the guiding > hand of a flesh-and-blood librarian. In this matter, it > seems, industrial curation can achieve clarity more easily than > open source.

4 3

Language Shootout
by Louis Wasserman 15 Feb '12

15 Feb '12

Out of curiosity, do we know why the Language Shootout has upgraded<http://shootout.alioth.debian.org/u64q/program.php?test=spectralnorm&lang=g…>to GHC 7.4.1, but still isn't using -fllvm for e.g. the spectral-norm benchmark? (It results in a nonnegligible speedup on my machine.) Louis Wasserman wasserman.louis(a)gmail.com http://profiles.google.com/wasserman.louis

2 1

Google Summer of Code 2012 Announced
by Johan Tibell 15 Feb '12

15 Feb '12

Hi all, Here's a heads-up that this year's Google of Code is kicking off. My experience from the last few years is that we can maximize the output we get from GSoC by being proactive and writing down semi-detailed explanations of what kind of projects we'd like to see, instead of letting the students pick themselves*. Here's three examples of such write-ups I did last year: http://blog.johantibell.com/2011/03/summer-of-code-project-suggestions.html Concretely: 1. Write down the project suggestions somewhere (e.g. on the wiki, your blog, etc). 2. Advertise the projects on haskell-cafe, reddit, twitter, Google+ 3. Profit. * The students tend to not know what makes a good GSoC project and often aim for something too difficult, like writing a new project from scratch instead of contributing to an old one. Contributing to widely used libraries or infrastructure usually results in * a larger benefit for the community, and * the students sticking around. My guess is that the student tend to stick around after the project is done if they contribute to infrastructure projects, as they'll see their stuff get used and will get feature requests/bug reports that will make them continue working on the project. -- Johan ---------- Forwarded message ---------- From: Carol Smith <carols(a)google.com> Date: Sat, Feb 4, 2012 at 8:43 AM Subject: Google Summer of Code 2012 Announced To: Google Summer of Code Announce < google-summer-of-code-announce(a)googlegroups.com> Hi all, We're pleased to announce that Google Summer of Code will be happening for its eighth year this year. Please check out the blog post [1] about the program and read the FAQs [2] and Timeline [3] on Melange for more information. [1] - http://google-opensource.blogspot.com/2012/02/google-summer-of-code-2012-is… [2] - http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc20… [3] - http://www.google-melange.com/gsoc/events/google/gsoc2012 Cheers, Carol -- You received this message because you are subscribed to the Google Groups "Google Summer of Code Announce" group. To post to this group, send email to google-summer-of-code-announce(a)googlegroups.com. To unsubscribe from this group, send email to google-summer-of-code-announce%2Bunsubscribe(a)googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-summer-of-code-announce?hl=en.

6 9

Building a Haskell project which depends on an old version of base
by Steven J. Murdoch 15 Feb '12

15 Feb '12

I'm trying to build a Haskell project[1] which appears to be dependent on an function in a old version of base (GHC.Handle.fillReadBuffer). I've downgraded to Haskell platform 2010.2.0.0 which comes with GHC 6.12.3 (installed by homebrew on MacOS X 10.6.8), and ghc-pkg appears to indicate the backwards compatible library for base is available: $ ghc-pkg list base /Users/ghc6/homebrew/Cellar/ghc/6.12.3/lib/ghc/package.conf.d base-3.0.3.2 base-4.2.0.2 /Users/ghc6/.ghc/i386-darwin-6.12.3/package.conf.d However, when I try to cabal build the package, I get the following error: $ cabal -v build Creating dist/build (and its parents) Creating dist/build/autogen (and its parents) Preprocessing executables for TorDNSEL-0.1.1... Building TorDNSEL-0.1.1... Building executable tordnsel... Creating dist/build/tordnsel (and its parents) Creating dist/build/tordnsel/tordnsel-tmp (and its parents) /Users/ghc6/homebrew/bin/ghc --make -o dist/build/tordnsel/tordnsel -hide-all-packages -fbuilding-cabal-package -package-conf dist/package.conf.inplace -i -idist/build/tordnsel/tordnsel-tmp -isrc -idist/build/autogen -Idist/build/autogen -Idist/build/tordnsel/tordnsel-tmp -optP-include -optPdist/build/autogen/cabal_macros.h -odir dist/build/tordnsel/tordnsel-tmp -hidir dist/build/tordnsel/tordnsel-tmp -stubdir dist/build/tordnsel/tordnsel-tmp -package-id HUnit-1.2.2.1-6237339c6ad5673d3cd8f82c4c7f6343 -package-id array-0.3.0.1-438000c197b51c147f229b54edbf632e -package-id base-3.0.3.2-260693a92016991a03206b5a55f6a411 -package-id binary-0.5.1.0-b72d4a82a5aecd9243ff0f50456b3d47 -package-id bytestring-0.9.1.7-64b858f95ff0dacc8acf396bb53cf4b9 -package-id containers-0.3.0.0-ee442470d8dcc9e45f31677c400c5379 -package-id directory-1.0.1.1-fa3418f21f42269537be694d5922dd6d -package-id mtl-1.1.0.2-336254fecf8a77054f76cb33671a5177 -package-id network-2.2.1.7-2eb3746f71ba30c3e4940b75df5ea8b8 -package-id stm-2.1.2.1-95639ea16e2e2cbf4accbabda98a1809 -package-id time-1.1.4-d9f7b7932dc3a4863006ed6b1d525856 -package-id unix-2.4.0.2-94ca14c8a73570f9d75aca642b4de0c6 -O -O2 -Wall -DVERSION="0.1.1-dev" src/tordnsel.hs -lcrypto [ 4 of 39] Compiling TorDNSEL.Util ( dist/build/tordnsel/tordnsel-tmp/TorDNSEL/Util.hs, dist/build/tordnsel/tordnsel-tmp/TorDNSEL/Util.o ) src/TorDNSEL/Util.hsc:143:23: Module `GHC.Handle' does not export `fillReadBuffer' src/TorDNSEL/Util.hsc:143:39: Module `GHC.Handle' does not export `readCharFromBuffer' src/TorDNSEL/Util.hsc:145:26: Module `GHC.IOBase' does not export `Buffer(..)' The flags to ghc indicate that base-3.0.3.2 is being requested, and the documentation for base-3.0.3.2 indicates that GHC.Handle.fillReadBuffer exists[2]. Can anyone explain why I am getting this error message? Could it be because Util is a .hsc rather than .hs file? In tordnsel.cabal I've made two changes: set Build-Depends to include "base>=2.0 && < 4" to tell ghc that I really want base-3 and removed -Werror from GHC-Options (to ignore the deprecation warnings). Otherwise it is as provided in this distribution[3]. Building manually has the same result (though oddly I get a deprecation warning from building the .hs files but not the .hsc file, which led to my suspicion about hsc being an issue. $ hsc2hs TorDNSEL/Util.hsc $ ghc -package-id base-3.0.3.2-260693a92016991a03206b5a55f6a411 --make TorDNSEL/Util.hs [1 of 3] Compiling TorDNSEL.DeepSeq ( TorDNSEL/DeepSeq.hs, TorDNSEL/DeepSeq.o ) TorDNSEL/DeepSeq.hs:1:0: Warning: Module `Prelude' is deprecated: You are using the old package `base' version 3.x. Future GHC versions will not support base version 3.x. You should update your code to use the new base version 4.x. TorDNSEL/Util.hs:1:11: Warning: -#include and INCLUDE pragmas are deprecated: They no longer have any effect [2 of 3] Compiling TorDNSEL.Compat.Exception ( TorDNSEL/Compat/Exception.hs, TorDNSEL/Compat/Exception.o ) TorDNSEL/Compat/Exception.hs:1:0: Warning: Module `Prelude' is deprecated: You are using the old package `base' version 3.x. Future GHC versions will not support base version 3.x. You should update your code to use the new base version 4.x. [3 of 3] Compiling TorDNSEL.Util ( TorDNSEL/Util.hs, TorDNSEL/Util.o ) TorDNSEL/Util.hsc:143:23: Module `GHC.Handle' does not export `fillReadBuffer' TorDNSEL/Util.hsc:143:39: Module `GHC.Handle' does not export `readCharFromBuffer' TorDNSEL/Util.hsc:145:26: Module `GHC.IOBase' does not export `Buffer(..)' Thanks in advance, Steven. [1] https://gitweb.torproject.org/tordnsel.git [2] http://hackage.haskell.org/packages/archive/base/3.0.3.2/doc/html/GHC-Handl… [3] https://gitweb.torproject.org/tordnsel.git/blob/HEAD:/tordnsel.cabal

3 2

Subject: A universal data store interface
by Greg Weber 15 Feb '12

15 Feb '12

Hi Sergiu, Thanks you for your interest in that proposal. I rushed it off a year ago. Since then we have made a lot of improvements to Persistent and the library forms a basic building block for most Yesod users and other Haskellers. Persistent offers a level of type-safety and convenience not available elsewhere (except perhaps for libraries like acid-state that are limited to in-memory storage). That being said, there are still a lot of improvements that could be made. With the effort of a GSoC volunteer we could probably get it to the point of being the go-to data storage library for Haskellers, at least those planning on using the subset of backends (likely SQL) with great support. This proposal is vague and we would need to work with you to narrow things down a bit. I am biased, but I believe the Yesod project is one of the most compelling in the Haskell ecosystem. There are a lot of different ways a GSoC project could help make things even better besides improving the associated Persistent library, and we would really like to mentor at least one GSoC student. I would open more tickets for this in the system, but I am not sure how helpful it will be. It seems that we need to reach out to more students like yourself, but I am not sure how to do that unless I see messages like these first. Greg Weber

7 19

Ranges over Rational.
by André Scholz 14 Feb '12

14 Feb '12

Hello, i stumbled upon the specification of the Enum instances of Floats, Doubles and Rationals. I understand the idea behind this specification for floating point numbers. But why was it also used for Rational? As Rationals are exact, it would make sense to use the same definition of ranges as used for Integers. The specification seems to nullify the difference between Rational and floating point types in this respect. I would consider this a, well, a mistake in the specification. Kind regards, André Scholz

1 0

How to increase performance using concurrency for sequential producer-consumer problem
by John Lato 14 Feb '12

14 Feb '12

I would use bounded STM channels (from the stm-chans package) for communication; this would keep the producer from getting too far ahead of the converters. You'd need to tag items as they're produced (an Integer should be fine) also, and keep track of the tags. A TVar should suffice for that. The basic outline is that the producer writes to a channel. Each converter thread reads from that channel, and when it's finished, checks the output index TVar. If the converter's item index is equal to the current output index,the converter puts its value into an output channel and increments the output index. A final consumer reads from the output channel and processes each item in turn. Or instead of a bounded input channel, the producer could write to a TMVar. Which is better probably depends on the details of your production pattern. You certainly could use something like iteratee-stm or the conduits variant, but they wouldn't directly help with concurrency of converters, nor with synchronization. What they would give you is concurrency between the producer, converter, and consumer. Of course you could build your own converter step to work within that framework. John L. > Date: Mon, 13 Feb 2012 16:12:22 +0100 > From: Roel van Dijk <vandijk.roel(a)gmail.com> > Subject: [Haskell-cafe] How to increase performance using concurrency > for sequential producer-consumer problem > To: Haskell Caf? <haskell-cafe(a)haskell.org> > Message-ID: > <CABw4ky7Pu_RJSj5hQ_7GS_VTqv-dA4iYV0VhEY_J1YxxYBLyvg(a)mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > Hello, > > I have a program which I believe can benefit from concurrency. But I > am wondering if the mechanisms I need already exist somewhere on > Hackage. > > Here is a sketch of my program, in literate Haskell: > >> module Problem where >> import Control.Monad ( forM_ ) > > The producer produces values. It blocks until there are now more > values to produce. Each value is given to a callback function. > >> type Producer a = (a -> IO ()) -> IO () > > The converter does some work with a value. This work is purely CPU and > it is the bottleneck of the program. The amount of work it has to do > is variable. > >> type Converter a b = a -> b > > The consumer does something with the value calculated by the > converter. It is very important that the consumer consumes the values > in the same order as they are produced. > >> type Consumer b = b -> IO () > > Dummy producer, converter and consumer: > >> producer :: Producer Int >> producer callback = forM_ [1..10] callback > >> converter :: Converter Int Int >> converter = (*10) > >> consumer :: Consumer Int >> consumer = print > > A simple driver. Does not exploit concurrency. > >> simpleDriver :: Producer a -> Converter a b -> Consumer b -> IO () >> simpleDriver producer converter consumer = producer (consumer . converter) > >> current_situation :: IO () >> current_situation = simpleDriver producer converter consumer > > Ideally I would like a driver that spawns a worker thread for each > core in my system. But the trick is in ensuring that the consumer is > offered results in the same order as they are generated by the > producer. > > I can envision that some kind of storage is necessary to keep track of > results which can not yet be offered to the consumer because it is > still waiting for an earlier result. > > Is there any package on Haskell that can help me with this problem? Or > do I have to implement it using lower level concurrency primitives? > > Regards, > Roel > >

1 0