March 2010 - Haskell-Cafe

[Haskell.org Google Summer of Code] trac signup issues
by Marc D. Migge 14 Mar '10

14 Mar '10

Hello, my name is Marc Migge. I would like to sign up as a student for this year's summer of code. Could you please add me to the Trac page using the email address marcmigge(a)gmx.net? Thank you. Best regards, Marc Migge

1 0

Data.Binary.GetT or ... ?
by Juraj Hercek 14 Mar '10

14 Mar '10

Hello, I'm thinking about using Data.Binary to parse binary stream of data. Binary data stream consists of messages which can have one or more (sometimes couple of hundreds) sub-messages. The stream is spitting out data slowly. I would like to parse this data with Data.Binary.Get monad, but I would like to send sub-messages to a STM channel while parsing, so "observers" could handle them during parsing process. I believe I could achieve this with a Data.Binary.GetT transformer, but I'm not aware it exists. Implementing GetT would take me a huge amount of time (I'm not that good haskeller). Splitting processing to message level and sub-message level and handling them by two separate "get"s in IO monad is a possibility, but not really appealing one (at least to me). Is there another (preferably cheap) way of doing what I want? Thanks, Juraj

2 2

[Haskell.org Google Summer of Code] trac signup issues
by Edward Kmett 13 Mar '10

13 Mar '10

It has come to my attention that there is an issue with creating accounts on the summer-of-code trac at the moment. We're diligently working on getting the right people involved to get this fixed. In the meantime, if you want to sign up as a student or mentor for this year's summer of code, please email me with your name and desired contact email address, and I'll make sure your name winds up on the list. I apologize for the inconvenience. -Edward Kmett

3 3

Can't get Regions to show up in HGL
by Brandon Simmons 13 Mar '10

13 Mar '10

I'm having trouble finding example code that uses Regions in the Haskell Graphics Library. But the following gives me a black window when I think it should give me a black window with a square in it: ------------------ START CODE ------------------------------------------ import Graphics.HGL main = runGraphics test test = do w <- openWindow "Test" (100,100) drawInWindow w $ withRGB (RGB 255 255 255) testRegion getLBP w closeWindow w testRegion = regionToGraphic $ rectangleRegion (5,5) (30,30) ------------------ END CODE ------------------------------------------ Am I using it correctly? If I am, then any other help is greatly appreciated. I am on Debian Squeeze using GHC 6.12.1 (thanks debian maintainers!), and using HGL-3.2.0.2 Thanks, Brandon Simmons http://coder.bsimmons.name

1 0

When is a bug GHC's fault/strange STM behaviour
by Michael Lesniak 13 Mar '10

13 Mar '10

Hello, In one of my example programs I have a strange behaviour: it is a very simple taskpool using STM; in pseudocode it's 1. generate data structures 2. initialize data structures 3. fork threads 4. wait (using STM) until the pool is empty and all threads are finished 5. print a final message In very few cases, which depend on the number of threads spawned, the program hangs *after* the final message of step 5 has been printed. "Few cases" means, for example, 50.000 good, terminating runs before it hangs. If you increment the number of spawned threads (to a few hundred or thousands), it hangs much faster. Since forked threads terminate after the main thread terminates (which it should after printing the message), this behaviour is quite unexpected. Since I've experienced strange behaviour in the past which was the fault of my system configuration[1], I am a bit cautious before reporting a bug on GHC's bugtracker, especially since its reproduction is so difficult and random. So my question is how much circumspection is expected/needed before one should enter a bug in the bug tracker? I've tested the attached code on three different systems (with different linux systems, but always GHC 6.12.1 (since it's a bit costly to install the older versions)) and observed the mentioned behaviour. Is this enough to justify a bug report? Or, on the other hand, could someone spot the error in the attached code. Given my history with strange parallel behaviour, I am much more sure that it's the fault of my code, but I can't spot the error and the described behaviour (halting *after* the final message) is really strange. Cheers, Michael [1] http://www.haskell.org/pipermail/haskell-cafe/2010-March/073938.html

3 4

Upgrading to Haskell Platform
by David Place 13 Mar '10

13 Mar '10

Hi: I am running GHC 6.10.4 on Mac OSX 10.6.2. Somehow, I have a broken version of Cabal (1.6.0.3) installed. In order to fix it, I thought I should just upgrade to the Haskell Platform. Is it necessary to uninstall my current GHC first? If, so, how? Thanks you. Cheers, David

3 4

Re: [Haskell-cafe] Space leak
by Arnoldo Muller 13 Mar '10

13 Mar '10

Hello Justin, I tried and what I saw was a constant increase in memory usage. Any particular profiling option that you would use? I do remember that there was a particular option in which the leak would dissapear (for the same amount of work) and that is why I stopped using the profiler. Thanks, Arnoldo On Wed, Mar 10, 2010 at 10:20 PM, Justin Bailey <jgbailey(a)gmail.com> wrote: > Have you use the profiling tools available with GHC? > > http://haskell.org/ghc/docs/latest/html/users_guide/profiling.html > > > On Wed, Mar 10, 2010 at 12:45 PM, Arnoldo Muller > <arnoldomuller(a)gmail.com> wrote: > > Hello, > > > > I am learning haskell and I found a space leak that I find difficult to > > solve. I've been asking at #haskell but we could not solve > > the issue. > > > > I want to lazily read a set of 22 files of about 200MB each, filter them > and > > then I want to output the result into a unique file. > > If I modify the main function to work only with one input file, the > program > > runs without issues. I will call this version A. > > Version B uses a mapM_ to iterate over a list of filenames and uses > > appendFile to output the result of filtering each file. > > In this case the memory usage grows sharply and quickly (profiles show > > constant memory growth). In less than a minute, memory > > occupation will make my system hang with swapping. > > > > This is version B: > > > > ------------------------------- Program B > > > -------------------------------------------------------------------------------------------------------------------- > > import Data.List > > import System.Environment > > import System.Directory > > import Control.Monad > > > > > > -- different types of chromosomes > > data Chromosome = C1 > > | C2 > > | C3 > > | C4 > > | C5 > > | C6 > > | C7 > > | C8 > > | C9 > > | C10 > > | C11 > > | C12 > > | C13 > > | C14 > > | C15 > > | C16 > > | C17 > > | C18 > > | C19 > > | CX > > | CY > > | CMT > > deriving (Show) > > -- define a window > > type Sequence = [Char] > > -- Window data > > data Window = Window { sequen :: Sequence, > > chrom :: Chromosome, > > pos :: Int > > } > > -- print a window > > instance Show Window where > > show w = (sequen w) ++ "\t" ++ show (chrom w) ++ "\t" ++ show (pos > w) > > > > -- Reading fasta files with haskell > > > > -- Initialize the > > main = do > > -- get the arguments (intput is > > [input, output, windowSize] <- getArgs > > -- get directory contents (only names) > > names <- getDirectoryContents input > > -- prepend directory > > let fullNames = filter isFastaFile $ map (\x -> input ++ "/" ++ x) > > names > > let wSize = (read windowSize)::Int > > -- process the directories > > mapM (genomeExecute output wSize filterWindow) fullNames > > > > > > -- read the files one by one and write them to the output file > > genomeExecute :: String -> Int -> (Window -> Bool) -> String -> IO () > > genomeExecute outputFile windowSize f inputFile = do > > fileData <- readFile inputFile > > appendFile outputFile $ fastaExtractor fileData windowSize f > > > > -- > > isFastaFile :: String -> Bool > > isFastaFile fileName = isSuffixOf ".fa" fileName > > > > > > -- fasta extractor (receives a Fasta String and returns a windowed string > > ready to be sorted) > > -- an example on how to compose several functions to parse a fasta file > > fastaExtractor :: String -> Int -> (Window -> Bool) -> String > > fastaExtractor input wSize f = printWindowList $ filter f $ readFasta > wSize > > input > > > > -- MAIN FILTER that removes N elements from the strings! > > filterWindow :: Window -> Bool > > filterWindow w = not (elem 'N' (sequen w)) > > > > -- print a window list (the printing makes it ready for output as raw > data) > > printWindowList :: [Window] -> String > > printWindowList l = unlines $ map show l > > > > -- read fasta, remove stuff that is not useful from it > > -- removes the > > readFasta :: Int -> [Char] -> [Window] > > readFasta windowSize sequence = > > -- get the header > > let (header:rest) = lines sequence > > chr = parseChromosome header > > in > > > > -- We now do the following: > > -- take window create counter > > remove newlines > > map (\(i, w) -> Window w chr i) $ zip [0..] $ slideWindow windowSize > $ > > filter ( '\n' /= ) $ unlines rest > > > > > > slideWindow :: Int -> [Char] -> [[Char]] > > slideWindow _ [] = [] > > slideWindow windowSize l@(_:xs) = take windowSize l : slideWindow > > windowSize xs > > > > > > > > -- Parse the chromosome from a fasta comment > > -- produce a more compact chromosome representation > > parseChromosome :: [Char] -> Chromosome > > parseChromosome line > > | isInfixOf "chromosome 1," line = C1 > > | isInfixOf "chromosome 2," line = C2 > > | isInfixOf "chromosome 3," line = C3 > > | isInfixOf "chromosome 4," line = C4 > > | isInfixOf "chromosome 5," line = C5 > > | isInfixOf "chromosome 6," line = C6 > > | isInfixOf "chromosome 7," line = C7 > > | isInfixOf "chromosome 8," line = C9 > > | isInfixOf "chromosome 10," line = C10 > > | isInfixOf "chromosome 11," line = C11 > > | isInfixOf "chromosome 12," line = C12 > > | isInfixOf "chromosome 13," line = C13 > > | isInfixOf "chromosome 14," line = C14 > > | isInfixOf "chromosome 15," line = C15 > > | isInfixOf "chromosome 16," line = C16 > > | isInfixOf "chromosome 17" line = C17 > > | isInfixOf "chromosome 18" line = C18 > > | isInfixOf "chromosome 19" line = C19 > > | isInfixOf "chromosome X" line = CX > > | isInfixOf "chromosome Y" line = CY > > | isInfixOf "mitochondrion" line = CMT > > | otherwise = error "BAD header" > > > > > > -------------------------------- End of program B > > > ------------------------------------------------------------------------------------------------ > > > > -------------------------------- Program A > > > --------------------------------------------------------------------------------------------------------- > > If instead of the main function given above I use the following main > > function to process only one input file, things work OK for even > > the largest files. Memory usage remains constant in this case. > > > > main = do > > -- get the arguments > > [input, output, windowSize] <- getArgs > > -- keep the input stream > > inpStr <- readFile input > > let wSize = (read windowSize)::Int > > writeFile output $ fastaExtractor inpStr wSize filterWindow > > > > > > It is not easy for me to see why is Haskell keeping data in memory. Do > you > > have any idea why program B is > > not working? > > > > Thank you for your help! > > > > Arnoldo Muller > > > > _______________________________________________ > > Haskell-Cafe mailing list > > Haskell-Cafe(a)haskell.org > > http://www.haskell.org/mailman/listinfo/haskell-cafe > > > > >

2 1

ANNOUNCE: attoparsec-iteratee v0.1
by Gregory Collins 13 Mar '10

13 Mar '10

Hi all, I've just uploaded the first version of the attoparsec-iteratee library to Hackage: http://hackage.haskell.org/package/attoparsec-iteratee It takes applicative parsers written using attoparsec and automagically converts them to iteratees that can parse things from streams in O(1) space. ....and the combination is fast: a prototype HTTP server I'm working on uses this library and can handle as many as 13k reqs/sec on my Macbook. G -- Gregory Collins <greg(a)gregorycollins.net>

2 1

mirroring patch tag
by Thomas Hartman 12 Mar '10

12 Mar '10

http://blog.patch-tag.com/2010/03/13/mirroring-patch-tag/ cheers, thomas.

1 0

generalized newtype deriving allows the definition of otherwise undefinable functions
by Wolfgang Jeltsch 12 Mar '10

12 Mar '10

Hello, some time ago, it was pointed out that generalized newtype deriving could be used to circumvent module borders. Now, I found out that generalized newtype deriving can even be used to define functions that would be impossible to define otherwise. To me, this is surprising since I thought that generalized newtype deriving was only intended to save the programmer from writing boilerplate code, not to extend expressiveness. Have a look at the following code: > {-# LANGUAGE > GeneralizedNewtypeDeriving, > MultiParamTypeClasses, > FlexibleInstances > #-} > > class Iso a b where > > conv :: item a -> item b > > instance Iso a a where > > conv = id > > newtype Wrapped a = Wrap a deriving (Iso a, Show) Now any value whose type contains some type t can be converted into a value of the type that you get if you replace t by Wrap t. Here is some code to demonstrate this for binary operations: > newtype BinOp a = BinOp (a -> a -> a) > > convBinOp :: (a -> a -> a) -> (Wrapped a -> Wrapped a -> Wrapped a) > convBinOp op = let BinOp op' = conv (BinOp op) in op' Now, you can enter convBinOp (*) (Wrap 5) (Wrap 3) into GHCi, and you will get Wrap 15 as the result. The point is, of course, that such conversions are not only possible for binary operations but for arbitrary values and that these conversions are done by a single generic function conv. I don’t think it would be possible to implement conv without generalized newtype deriving. Any thoughts? Best wishes, Wolfgang

11 19