- Haskell-Cafe - Haskell.org

Fwd: [Haskell-cafe] hGetContents and lazyness
by Rafael Gustavo da Cunha Pereira Pinto 22 Sep '08

22 Sep '08

---------- Forwarded message ---------- From: Rafael Gustavo da Cunha Pereira Pinto <rafaelgcpp(a)gmail.com> Date: Mon, Sep 22, 2008 at 15:46 Subject: Re: [Haskell-cafe] hGetContents and lazyness To: Max Vasin <max.vasin(a)gmail.com> Why don't you use OpenFile? > getPackageList packageFile = do > h <- OpenFile packageFile ReadMode > c <- hGetContents h > return $ map (drop 10) $ filter (startsWith "Filename:") $ lines c I am not at home so I don't have GHC around here, but this usually works for me! Anyway, it also depends on who is consuming the IO [FilePath] data returned. It will not be evaluated unless asked for! On Mon, Sep 22, 2008 at 14:52, Max Vasin <max.vasin(a)gmail.com> wrote: > Hello, haskellers! > > Suppose we have function (it filters package filenames from apt Packages > file): > > > getPackageList :: FilePath -> IO [FilePath] > > getPackageList packageFile = withFile packageFile ReadMode $ > > \h -> do c <- hGetContents h > > return $ map (drop 10) $ filter > (startsWith "Filename:") $ lines c -- (1) > > where startsWith [] _ = True > > startsWith _ [] = False > > startsWith (x:xs) (y:ys) | x == y = startsWith xs ys > > | otherwise = False > > When, I apply it to a Packages file I (surely) get an empty list. This is > an expected result due to > lazyness of hGetContents. I tried changing line (1) to > > > return $ map (drop 10) $ filter (startsWith "Filename:") $! lines c > > or > > > return $ map (drop 10) $! filter (startsWith "Filename:") $! lines c > > with no success. > > Chaning it to > > > return $! map (drop 10) $ filter (startsWith "Filename:") $ lines c > > makes getPackageList function return several (but not all) filenames. > > What I'm missing? And how can I fix my code? > > -- > WBR, > Max Vasin. > > _______________________________________________ > Haskell-Cafe mailing list > Haskell-Cafe(a)haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe > -- Rafael Gustavo da Cunha Pereira Pinto Electronic Engineer, MSc. -- Rafael Gustavo da Cunha Pereira Pinto Electronic Engineer, MSc.

1 0

RE: readFile and closing a file
by John Lato 22 Sep '08

22 Sep '08

> On Wed, 17 Sep 2008, Mitchell, Neil wrote: > >> I tend to use openFile, hGetContents, hClose - your initial readFile >> like call should be openFile/hGetContents, which gives you a lazy >> stream, and on a parse error call hClose. > > I could use a function like > withReadFile :: FilePath -> (Handle -> IO a) -> IO a > withReadFile name action = bracket openFile hClose ... > > Then, if 'action' fails, the file can be properly closed. However, there > is still a problem: Say, 'action' is a parser which produces a data > structure lazily. Then further processing of that data structure of type > 'a' may again stop before completing the whole structure, which would also > leave the file open. We have to force users to do all processing within > 'action' and to only return strict values. But how to do this? I used rnf from Control.Parallel.Strategies when dealing with a similar problem. Would it work in your case? To merge discussion from a related thread: IMO, the question is how much should a language/library prevent the user from shooting himself in the foot? The biggest problem with lazy IO, IMO, is that it presents several opportunities to do so. The three biggest causes I've dealt with are handle leaks, insufficiently evaluated data structures, and problems with garbage collection as in the naive 'mean xs = sum xs / length xs' implementation. There are some idioms that can help with the first two cases, namely the 'with*' paradigm and 'rnf', but the third problem requires that the programmer know how stuff works to avoid poor implementations. While that's not bad per se, in some cases I think it's far too easy for the unwitting, or even the slightly distracted, to get caught in traps. I'm facing a design decision ATM related to this. I can use something like lazy bytestrings, in which the chunkiness and laziness is reified into the datastructure, or an Iterator-style fold for consuming data. The advantage of the former approach is that it's well understood by most users and has proven good performance, while on the downside I could see it easily leading to memory exhaustion. I think the problem with lazy bytestrings, in particular, is that the foldChunks is so well hidden from most consumers that it's easy to hold references that prevent consumed chunks from being reclaimed by the GC. When dealing with data in the hundreds of MBs, or GB range, this is a problem. An Enumerator, on the other hand, makes the fold explicit, so users are required to think about the best way to consume data. It's much harder to unintentionally hold references. This is quite appealing. Based on my own tests so far performance is certainly competitive. Assuming a good implementation, handle leaks can also be prevented. On the downside, it's a very poor model if random access is required, and users aren't as familiar with it, in addition to some of the questions Don raises. Back onto the topic at hand - 'action' could be a parser passed into an enumerator. Since it would read strictly, the action could end the read when it has enough data. That's another approach that I think would work with your problem. Well, that's my 2cents. John Lato

2 1

Ropes
by Rafael Gustavo da Cunha Pereira Pinto 22 Sep '08

22 Sep '08

Hi all, Is there any implementation of the rope data structure in Haskell? I couldn't find any on Hackage, and I am intending to implement it. Regards, Rafael Gustavo da Cunha Pereira Pinto Electronic Engineer, MSc.

5 6

Munging wiki articles with tagsoup
by Gwern Branwen 22 Sep '08

22 Sep '08

Hiya Neil. So recently I've been trying to come up with some automated system to turn The Monad Reader articles like those in <http://sneezy.cs.nott.ac.uk/darcs/TMR/Issue11> into wiki-formatted articles for putting on Haskell.org. Thus far, I've had the most success with SVN Pandoc. Pandoc does a good job - you can see an example conversion at <http://haskell.org/haskellwiki/?title=User:Gwern/kenn&oldid=22808>. Modulo the errors which are largely due to haskell.org problems and a few limitations in Pandoc (no comments, no real support for references), it's fine. But Pandoc's author will not support <haskell></haskell> tags inasmuch as they are an extension to MediaWiki and not universal; he prefers <pre> or <pre class="haskell"> tags. He suggested I use TagSoup to convert them into <haskell> tags. Well, alright. They're tags, TagSoup does tags - seems natural. After an hour, I came up with a nice clean little script: ---- import Text.HTML.TagSoup.Render import Text.HTML.TagSoup main :: IO () main = interact convertPre convertPre :: String -> String convertPre = renderTags . map convertToHaskell . canonicalizeTags . parseTags convertToHaskell :: Tag -> Tag convertToHaskell x | isTagOpenName "pre" x = TagOpen "haskell" (extractAttribs x) | isTagCloseName "pre" x = TagClose "haskell" | otherwise = x where extractAttribs :: Tag -> [Attribute] extractAttribs (TagOpen _ y) = y extractAttribs _ = error "The impossible happened." ---- On an aside, may I note that TagSoup doesn't seem to support transformations particularly well? Or if it does, I didn't notice any examples. I spent most of my time just figuring out how to convert the 'x' from a <pre>stuff to <haskell>stuff. Also, it might be nice to define an 'interact' alike, which is (String -> String), and defined, I supposed, as 'interact f = renderTags . f . canonicalizeTags . parseTags'. Extraction functions would be good as well - you'd only need 3 groups, I think; 1 for the 2 items in TagOpen, 1 for TagPosition's 2 positions, and 1 which extracts the String from the rest. Anyway, so my script seems to work. I ran the wiki output through it and this is the diff: <http://haskell.org/haskellwiki/?title=User%3AGwern%2Fkenn&diff=22827&oldid=…>. Ok, good, it replaces all the tags... But wait, what's all this other stuff? It is replacing all my apostrophes with '! No doubt this has something to do with XML/HTML/SGML or whatever, but it's not ideal. Even if it doesn't break the formatting (as I think it does), it's still cluttering up the source. So, how can I fix this? Am I just barking up the wrong tree and should be writing a simple-minded search-and-replace sed script which replaces <pre> with <haskell>, </pre> with </haskell>...? -- gwern USS Enforcers SORO Morwenstow MOD Albright MI5 AOL 701 GCHQ

2 3

a question about concurrent haskell
by Manlio Perillo 22 Sep '08

22 Sep '08

Hi. I have a question about concurrent Haskell in GHC. Suppose I want to write a native pure Haskell PostgreSQL client. I have done this for Python (using Twisted): http://hg.mperillo.ath.cx/twisted/pglib/ and I would like to do this in Haskell, as an exercise. The main problem is with multiple concurrent queries to the same connection, from multiple threads. PostgreSQL supports multiple requests but it's better to execute only one request at a time: http://www.postgresql.org/docs/8.3/interactive/protocol-flow.html#AEN73647 In the Twisted version I queue all the requests, and every time a request completes I call the callback associated with the request and execute the next available request. But how do I solve this problem with concurrent Haskell? Suppose thread A execute a query, and while this query is still active, a thread B execute another query. I should suspend thread B (calling yield), but how do I resume it when the query executed by thread A completes? The GHC concurrent Haskell does not have a function to resume a given thread (as found, as an example, in Lua). P.S.: another question. Why, in ghci, every time I call myThreadId, I get a different value? Prelude Control.Concurrent> myThreadId ThreadId 40 Prelude Control.Concurrent> myThreadId ThreadId 41 Thanks Manlio Perillo

5 5

readFile and closing a file
by Henning Thielemann 22 Sep '08

22 Sep '08

Say I acquire a text by readFile, feed it to a lazy parser and the parser stops reading because of a parse error. After the parser error I won't fetch more characters from the text file, but readFile does not get to know that and cannot close the file. Actually, when encountering a parser error I could read the remaining characters from the file in order to let readFile close the file eventually, but this sounds rather inefficient. So, what is the right way to go, or do I have to stay away from readFile (and maybe unsafeInterleaveIO at all)?

4 6

ANNOUNCE: citeproc-hs, a Haskell implementation of the Citation Style Language designed for Pandoc
by Andrea Rossato 22 Sep '08

22 Sep '08

Hello, I'm happy to announce the first release of citeproc-hs, a Haskell implementation of the Citation Style Language. citeproc-hs adds to Pandoc, the famous Haskell text processing tool, a Bibtex like citation and bibliographic formatting and generation facility. ABOUT The Citation Style Language (CSL) is an XML language for specifying citation and bibliographic formatting, similar in principle to BibTeX .bst files or the binary style files in commercial products like Endnote or Reference Manager. CSL is used by Zotero for bibliographic style formatting, and a huge number of CSL styles have been developed by the Zotero community. There are plans to use CSL in the future release of OpenOffice: http://bibliographic.openoffice.org/ citeproc-hs is a library that exports functions to parse CSL styles and MODS collections, to process lists of citation groups and to format the processed output. The output is a Haskell data type that can be further processed for conversion to any kind of formats (at the present time plain ASCII and the Pandoc native format) citeproc-hs was developed in order to add to Pandoc Bibtex like citations and automatic reference and bibliography generation. More information, with installation instructions, can be found here: http://code.haskell.org/citeproc-hs/ DOWNLOADS citeproc-hs can be downloaded from Hackage: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/citeproc-hs To get the darcs source run: darcs get http://code.haskell.org/citeproc-hs/ KNOWN ISSUES citeproc-hs is in an early stage of development and the CSL is not complete yet. Specifically, citation collapsing is not implemented, and some formatting options are not working neither. The MODS parser needs some refinement too. BUG REPORTS To submit bug reports you can you the Google code bug tracking system available at the following address: http://code.google.com/p/citeproc-hs/issues CREDITS Bruce D'Arcus, the author of CSL, has been very kind and patient with me when I was trying to understand the CSL schema, and provided me with ideas, comments and suggestions that made it possible to come to something usable. John MacFarlane, the author of Pandoc, has been very supportive of the project and provided a lot of useful feed back, comments and suggestions. Hope you'll enjoy, Andrea Rossato

2 2

Re: [Haskell-cafe] Line noise
by Richard A. O'Keefe 22 Sep '08

22 Sep '08

On 22 Sep 2008, at 9:19 am, Albert Y. C. Lai wrote: > Everyone should use exclusively ancient Egyptian iconography. It is > the only universally readable language. I know this was full of (:-) (:-), but it seems like the perfect chance to mention Blissymbols. http://unicode.org/roadmaps/smp/smp-4-14.html (the Unicode SMP road map) has a link to a proposal for adding them to Unicode, and they _are_ in active use... (A Haskell program in Blissymbols would be, well, bigger.)

1 0

Bird-style and blank lines
by Paulo Tanimoto 22 Sep '08

22 Sep '08

Hi, I'm a big fan of literate haskell, especially Bird-style, but there's one behavior that I find rather annoying: the requirement that code be preceded and followed by a blank line. Quoth the Report: "To capture some cases where one omits an ">" by mistake, it is an error for a program line to appear adjacent to a non-blank comment line, where a line is taken as blank if it consists only of whitespace." While that is something I do anyway, since it looks better, there are some cases where it would be nice to relax that. For example: 1. I see a blog post in literate style, but the author did not provide a .lhs file, sometimes copying and pasting the text will result in code being adjacent to the text. Then I have to fix that manually. 2. It would be nice for prototyping if you could quickly comment out code just by removing a ">" character, instead of inserting "--". 3. Pandoc, via markdown, is a very interesting alternative to latex. Right now you can write a block of code like this: ~~~ > fact :: Integer -> Integer > fact 0 = 1 > fact n = n * fact (n-1) ~~~ except that that won't work, precisely because you must have a blank line before and after the code. Unfortunately, when you go and add the blank lines, the output will show them. You may tell me to change what Pandoc does, but my point is that getting any markup to work with Bird-style will have this issue. The only argument I can see for keeping the requirement is what the report says, but feel free to provide others [1]. However, isn't that a bit like saying "we will require two semicolons because forgetting a semicolon is the number one source of mistake for our programmers"? Besides, I suspect most of us uses an editor with syntax highlighting (vi, emacs, yi, etc), and then it's immediately obvious you forgot something. Would it be hard to change this behavior in GHC? Is this open for discussion for Haskell Prime? Thanks, Paulo [1] I don't think it's that common to add a ">" by accident.

3 4

Language.Haskell and strings
by Maurício 22 Sep '08

22 Sep '08

Hi, I'm using Language.Haskell.* and would like to know if it's possible to pretty-print big strings like this: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" into something like this: "aaaaaaaa\ \aaaaaaaa\ \aaaaaaaa\ \aaaaaaaa\ \aaaaaaaa\ \aaaaaaaa\ \aaaaaaaa" to respect the limit on number of columns. Can you help me? Is it possible to do that? Thanks, Maurício

3 3