August 2008 - Haskell-Cafe

more adjunction questions -- where to send them?
by Jason Dusek 06 Aug '08

06 Aug '08

I've gotten most of my category theory help through the Haskell community one way or another, but I hesitate to ask more questions on this list if there is a better forum. Googling turned up an old, closed list. Is there somewhere else I should be asking my questions -- for example, I am now working on the adjunction of "right product" and "exponentiation" -- or is the Haskell Cafe list a good place? -- _jsn

1 0

code review? store server, 220loc.
by Tim Newsham 05 Aug '08

05 Aug '08

Anyone interested in critiquing some code? I'm looking for ideas for making it faster and/or simpler: http://www.thenewsh.com/%7Enewsham/store/Server5.hs This is an exercise to see how well a server in Haskell would perform. My goals are roughly: - retargetability to other server types (ie. easy to replace request and response structures and business logic). - readability. - performance. My measurements show that a simple dummy server (accept, forkio, recv byte) handles roughly 7500 requests/connects per second, the server/client that do real messages do about 4500 req and connections per second. If all requests are on the same connection one after another it does about 13500 requests/second. For comparisons, a C ping-pong server does about 3600/second if it has to fork for each new connection/request, and about 35000/sec if its all on the same connection. So it seems at least competitive with a forking C server. I havent tested threaded C servers. Tim Newsham http://www.thenewsh.com/~newsham/

5 8

Stripping text of xml tags and special symbols
by Pieter Laeremans 05 Aug '08

05 Aug '08

Hi, I 've got a lot of files which I need to proces in order to make them indexable by sphinx. The files contain the data of a website with a custom perl based cms. Unfortunatly they sometimes contain xml/html tags like <i> And since most of the texts are in dutch and some are in French they also contain a lot of special characters like ë é, ... I'm trying to replace the custom based perl based cms by a haskell one. And I would like to add search capability. Since someone wrote sphinx bindings a few weeks ago I thought I try that. But transforming the files in something that sphinx seems a challenge. Most special character problems seem to go aways when I use encodeString (Codec.Binary.UTF8.String) on the indexable data. But the sphinx indexer complains that the xml isn't valid. When I look at the errors this seems due to some documents containing not well formed html. I would like to use a programmatic solution to this problem. And is there some haskell function which converts special tokens lik & -> & and é -> &egu; ? thanks in advance, Pieter -- Pieter Laeremans <pieter(a)laeremans.org> "The future is here. It's just not evenly distributed yet." W. Gibson

3 2

Brainstorming on how to parse IMAP
by John Goerzen 05 Aug '08

05 Aug '08

Hi folks, I'm interested in writing a library to work with IMAP servers. I'm interested in thoughts people have on parsing libraries and methods. I'm a huge fan of Parsec overall -- it lets me have a single-stage parser, for instance. But it isn't sufficiently lazy for this task, and I probably will need to deal with ByteStrings instead of Strings, since some IMAP messages may be 30MB or more. So to give a very, very brief rundown of RFC3501, there are lots of ways that an IMAP server can encode things. For instance, we could see this: A283 SEARCH "TEXT" "string not in mailbox" which is the same as: A283 SEARCH TEXT "string not in mailbox" and the same as: A283 SEARCH {4} "string not in mailbox" TEXT The braces mean that the given number of octets follows after the CRLF at the end of the given line. We could even see: A283 SEARCH {4} {21} TEXTstring not in mailbox Note that when downloading messages, I would fully expect to see things like * FETCH {10485760} representing a 10MB message. Also, quoted strings have escaping rules. [ please note that the above is paraphrased and isn't really true RFC3501 for simplicity sake ] Now then... some goals. 1) Ideally I could parse stuff lazily. I have tried this with FTP and it is more complex than it seems at first, due to making sure you never, never, never consume too much data. But being able to parse lazily would make it so incredibly easy to issue a command saying "download all new mail", and things get written to disk as they come in, with no buffer at all. 2) Avoiding Strings wherever possible. 3) Avoiding complex buffering schemes where I have to manually buffer data packets. Thoughts and ideas? BTW, if any of you have heard of OfflineIMAP, yes I am considering rewriting OfflineIMAP in Haskell. -- John

7 11

Capitalization and associated type families
by Brian Hulley 05 Aug '08

05 Aug '08

Hi, I'm familiar with the capitalization rules for identifiers in Haskell and know that they are very useful and practical e.g. by allowing variables to be distinguished from "constants" in patterns etc. However I'm trying to understand on a much deeper level whether or not they can really be justified and indeed if there is a danger that they can lead to inconsistency in the light of future extensions to the language. For example, why is there any distinction between a type "variable" and a type "constant"? To make this clearer consider the type of (map): (a -> b) -> List a -> List b (ignoring the special [] syntax for list types which would just confuse the issue here). With explicit quantifiers, we'd write: forall a b. (a -> b) -> List a -> List b Now this begs the question: why does "List" need to start with a capital? (supposing that quantifiers were always written explicitly) The same distinction between "constants" and "variables" is also used in predicate logic e.g. On(x, Table) & Colour(x, Red) but similarly the above could surely be written just as easily by: exists x. on(x, table) & colour(x, red) where (on), (table), (colour), and (red) are just considered to be variables that have already been introduced (by some enclosing scope) instead of having to think of them as "constants" and make a distinction between "constant" and "variable". Anyway to get to the point of this post what I'm really trying to find is some explanation of why there is the concept of "constant" as opposed to "variable" or whether this concept is now redundant in the light of lexical scoping/ explicit quantification? Somewhat related to this issue is the syntax of associated type families. For example in section 5.2.2 of http://www.haskell.org/haskellwiki/GHC/Indexed_types there is the example: class C a b where data T a which raises in my mind several questions: 1) Why is (T) given a parameter? I.e. why does an associated type declared inside a class decl not automatically get all the parameters of the class passed to it so that one would just write: class C a b where data T f :: T -> T === data family T a b class C a b where ... f :: T a b -> T a b 2) Compare: class C a b t | a b -> t with class C a b where data T In other words, why is (T) a "constant" when it is declared inside the class but a "variable" when passed as a parameter? Is the fact that (T) needs to be given a parameter even when it is declared inside the "scope" of the class type parameters the result of needing to fulfill the idea of "constant" vs "variable"? 3) In the syntax: class C a b data T a it seems rather odd that the "a" in "T a" should be related to the "a" in "C a b" since one might expect that the name chosen for a parameter should be local to the data decl. Apologies for such a rambling post. I've spent literally at least 6 months scouring the internet trying to find a real discussion about capitalization rules because I'm trying to answer the following question: if I make use of a similar capitalization rule in my own language, can I guarantee that this rule will not lead to an inconsistency at some future point? Ie is there a real rock solid theoretical basis for making a distinction between "constants" and "variables" that holds even in the presence of lexical scoping? Thanks, Brian.

4 7

Re: [Haskell-cafe] Brainstorming on how to parse IMAP
by Donn Cave 05 Aug '08

05 Aug '08

Quoth John Goerzen <jgoerzen(a)complete.org>: | Donn Cave wrote: ... |> So I would let the application come up with the data. In general, | | Well, your response here begs the question of how much you want to | automate from the application. Yes, there are multiple ways of | communicating with IMAP servers, but you have the same synchronization | issues with all of them. Yes, I plan to let the application supply | functions to read data. But in the end, that is pointless if those | functions can't be written in Haskell! Sorry, I didn't entirely understand that, but the way I see it, you might or might not be able to map the set of issues with one transport vs. another onto some common abstraction. And anyway it's much easier and cleaner for you to implement the protocol parser as a pure function of bytestring -> response, than tangle it up with I/O. OK, here's a trivial example of what I see as the virtues of this approach. I don't think I actually return any size information along with my `incomplete' failure response, but it occurs to me that I should. Then a graphic application can use that to start a progress indicator for large transfers. It will be easy for my application to update that progress indicator, because it got the expected size and is getting the data. Harder to make that work through generic I/O functions invoked inside the parser, I think. | > I don't think there's any way to specify how much to read - I mean, | > the counted literal certainly provides that information, but that's the | > exception - so I would assume the application will need some kind of | > recv(2)-like function that reads data as available. | | Exactly. But there is no recv()-like function, except the one that | returns IO String. There is no recv()-like function that returns IO | ByteString. Well, right, that's needed. I just hacked out my own FFI. (Likewise for SSL.) In a platform context where you're forced to use buffered input semantics, you might use hGetLine and then read fixed amounts as required for counted literals (so the parser would need to return a bytes-needed-to-complete value to support that.) Donn

1 0

Re: [Haskell-cafe] Brainstorming on how to parse IMAP
by Donn Cave 05 Aug '08

05 Aug '08

Quoth John Goerzen <jgoerzen(a)complete.org>: ... | One problem with that is that if I use specific parsing library foo, | then only others that are familiar with specific parsing library foo can | hack on it. Well, you asked. For me, the parsing is a relatively minor part of the problem and maybe not one that benefits greatly from external support. MIME might be a different story. | 2) A lot of RFC protocols -- and IMAP in particular -- can involve | complex responses from the server with hierarchical data, and the parse | of, say, line 1 and of each successive line can indicate whether or not | to read more data from the server. Parsing of these lines is a stateful | activity. I mentioned that my parser may return an incomplete status. In principle, something like parseResponse :: ByteString -> Maybe (IMAPResponse, ByteString) That means the parse needs to be repeated each time until enough input data has accumulated. I worried a little about how to represent a useful incomplete parse state, but decided that it isn't worth the trouble - the amount of parsing in an ordinary response is too trivial. | 3) The linkage between Parsec and IO is weak. I cannot write an | "IMAPResponse" parser. I would have a write a set of parsers to parse | individual components of the IMAP response as part of the IO monad code | that reads the IMAP response, since the result of one dictates how much | network data I attempt to read. The parser should just parse data, and not read it. You don't need to worry about whether you can get recv(2) semantics on a socket with bytestrings, and you don't need to saddle users of this parser with whatever choice you might make there. You don't need to supply an SSL input function with the same semantics, or account for the possibility that data might actually not be coming from a socket at all (UNIX pipes are not unheard of.) You don't need to lock users into whatever execution dispatching might be supported by that I/O, potentially ruling out graphics libraries etc. that might not be compatible. So I would let the application come up with the data. In general, I don't think there's any way to specify how much to read - I mean, the counted literal certainly provides that information, but that's the exception - so I would assume the application will need some kind of recv(2)-like function that reads data as available. By the way, if you haven't already run across this, you may be interested to read about the IMAP "IDLE" command, cf. RFC2177. I think the value of this feature can be overstated, but it depends on the server, and some IMAP client implementators are very fond of it. At this point, the reason it might be interesting is that it moves away from the call & response pattern. Donn

2 1

ActiveX and COM
by Günther Schmidt 05 Aug '08

05 Aug '08

Hi, what is currently the recommend way to interface with COM from haskell? I need to create an Excel Sheet and thus need to use Excel via COM. Günther

4 4

Re: [Haskell-cafe] array fusion interface
by Henning Thielemann 05 Aug '08

05 Aug '08

On Sat, 2 Aug 2008, Evan Laforge wrote: > >> I have streams of samples with irregular sampling rates, so they look > >> like [(Time, SampleVal)]. In practice, this means [(Double, Double)]. > > > > Maybe I have already mentioned my eventlist package on Hackage which > > supports such resampling operations - but is based on lists. > > http://hackage.haskell.org/cgi-bin/hackage-scripts/package/event-list > > Ah, I had seen event-list and even downloaded and read parts of it, > but it was hard for me to figure out what was going on and how to > actually use it based on the scattered comments in the source. I commented only non-trivial parts, because I thought it is clear, that 'mapTime' is a 'map' on the time stamps and so on. > Apparently there are absolute and relative time variants, but within > that there are further "body" and "time" variants, and they're built > on a similarly sparsely documented alternating list library. Are > there some simple examples of its use out there? There are all four variants of whether a list starts and ends with a time or a data body. The 'midi' package uses 'event-list'. > Unfortunately, storable vector, unlike ByteString, doesn't export > 'create' and whatnot so I have to keep a locally patched version of > it. Well, it does export 'create': http://hackage.haskell.org/packages/archive/storablevector/0.1.2.2/doc/html… However in my applications I could use the safe 'unfoldrN' in most cases.

1 0

timing question
by Arie Groeneveld 05 Aug '08

05 Aug '08

Suppose I've: f = map g I want to know how much time it takes (interpreted mode) to fully process list xs (at least 1e6 elements) with function g. Is it sufficient to execute: *Main> last . f $ xs <result> (x.xx secs, yyyyyyyyyyy bytes) Are there any hidden difficulties involved? Reason is: comparing timings Haskell vs an interpreted language without laziness. thanks =@@i

5 7