- Haskell-Cafe - Haskell.org

How to make asynchronous I/O composable and safe?
by Joey Adams 18 Jan '12

18 Jan '12

I'm not happy with asynchronous I/O in Haskell. It's hard to reason about, and doesn't compose well. At least in my code. I'm currently trying to build a networking layer for my application using Network.TLS. Here is a rather minimalist API: newtype Connection = Connection (TLSCtx Handle) connectClient :: Handle -- ^ Connection handle, as returned by 'connectTo' -> X509 -- ^ TLS certificate (i.e. public key) -> IO Connection connectServer :: Handle -- ^ Connection handle, as returned by 'accept' -> X509 -- ^ TLS certificate (i.e. public key) -> TLS.PrivateKey -- ^ TLS private key -> IO Connection close :: Connection -> IO () sendMessage :: Connection -> Message -> IO () recvMessage :: Connection -> ByteString -> IO (Message, ByteString) The module provides little more than connection initialization and message serialization. I don't try to use locks or STM to multiplex the connection or, in the case of recvMessage, hide connection state. I just be sure to only use sendMessage in one thread at a time, only use recvMessage in one thread at a time, and marshal the "extra bytes" parameter of recvMessage from call to call (with the help of StateT). I wrote a simple "chat server" to test it. The client turned out okay: main :: IO () main = do cert <- getCertificate handle <- connectTo "localhost" (PortNumber 1337) conn <- connectClient handle cert _ <- forkIO $ forever $ do s <- getLine sendMessage conn $ TestMessage s forever $ flip runStateT B.empty $ do msg <- StateT $ recvMessage conn case msg of TestMessage s -> liftIO $ putStrLn s _ -> liftIO $ hPrintf stderr "Warning: unrecognized message from server: %s\n" (messageTypeName msg) The only glaring problem is that, if the user presses Ctrl+D, the forked (sending) thread dies, but the main (receiving) thread lingers. I'd have to add exception handlers to ensure that when one thread dies, the other thread dies too. However, the server is an abomination (see attachment). Unfortunately, it's not as simple as "spawn one thread per client". We need at least two threads, one to listen for messages from the client, and another to send messages to the client. GHC won't let us simultaneously, in the same thread, wait for input from a connection and wait for an STM transaction to succeed. Another source of complexity is: what if we throw an exception at a thread while it is in the middle of sending a packet? Then we can't shut down the connection properly (i.e. Network.TLS.bye), because the receiver might think the close_notify packet is part of the interrupted packet. Having a thread for each client is good, as it: * Lets us think about each client separately. No need to turn our code inside out or write one big loop that juggles all the clients. * Isolates exceptions. If sendMessage or recvMessage throws an exception, it doesn't bring the whole server down. On the other hand, having multiple threads interact with a single client is hard to think about: * We have to synchronize the threads (e.g. when one dies, kill the other one) * Multiple places where an exception can arise * Can multiple threads interact with the connection handle simultaneously? So why don't I make my connection API handle some of this? Well, I tried. There are so many ways to do it, and I couldn't find a way that simplified usage much. The approach used by Handle and by Network.TLS is to use MVars and IORefs to ensure that, if two threads access the same connection, the connection doesn't become totally corrupt. If I do the same, then I'll have *three* layers of locking under the hood. Worse, the locking done by Handle and Network.TLS doesn't guarantee much. I don't know if it's safe to have one thread sending and another thread receiving. Especially in the case of Network.TLS, where 'recvData' automatically handshakes in some cases, which sends packets. Since I don't know how much thread safety to expect, I can't write networking code and know for sure that it is safe. I'm certainly not protected from interleaved data if multiple threads send on the same handle. For example: import Control.Concurrent import System.IO main :: IO () main = do hSetBuffering stdout NoBuffering _ <- forkIO $ putStrLn "One sentence." putStrLn "Another sentence." produces: AnOonteh esre nsteenntceen.c e. That is, I can't rely on putStrLn being "atomic". To produce intelligible output (without changing the buffering mode), I have to "lock" the output each time I write something. putStrLn doesn't do it for me. === Summary === In Haskell, sound logic and a great type system lead to elegant, composable code in a variety of domains, such as: * Expression evaluation * Parsing * Concurrent programming (thanks to STM) Asynchronous I/O is tricky. However, Haskell currently does little to alleviate the complexity (at least for me). How can we structure network protocol APIs so that they stack well (e.g. only lock once, rather than locking each layer's connection state)? How can we deal with I/O errors without having to think about them at every turn? For now, how can I structure my application's communication API so it's less messy to use? Thanks, - Joey

7 14

Book?
by Gregory Guthrie 18 Jan '12

18 Jan '12

I've seen a book: "The Practice Of Monadic Interpretation" Dan Popa Nov. 2008 Or "Practical Monadic Interpretation" Dan Popa Which seem that they might be the same book? As reported on Haskell Wiki/books as published in 2008, but Don't find it available anywhere under either title. Any pointers or references?

2 1

Need advice: Haskell in Web Client
by dokondr 18 Jan '12

18 Jan '12

Hi all, I hope to use Haskell for graphics (charts) programming in Web client. My current implementation in brief: Server side, Haskell modules: 1) collecting various statistics from Twitter 2) generating text data for Gnuplot (http://www.gnuplot.info/) 3) Gnuplot creates png files with charts Web client: GWT (Google Web Toolkit) web UI that allows user to enter queries and see resulting charts in Web browser. Charts are png files generated by Gnuplot on the server side. Ideally, on the server side instead of using Gnuplot I would like Haskell to generate Javascript to be downloaded to Web client and draw charts in the browser. Something, very approximately, similar to what GWT does :) This code will be specific of course to plotting Javascript framework, no problem. Looking at "Haskell in web browser - HaskellWiki:" http://www.haskell.org/haskellwiki/Haskell_in_web_browser#Haskell_web_toolk… I feel a little confused about current state of the available Haskell tools for this task. Any ideas? Thanks! Dmitri

3 2

Please upgrade http-enumerator and/or http-conduit
by Michael Snoyman 18 Jan '12

18 Jan '12

Hi all, Thanks to Felipe for catching this issue[1]. Both http-enumerator and http-conduit will not properly reject invalid certificates. I've released updated versions of both packages: http-enumerator-0.7.2.4 http-conduit-1.1.2.2 All users are recommended to upgrade. Michael [1] https://github.com/snoyberg/http-conduit/pull/6

1 0

Pruned Sparks in Original Strategies
by Burak Ekici 17 Jan '12

17 Jan '12

Dear List, I am trying to parallelize some number of codes with using Haskell's original and second generation strategies in order to be able to compare them. As far as I understood, in original strategies, ROOT garbage collection mechanism is used which does not allow any spark to be pruned. Since, they do not return any information back to its caller. (a type of void reference making fizzling of a spark impossible, is returned), However, in most of my measurements, spraks are being pruned. I wonder the reasons why this incident occurs? Do you have any idea? Many thanks, Burak.

1 0

__GLASGOW_HASKELL__ for GHC 7.4
by Eugene Kirpichov 17 Jan '12

17 Jan '12

Hi, I'm fixing a build error in a package that depends on the RTS API, which changed in 7.4, using #if __GLASGOW_HASKELL__ >= 740. However, build log shows that __GLASGOW_HASKELL__ is still 704, not 740 as I'd expect. Is this a bug? -- Eugene Kirpichov Principal Engineer, Mirantis Inc. http://www.mirantis.com/ Editor, http://fprog.ru/

2 2

Re: [Haskell-cafe] decoupling type classes
by Yin Wang 17 Jan '12

17 Jan '12

On Sat, Jan 14, 2012 at 2:38 PM, Dominique Devriese <dominique.devriese(a)cs.kuleuven.be> wrote: >> I may or may not have thought about it. Maybe you can give an example >> of parametric instances where there could be problems, so that I can >> figure out whether my system works on the example or not. > > The typical example would be > > instance Eq a => Eq [a] where > [] == [] = True > (a : as) == (b : bs) = a == b && as == bs > _ == _ = False It can handle this case, although it doesn't handle it as a parametric instance. I suspect that we don't need the concept of "parameter instances" at all. We just searches for instances recursively at the call site: 1. If "g" has an implicit parameter "f", search for values which matches the name and instantiated type in the current scope. 2. If a value is found, use it as the argument. 3. Check if the value is a function with implicit parameters, if so, search for values that matches the name and type of the implicit parameters. 4. Do this recursively until no more arguments contain implicit parameters. > This coupling you talk about is not actually there for instance > arguments. Instance arguments are perfectly usable without records. > There is some special support for automatically constructing record > projections with instance arguments though. Cool. So it seems to be close to what I had in mind. > I am not sure about the exact workings of your system, but I want to > point out that alternative choices can be made about the workings of > inferencing and resolving type-class instances such that local > instances can be allowed. For example, in Agda, we do not infer > instance arguments and we give an error in case of ambiguity, but > because of this, we can allow local instances... Certainly it should report error when there are ambiguities, but sometimes it should report an error even there is only one value that matches the name and type. For example, foo x = let overload bar (x:Int) = x + 1 in \() -> bar x baz = in foo (1::Int) Even if we have only one definition of "bar" in the program, we should not resolve it to the definition of "bar" inside foo. Because that "bar" is not visible at the call site "foo (1::int)". We should report an error in this case. Think of "bar" as a "typed dynamically scoped variable" helps to justify this decision.

2 3

fixity declaration in GHCi
by Takayuki Muranushi 17 Jan '12

17 Jan '12

Hello everyone, while I'm playing with fixity declaration in GHCi, I found a strange behavior. Can anyone explain this? Is this actually an expected behavior, or is it a glitch? ---------------------------------------------------------------- $ ghci GHCi, version 7.0.3: http://www.haskell.org/ghc/ :? for help Loading package ghc-prim ... linking ... done. Loading package integer-gmp ... linking ... done. Loading package base ... linking ... done. Loading package ffi-1.0 ... linking ... done. Prelude> minus --(1) <interactive>:1:1: Not in scope: `minus' Prelude> let minus x y = x-y; infixl 6 `minus` --(2) Prelude> 10^5 `minus` 2 --(3) 1000 Prelude> let infixl 6 `minus` --(4) <interactive>:1:14: The fixity declaration for `minus' lacks an accompanying binding Prelude> let subtract x y = x-y --(5) Prelude> 10^5 `subtract` 2 --(6) 1000 Prelude> let f x y = x-y; infixl 6 `f` in 10^5 `f` 2 --(7) 99998 Prelude> ---------------------------------------------------------------- (1) suggests that `minus` is not defined here. in (2), I try to define `minus` with precedence level 6. However, (3) suggests that precedence level of `minus` is stronger than ^ . (4) suggests that you cannot redefine fixity here. (5)(6) is consistent with Haskell 98 report section 4.4.2; "Any operator lacking a fixity declaration is assumed to be infixl 9." in (7), precedence level is set as expected. So, what is happening in (3)? Where did the fixity declaration in (2) went? Best, -- MURANUSHI Takayuki The Hakubi Center, Kyoto University : http://www.hakubi.kyoto-u.ac.jp/

1 0

Network.Browser and Network.TLS
by Myles C. Maxfield 17 Jan '12

17 Jan '12

Hello! I am interested in extending the Network.HTTP code in the HTTP package to support HTTPS via TLS. A clear candidate is to use the Network.TLS module in the TLS library (because its TLS logic is written in pure Haskell, rather than any of the FFI libraries like Network.Curl or the OpenSSL package). It's simple enough to provide an implementation of the Network.Stream.Stream typeclass around a TLSCtx, and this works for the Network.HTTP.Stream functions. However, I am interested in using the functionality in the Network.Browser module. This module uses the Network.HTTP.HandleStream interface, which is implemented directly on top of the Handle datatype and the Network.BufferType.BufferOp functions. HandleStreams seem to be for allowing the user to pull out an arbitrary data type out of an HTTP stream, not for doing any stream processing the way TLS does. As far as I can tell, the current typeclass system does not allow a TLSCtx to piggypack off of a HandleStream. My assumption is that this interface is used for speed so the user doesn't have to convert some canonical type into the type that he/she desires in client code. TLS, however, must use a specific type to decode the bytes that it pulls out of the stream. I don't think it's reasonable to try to modify the TLS library to decode bytes from an arbitrary type. Decoding necessarily needs byte-level access to its input (and therefore output) streams in a manner extremely similar to the functions that ByteString provides. Perhaps I'm wrong about this, but the conclusion I've reached is that it doesn't make sense for TLS to use an arbitrary typeclass because the interface it requires is so similar to the existing ByteString datatype. If an application wants a specific type out of a TLS stream, it must necessarily convert the type in software. Any speed that might be gained by pulling your native type out of a network connection will be dwarfed anyway by the cost of decryption. The Network.Stream functions allow this by using String type for all data transfers (which is counterintuitive for binary data transfers). An implementation of Network.Stream.Stream using TLS would convert TLS's output ByteString into a String (possibly by doing something like ((map (toEnum . fromIntegral)) . unpack) which doesn't make a whole lot of sense and is fairly wasteful). A client program might even convert it back to a ByteString, so the client program must have knowledge about how the bytes are packed into the String. Network.Browser only seems to have one function which isn't simply a state accessor/mutator: 'request'. This function gives the connection a type of HStream ty => HandleStream ty. As stated before, the HandleStream directly uses the Handle type. This means that, as far as I can tell, there is no way to fit TLS into the Network.Browser module as it stands because the types don't allow for it. Supporting TLS in Network.Browser would have to change the type of 'request' and therefore break every program out there which uses the Network.Browser module. It would be possible to create something like a 'requestHTTPS' function which returns a different type, but this is quite inelegant - there should be one function that inspects the scheme of the URI it is handed. I am left with the conclusion that it is impossible to support TLS in Network.Browser without breaking many Haskell programs. It is obviously possible to fork the HTTP library, but I'd like to improve the state of the existing libraries. Likewise, it is possible to create a new module that supports HTTPS but has different typed functions and lots of code duplication with Network.Browser, but that is quite inelegant. I suppose this is mostly directed at the maintainers of the HTTP and TLS libraries, Sigbjorn Finne and Vincent Hanquez, but I'd be greatful for your input on what I can do to contribute to the Haskell community regarding Network.Browser and Network.TLS. Perhaps I should just use the Network.HTTP.Enumerator module and not deal with Network.Browser? Maybe I'm going about this in entirely the wrong way. Thanks, Myles C. Maxfield

4 3

Happy holidays Haskell
by R J 17 Jan '12

17 Jan '12

hey Haskell I didn't believe this at all until I started http://www.news13wise.com

1 0