- web-devel - Haskell.org

ResponseEnumerator
by Kazu Yamamoto 13 Dec '11

13 Dec '11

Hello, I have a question about WAI. ResponseEnumerator is defined as: type ResponseEnumerator a = (Status -> ResponseHeaders -> Iteratee Builder IO a) -> IO a Are there any reasons to use Builder instead of ByteString. I could not find code which makes use of Iteratee Builder IO a in Yesod. Michael posted the following code before. This code converts ByteString to Builder. But Builder is converted to ByteString in Warp. It seems to me that "Iteratee ByteString IO a" is more reasonable than "Iteratee ByteString IO a". Am I missing something? --Kazu ---- data Proxy = Proxy mkYesod "Proxy" [parseRoutes| / RootR GET |] instance Yesod Proxy where approot _ = "" getRootR :: GHandler Proxy Proxy () getRootR = do req <- liftIO $ parseUrl "http://www.yesodweb.com/" sendWaiResponse $ ResponseEnumerator $ \f -> withManager $ \m -> run_ (http req (blaze f) m) blaze :: (Status -> ResponseHeaders -> Iteratee Builder IO a) -> Status -> ResponseHeaders -> Iteratee ByteString IO a blaze f s h = joinI $ EL.map fromByteString $$ f s h' where h' = filter go h go ("Content-Encoding", _) = False go _ = True main :: IO () main = warpDebug 3000 Proxy ----

3 6

Any examples of using RequestBodyBuilder from http-enumerator
by Erik de Castro Lopo 09 Dec '11

09 Dec '11

Hi all, I'm a bit stumped on how to use the RequestBodyBuilder constructor: http://hackage.haskell.org/packages/archive/http-enumerator/latest/doc/html… Is anyone able to point me at some code that uses this? Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/

3 5

Advice on implementing a web proxy
by Erik de Castro Lopo 08 Dec '11

08 Dec '11

Hi all, I'm working on a simple web proxy. I have the proxying of HTTP working correctly (as least as far as I have tested it) and would like to work on proxying HTTPS. The way HTTPS proxying works is as follows: a) Client sends "CONNECT host:port HTTP/1.1" to the proxy in clear text. b) Proxy makes a connection to host:port and if successful sends "HTTP/1.0 200 Connection established" to the client. c) The proxy then blindly transfers bytes from the client to the server and bytes from server to the client. d) The client does TLS negotiation over the bi-directional pipe established and maintained by the proxy. The git repo containing the code for my proxy is here: https://github.com/erikd/simple-web-proxy and the core of the actual proxy is here: https://github.com/erikd/simple-web-proxy/blob/master/src/simple-web-proxy.… The proxying function should have a type signature of: sslConnectRequest :: ByteString -> Int -> Wai.Request -> Proxy Wai.Response where the ByteString contains the host name and the Int the port number. My plans for the sslConnectRequest function is for it to open a socket connection to the server and then wrap that socket inside and enumerator. Is that a reasonable plan? Is there a better way? Any existing code that does something similar for me to hack? Clues? Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/

5 16

ANN: happstack-server 6.4.2 - import security update
by Jeremy Shaw 06 Dec '11

06 Dec '11

Hello, I have uploaded a new version of happstack-server to hackage. The new version fixes a possible denial of service attack discovered by Gracjan Polak. The Read instances for integral types can be abused to created very large numbers. For example: > main = print (read "1e10000000000000" :: Integer) will cause the app to attempt to allocate enough ram to hold an Integer with all those zeros. This effects all integer types such as Int, Int8, Word, etc. because their 'read' instances first read things into an Integer. The 'fix' is to use 'readDec' instead of 'read' for Integral values. In this release we make the following changes: 1. change all internal uses of read to readDec 2. change the integral instances for FromReqURI to use readDec (as implied by #1) 3. the readRq helper function in RqData now uses FromReqURI instances instead of Read 4. the old readRq function is now called unsafeReadRq. 5. exported: readDec' :: (Num a) => String -> Maybe a, which is just a wrapper around readDec with a friendlier type. Unfortunately, there are still some issues we can't do anything about. For example, if users (that means *you*) have written: > do i <- read <$> look "int" Then only you can fix it. Also, if you create a type that embeds integral types: > newtype MyId = MyId { unMyId :: Integer } deriving (Show, Read) That is still affected by the read issue. If you create a FromReqURI instance, you better not use 'read'. So, 'read' is still a hidden danger to users that do not know about this pitfall. Not sure what we can do about it -- aside from changing the behavior of Read. It is not clear to me if this behavior is defined by H98, or a special 'feature' of GHC. However, this issue affects all applications which attempt to use 'read' on untrusted data, not just Happstack. In happier news, happstack-server no longer depends on happstack-util (or happstack-anything-else). The tradeoff is that it does add a dependency on base64-bytestring. The upshot is that we can now deprecate happstack-util and all the cruft that has accumulated in there. Less code to maintain == happier maintainers. - jeremy p.s. actually. building with -ftests probably still requires happstack-util.. for now. But all the tests should be moved into a separate library anyway, which will resolve that issue.

1 0

Some upcoming Lucius changes (now on Github)
by Michael Snoyman 02 Dec '11

02 Dec '11

Hi all, I wanted to announce some upcoming changes for Lucius. 1. A number of people have requested variable support. I was hesitant at first, as this overlaps with Haskell variables, but it really is too useful a feature not to have. Syntax is stolen from Less: @myColor: red; p { color: #{myColor}; } div { color: #{myColor}; } Note that you can only declare raw strings as the values of a variable, not refer to other variables. We can consider relaxing this in the future. 2. Runtime Lucius, as a parallel to runtime Hamlet. It does not support any URL interpolation (I can't think of a use case where that would be relevant), only interpolating raw text values. 3. Until a better name comes along: servius, a standalone server (based on Warp and wai-app-static) that will render Hamlet and Lucius on-the-fly. This trifecta comes in *very* handy for mocking up some HTML/CSS before integrating it into an actual site. Michael

1 0

blaze-textual-native and aeson-native are being deprecated
by Michael Snoyman 28 Nov '11

28 Nov '11

Hi all, In the last Yesod release, we created two new packages: blaze-textual-native and aeson-native. These were direct forks of their source packages, with just two changes: blaze-textual-native did not include a double-conversion dependency, and aeson-native depended on blaze-textual-native instead of blaze-textual. The reason for all this is that there is a GHC bug which prevents C++ code from working correctly with GHCi and Template Haskell[1]. Bryan recently released a new version of blaze-textual which no longer uses double-conversion by default (it's still available with compile-time flags if you want the extra speed and don't need TH/GHCi), and as a result, we are very happy to be deprecating the native siblings. Fortunately, there are not that many packages on Hackage using these libraries; in fact, the only packages for which I'm not a maintainer are dingo-core, dingo-widgets and hledger-web. However, since these are Yesod dependencies, I'm guessing many Yesod users reference aeson-native in their site cabal files. Technically speaking, switching from aeson-native to aeson in a library should constitute a major version bump. However, in this case, I think we want these deprecated packages to die as quickly as possible. So my plan is to switch all of my packages from using aeson-native to aeson some time next week. The result will likely be broken builds for some people; all you'll need to do is edit your cabal file and replace aeson-native with aeson (likewise for blaze-textual). Are there any objections to this course of action, or better ideas on how to handle the situation? Michael [1] Please don't ask me for details, I don't know them. And for all I know, the bug has already be fixed in newer GHCs.

2 1

ANNOUNCE: clientsession-0.7.3.4 (performance enhancement)
by Michael Snoyman 27 Nov '11

27 Nov '11

Hi all, tl;dr: randomIV is now much faster, API is the same, upgrade ASAP to avoid painful slowdowns. Version 0.7 of clientsession brought with it a much enhanced encryption and hashing algorithm, courtesy of Felipe Lessa. Beginning with this version, you now need to provide an Initialization Vector (IV) for encrypting cookies, and these IVs need to be randomly generated. To start off with, we simply used the getIVIO[1] function to generate these values. Recently, some Yesod users reported that version 0.9 of Yesod no longer lived up to previous benchmarks. After some research, I determined that the slowdown came from the call to randomIV in clientsession. The initial fix was simply to disable this call when it wasn't used, but it was still a significant slowdown for most users of Yesod. Over the past few days, Felipe and I put together a significantly more efficient version of randomIV. getIVIO simply opens up /dev/urandom on each request, which involves I/O. As a result, the average run time of the function in our testing was 1.686ms. Instead, our new code initializes with a call to /dev/urandom for the base randomness. It then uses AES encryption to work as a cryptographic random number generator for generating successive IVs. Additionally, every 1000 calls (we'll probably change that number over time), it spawns a new thread to reseed the generator from /dev/urandom again. The result is a function that takes 7.144764 us, aka 236 times faster. Thanks for the initial reporters who commented on the slowdown in Yesod. If you install the newest clientsession and rerun the benchmarks, you should see *significantly* better results. Michael [1] http://hackage.haskell.org/packages/archive/crypto-api/0.8/doc/html/Crypto-…

1 0

reverse proxy
by Kazu Yamamoto 15 Nov '11

15 Nov '11

Hello, I'm implementing a reverse proxy with http-enumerator on Warp. My reverse proxy can rely HTTP headers but I have no idea on how to relay HTTP bodies. I want to know how to specify Network.HTTP.Enumerator.Request.requestBody on WAI. My current code is available from: https://github.com/kazu-yamamoto/wai-app-file-cgi/blob/master/Network/Wai/A… Thanks in advance. --Kazu

2 1

ANN: websockets 0.4.0.0
by Jasper Van der Jeugt 04 Nov '11

04 Nov '11

Hello all, I'm pleased to announce the release of websockets [1] 0.4.0.0 today. You can grab it on Hackage [2]. This update brings an (almost completely rewritten) library which has support for multiple versions of the protocol, meaning you can choose to write backwards-compatible applications, or use the latest and greatest, all using a common API. All feedback is obviously welcome, I'd be glad to hear what you guys think. For the next version, client-side support is planned, so the library user can write clients in addition to servers. If there's any other feature you would like to see, feel free to let me know. [1]: http://jaspervdj.be/websockets/ [2]: http://hackage.haskell.org/package/websockets Cheers, Jasper

3 4

[persistent] [postgres] PoolExhaustedException in Persistent
by Max Cantor 04 Nov '11

04 Nov '11

Hi, I'm getting PoolExhaustedException's on persistent-postgres. I'm on persistent 6 and yesod 9. any ideas where to start looking for a source? my runDB is straight out of the foundation: runDB f = liftIOHandler $ fmap connPool getYesod >>= Settings.runConnectionPool f thanks, max

2 3