Re: [web-devel] proxy with wai

newer
Yesod Mform and hamlet

older
proxy with wai

Michael Snoyman

24 Mar 2015 24 Mar '15

3:01 a.m.

On Sun, Mar 22, 2015 at 10:01 AM Jos van den Oever wrote:

...

On Sunday 08 March 2015 07:28:30 you wrote:

...
I've updated that Gist to be compatible with WAI 3.0. The basic changes I made were:

1. Change the way Warp settings are applied (no longer use record syntax, but explicit set* functions). 2. conduit is no longer the standard streaming interface for WAI, so imported Network.Wai.Conduit and used helper functions from there. 3. The specific question you asked about: the semantics for how to do resource allocation in an application have changed. There were some holes in the previous approach that allowed async exceptions to leak in at the wrong time. Instead, WAI 3.0 uses a similar interface to most resource allocation functions in Haskell (like bracket or withFile). Specifically, this looks like:

HCl.withResponse req2 man $ \res -> do let body = mapOutput (Chunk . fromByteString) $ HCC.bodyReaderSource $ HCl.responseBody res headers = filter safeResHeader $ HCl.responseHeaders res respond $ responseSource (HCl.responseStatus res) headers body

I'd like my proxy to be a caching proxy so each request shoule be saved to a file. I've tried adding an additional sink, (sinkFile) or conduit (conduitFie) to the HCC.bodyReaderSource, but cannot it right. What would be a good way to do this?

What have you tried so far? Michael

Attachments:

attachment.html (text/html — 1.9 KB)

Show replies by date

Jos van den Oever

26 Mar 26 Mar

3:10 p.m.

New subject: proxy with wai

On Tuesday 24 March 2015 07:01:39 Michael Snoyman wrote:

...

On Sun, Mar 22, 2015 at 10:01 AM Jos van den Oever

wrote:

...
On Sunday 08 March 2015 07:28:30 you wrote:

...
I've updated that Gist to be compatible with WAI 3.0. The basic changes I made were:

1. Change the way Warp settings are applied (no longer use record syntax, but explicit set* functions). 2. conduit is no longer the standard streaming interface for WAI, so imported Network.Wai.Conduit and used helper functions from there. 3. The specific question you asked about: the semantics for how to do resource allocation in an application have changed. There were some holes in the previous approach that allowed async exceptions to leak in at the wrong time. Instead, WAI 3.0 uses a similar interface to most resource allocation functions in Haskell (like bracket or withFile). Specifically,

this looks like: HCl.withResponse req2 man $ \res -> do

let body = mapOutput (Chunk . fromByteString) $

HCC.bodyReaderSource $ HCl.responseBody res

headers = filter safeResHeader $ HCl.responseHeaders res

respond $ responseSource (HCl.responseStatus res) headers body

I'd like my proxy to be a caching proxy so each request shoule be saved to a file. I've tried adding an additional sink, (sinkFile) or conduit (conduitFie) to the HCC.bodyReaderSource, but cannot it right. What would be a good way to do this?

What have you tried so far?

This is the code I have so far. I've annotated with a types. This code gives a compile error because of the line 'source2 = bodySource C.=$= fileSaver'. The MonadResource is not arising from the use in fileSaver. Somehow, I've to get runResourceT in there. === HCl.withResponse req2 man $ \res -> do --saver <- CB.conduitFile "/tmp/out" let path = "/tmp/out" :: FilePath fileSaver :: MonadResource m => C.Conduit S8.ByteString m S8.ByteString fileSaver = CB.conduitFile path bodyReader :: HCl.BodyReader bodyReader = HCl.responseBody res bodySource :: MonadIO m => C.ConduitM () S8.ByteString m () bodySource = HCC.bodyReaderSource bodyReader source2 = bodySource C.=$= fileSaver chunk :: S8.ByteString -> C.Flush Builder chunk = C.Chunk . fromByteString body :: C.ConduitM () (C.Flush Builder) IO() body = C.mapOutput chunk source2 headers = filter removeEncodingHeaders $ HCl.responseHeaders res time <- getPOSIXTime runSqlPool (insert_ $ Response (round time) reqid Nothing) pool respond $ WC.responseSource (HCl.responseStatus res) headers body === Cheers, Jos

Michael Snoyman

27 Mar 27 Mar

1:03 a.m.

New subject: proxy with wai

Can you provide: * A complete file * The error messages you're getting On Thu, Mar 26, 2015 at 9:10 PM Jos van den Oever wrote:

...

...
On Sun, Mar 22, 2015 at 10:01 AM Jos van den Oever
wrote:

...
On Sunday 08 March 2015 07:28:30 you wrote:

...
I've updated that Gist to be compatible with WAI 3.0. The basic changes I made were:

1. Change the way Warp settings are applied (no longer use record syntax, but explicit set* functions). 2. conduit is no longer the standard streaming interface for WAI, so imported Network.Wai.Conduit and used helper functions from there. 3. The specific question you asked about: the semantics for how to do resource allocation in an application have changed. There were some holes in the previous approach that allowed async exceptions to leak in at

...
...
...
wrong time. Instead, WAI 3.0 uses a similar interface to most resource allocation functions in Haskell (like bracket or withFile). Specifically,

this looks like: HCl.withResponse req2 man $ \res -> do

let body = mapOutput (Chunk . fromByteString) $

HCC.bodyReaderSource $ HCl.responseBody res

headers = filter safeResHeader $ HCl.responseHeaders res

respond $ responseSource (HCl.responseStatus res) headers

On Tuesday 24 March 2015 07:01:39 Michael Snoyman wrote: the body

...
...
I'd like my proxy to be a caching proxy so each request shoule be

saved to

...
a file. I've tried adding an additional sink, (sinkFile) or conduit (conduitFie) to the HCC.bodyReaderSource, but cannot it right. What would be a good way to do this?

What have you tried so far?

This is the code I have so far. I've annotated with a types. This code gives a compile error because of the line 'source2 = bodySource C.=$= fileSaver'. The MonadResource is not arising from the use in fileSaver. Somehow, I've to get runResourceT in there.

=== HCl.withResponse req2 man $ \res -> do --saver <- CB.conduitFile "/tmp/out"

let path = "/tmp/out" :: FilePath fileSaver :: MonadResource m => C.Conduit S8.ByteString m S8.ByteString fileSaver = CB.conduitFile path bodyReader :: HCl.BodyReader bodyReader = HCl.responseBody res bodySource :: MonadIO m => C.ConduitM () S8.ByteString m () bodySource = HCC.bodyReaderSource bodyReader source2 = bodySource C.=$= fileSaver chunk :: S8.ByteString -> C.Flush Builder chunk = C.Chunk . fromByteString body :: C.ConduitM () (C.Flush Builder) IO() body = C.mapOutput chunk source2 headers = filter removeEncodingHeaders $ HCl.responseHeaders res

time <- getPOSIXTime runSqlPool (insert_ $ Response (round time) reqid Nothing) pool respond $ WC.responseSource (HCl.responseStatus res) headers body ===

Cheers, Jos

_______________________________________________ web-devel mailing list web-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/web-devel

Jos van den Oever

4:01 a.m.

New subject: proxy with wai

On Friday 27 March 2015 05:03:18 Michael Snoyman wrote:

...

Can you provide:

* A complete file * The error messages you're getting

Of course. ===================== No instance for (MonadResource IO) arising from a use of `fileSaver' In the second argument of `(C.=$=)', namely `fileSaver' In the expression: bodySource C.=$= fileSaver In an equation for `source2': source2 = bodySource C.=$= fileSaver HttpWrite.hs /proxy/src line 35 Haskell Problem ===================== I know that I should somehow put some of the code a runResourceT, but I cannot figure out how to combine it with Wai.Application. Cheers, Jos

Michael Snoyman

5:17 a.m.

New subject: proxy with wai

Probably the most expedient thing is to use `withFile` to get a Handle, and then use `conduitHandle` instead of `conduitFile`. However, if you want to use ResourceT, it's certainly possible, but a bit hairier: runResourceT $ withInternalState $ \is -> do respond $ WC.responseSource (HCl.responseStatus res) headers $ C.transPipe (flip runInternalState is) body On Fri, Mar 27, 2015 at 11:01 AM Jos van den Oever wrote:

...

On Friday 27 March 2015 05:03:18 Michael Snoyman wrote:

...
Can you provide:

* A complete file * The error messages you're getting

Of course.

===================== No instance for (MonadResource IO) arising from a use of `fileSaver' In the second argument of `(C.=$=)', namely `fileSaver' In the expression: bodySource C.=$= fileSaver In an equation for `source2': source2 = bodySource C.=$= fileSaver HttpWrite.hs /proxy/src line 35 Haskell Problem =====================

I know that I should somehow put some of the code a runResourceT, but I cannot figure out how to combine it with Wai.Application.

Cheers, Jos _______________________________________________ web-devel mailing list web-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/web-devel

3707

Age (days ago)

3710

Last active (days ago)

List overview

Download

4 comments

2 participants

participants (2)

Jos van den Oever
Michael Snoyman