- web-devel - Haskell.org

[Wai] Sending partial files
by Bardur Arantsson 03 Feb '11

03 Feb '11

Hi all, Is there a way to send a part of a file in response to a request using WAI? I'm assuming it's possible, but I'm having a little trouble figuring out the type signatures in Network.Wai. I think I'll be able to adapt the Data.Enumerator.Binary.enumFile and enumHandle functions to handle a range, so I guess the question is really: How can I send a status + headers + arbitrary output from an enumerator in WAI? Cheers,

2 2

Re: [web-devel] [Haskell-cafe] http-enumerator: redirects, streaming and keep-alive
by Michael Snoyman 03 Feb '11

03 Feb '11

On Thu, Feb 3, 2011 at 12:00 AM, Antoine Latter <aslatter(a)gmail.com> wrote: > On Wed, Feb 2, 2011 at 3:01 PM, Felipe Almeida Lessa > <felipe.lessa(a)gmail.com> wrote: >> On Wed, Feb 2, 2011 at 6:30 PM, Antoine Latter <aslatter(a)gmail.com> wrote: >>> Or you could remove the socket from the map while it's in use. >> >> And what about connection limits? We shouldn't create a thousand >> connections to the same host =). >> > > Not a bad idea, but that may be creeping outside the scope of the bug > as reported. > > If you're writing a web-scraper/spider, it is true you might need some > sort of higher-level manager to handle concurrent access. I'm not sure > what that would look like, though > >> -- >> Felipe. >> > I think Felipe's point is that using the approach I outlined, we will never spawn more than one connection to a single host. By simply taking the socket out of the map, there's no limit to the number of sockets that will be created. Michael

1 0

http-enumerator: redirects, streaming and keep-alive
by Michael Snoyman 02 Feb '11

02 Feb '11

Hi all, There are a number of new feature requests for http-enumerator, and I wanted to discuss some possible API changes with everyone: * Allow keep-alive requests, which will reuse the same connection. * Allow client code to determine whether it accepts a server's SSL certificate. * Allow the request body to be an Enumerator so we can send large request bodies without using lazy IO. This is already implemented via streamingHttp, but it would make more sense to modify the Request datatype to replace L.ByteString with (Int, Enumerator ByteString IO ()). * Fix redirect behavior. It seems that the right thing to do is change a POST into a GET and remove the request body for a 303, and resend the same request for all other 3xx codes. However, when using an Enumerator for the request body, it is not guaranteed that we can resend an Enumerator. For addressing the last two, my plan is to: * Change the Request datatype as mentioned, but provide a convenience method for converting a lazy ByteString into a (Int, Enumerator ByteString IO ()). * Implement redirecting as I have described, and explain in the documentation that automatic redirecting can only work with Enumerators that can be run multiple times. AFAIK, this works with most standard Enumerators, such as enumFile, and with the helper function I will provide to convert lazy ByteStrings to Enumerators. * Remove streamingHttp, as it will no longer be necessary. As far as keep-alive goes, I still need to do a bit more research, but my basic idea (with credit to Bryan O'Sullivan): * http (and family) will all take an extra argument, Maybe Manager. * Manager will be an abstract type that will keep an MVar (Map (Host, Port, IsSecure) Socket). * If http is provided with a Manager, then it uses the Socket available in the Manager. If none is available, it creates a new Socket and places it in the Manager. * If http is *not* provided with a Manager, then it creates a new socket and closes it before returning. * There will be a newManager :: IO Manager, and a closeManager :: Manager -> IO (), which closes all Sockets in the Manager and empties out the inner Map. Some open questions are: * Who is responsible to recover if a server is no longer responding on a Socket from the Manager: http or the client code? I would assume http. * I haven't fully thought through how this will work with secure connections: most likely in addition to storing the Socket in the Manager I will need to store some certificate data. And speaking of certificates, the main concern I have here is data types: since http-enumerator can use either tls or OpenSSL, there isn't a single certificate datatype I can use. If someone wants to help out and write some code to convert an OpenSSL X509 datatype into a Certificate from the certificate package (which tls uses), I will be very grateful. Assuming we had such a unified datatype, I would recommend changing Request's secure record to be Certificate -> IO Bool, where returning True means the certificate is accepted and False means it is rejected. To get the current functionality of trusting any certificate, we would use (const $ return True). Any thoughts? Michael

2 2

[Yesod] How to write approot to handle http and https.
by Katsutoshi Itoh 01 Feb '11

01 Feb '11

Hi I'd like to host application for http and https. I set up apache and use rewrite rules. In this case, how do i write approot of Yesod instance? when client request by http://www.foo.com/a/b/c/, approot must be http://www.foo.com, one the other hand, when client request by https://www.foo.com/a/b/c/, approot must be https://www.foo.com. how do i write?

2 1

Re: [web-devel] Proposal: http-types
by Aristid Breitkreuz 01 Feb '11

01 Feb '11

Did I accidentally reply just to you? Sorry, I was expecting better default behavior of Gmail... After a closer look, I now see that WAI really is mostly just types, but those types DO seem to be fairly server-centric. Response especially does not seem to be particularly useful for clients, but Request might need some modifications too. Aristid 2011/2/2 Jeremy Shaw <jeremy(a)n-heptane.com> > I am not sure what you mean. WAI does not actually include a server. > It is just the Request/Response types, status codes, and some headers, > > > http://hackage.haskell.org/packages/archive/wai/0.3.1/doc/html/Network-Wai.… > > if you want an actual server (which you don't) then you would need > warp or something. > > - jeremy > > On Tue, Feb 1, 2011 at 4:59 PM, Aristid Breitkreuz > <aristidb(a)googlemail.com> wrote: > > WAI is too wide in scope, IMO. I would like to be able to use http-types > in > > client-only code that does not want any of the HTTP server functionality. > > > > Aristid > > > > 2011/2/1 Jeremy Shaw <jeremy(a)n-heptane.com> > >> > >> This is essentially what WAI is supposed to be. (Though WAI is more > >> server focused). > >> > >> I think it is a good idea, but people seem unhappy to standardize on > >> what those types should look like... > >> > >> - jeremy > >> > >> On Tue, Feb 1, 2011 at 1:55 PM, Aristid Breitkreuz > >> <aristidb(a)googlemail.com> wrote: > >> > Hi, > >> > Hear me out, I have an idea! :-) > >> > How about creating a new package "http-types" containing Request / > >> > Response > >> > / status code / header name types for HTTP. These could be used by > both > >> > server and client software, and might even allow switching between > HTTP > >> > client libraries (say, if I want to switch between http-enumerator and > >> > HTTP). > >> > What do you think? > >> > > >> > Aristid > >> > _______________________________________________ > >> > web-devel mailing list > >> > web-devel(a)haskell.org > >> > http://www.haskell.org/mailman/listinfo/web-devel > >> > > >> > > > > > >

2 1

Re: [web-devel] [Haskell-cafe] plugins and internal error: stg_ap_v_ret
by Michael Snoyman 01 Feb '11

01 Feb '11

Thanks for the link Andy, this definitely looks like the right I will ultimately need to take. I just found out that plugins does not install very well on Windows, which is something I need to provide good support for with wai-handler-devel. Michael On Tue, Feb 1, 2011 at 1:15 PM, Andy Stewart <lazycat.manatee(a)gmail.com> wrote: > Hi Michael, > > I have write some dynamic-loading code for my Manatee project > (http://hackage.haskell.org/package/manatee) > > Dynload.hs use GHC API, if you interested it, you can read source code: > > https://patch-tag.com/r/AndyStewart/manatee-core/snapshot/current/content/p… > > Cheers, > > -- Andy > > Michael Snoyman <michael(a)snoyman.com> writes: > >> Hi Andy, >> >> plugins *is* working in general for me for some trivial test cases. >> It's specifically this use case with WAI that's causing trouble, which >> implies to me I'm misusing the API somehow. >> >> Michael >> >> On Tue, Feb 1, 2011 at 4:22 AM, Andy Stewart <lazycat.manatee(a)gmail.com> wrote: >>> Hi Michael, >>> >>> plugins use it's own function instead GHC API, so it's easy to break >>> with new version GHC. >>> >>> -- Andy >>> >>> Michael Snoyman <michael(a)snoyman.com> >>> writes: >>> >>>> Hi all, >>>> >>>> I'm trying to convert wai-handler-devel to use plugins instead of >>>> hint, but cannot even get some basic usages to work properly. I've put >>>> together a minimal example that loads a WAI application from a >>>> separate file and runs it, but this immediately causes the program to >>>> crash saying: >>>> >>>> loader: internal error: stg_ap_v_ret >>>> (GHC version 6.12.3 for i386_unknown_linux) >>>> Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug >>>> >>>> Is this an actual bug in GHC, or am I misusing the plugins package? >>>> >>>> The two source files: >>>> >>>> MyModule.hs >>>> {-# LANGUAGE OverloadedStrings #-} >>>> module MyModule where >>>> >>>> import Network.Wai >>>> import Data.ByteString.Lazy.Char8 () >>>> >>>> myapp _ = responseLBS status200 [("Content-Type", "text/plain")] "myapp" >>>> >>>> loader.hs >>>> import System.Plugins.Make >>>> import System.Plugins.Load >>>> import Network.Wai.Handler.Warp (run) >>>> >>>> main :: IO () >>>> main = do >>>> MakeSuccess _ obj <- makeAll "MyModule.hs" [] >>>> LoadSuccess _ app <- load_ obj [] "myapp" >>>> run 3000 app >>>> >>>> Thanks, >>>> Michael >>> >>> >>> _______________________________________________ >>> Haskell-Cafe mailing list >>> Haskell-Cafe(a)haskell.org >>> http://www.haskell.org/mailman/listinfo/haskell-cafe >>> >

1 0

plugins and internal error: stg_ap_v_ret
by Michael Snoyman 01 Feb '11

01 Feb '11

Hi all, I'm trying to convert wai-handler-devel to use plugins instead of hint, but cannot even get some basic usages to work properly. I've put together a minimal example that loads a WAI application from a separate file and runs it, but this immediately causes the program to crash saying: loader: internal error: stg_ap_v_ret (GHC version 6.12.3 for i386_unknown_linux) Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug Is this an actual bug in GHC, or am I misusing the plugins package? The two source files: MyModule.hs {-# LANGUAGE OverloadedStrings #-} module MyModule where import Network.Wai import Data.ByteString.Lazy.Char8 () myapp _ = responseLBS status200 [("Content-Type", "text/plain")] "myapp" loader.hs import System.Plugins.Make import System.Plugins.Load import Network.Wai.Handler.Warp (run) main :: IO () main = do MakeSuccess _ obj <- makeAll "MyModule.hs" [] LoadSuccess _ app <- load_ obj [] "myapp" run 3000 app Thanks, Michael

2 2

http-enumerator and redirects
by Aristid Breitkreuz 31 Jan '11

31 Jan '11

Hi, Michael Snoyman plans to expose http-enumerator's HTTP redirect logic in a public API. For this reason, we had a short discussion over e-mail, which he asked me to resume here. The problem at stake is how to deal with redirects in the case of POST requests, especially with streaming request bodies (streamingHttp as of 0.3.1). Currently, the same HTTP method is always used, but this is not really the correct course of action. RFC 2616 mandates that for 301 and 302 responses, the request should be retried with the same HTTP method (side problem: restarting streams?). 303 triggers a GET, however, and I'm not sure about 304-307. RFC 2616 also mentions that most existing clients actually unconditionally use GET for 302 responses, which we should take into account. Michael proposed always forcing GET, but I'm not sure if this is correct either, because for example 301 clearly demands POST to be re-used. So, the question is, how do we solve these problems? Should there be an element of configurability? Aristid

1 0

yesod's hamletFileDebug
by Rafael Cunha de Almeida 29 Jan '11

29 Jan '11

Hello, I'm new to yesod and haskell web development in general. I read a bit about yesod and it looked nice. I thought it was really cool that I'd be able to view my changes to a template without recompiling, something I've always found very appealing in interpreted language's frameworks. However, after running yesod it created a Setting.hs file which uses H.cassiusFileDebug and H.juliusFileDebug for the purpose of dynamically generating such files, but hamlet files doesn't seem to work that way. Why was that changed? Am I to expect such feature will come back in the future? Or is already possible it just need to be set somewhere else? []'s Rafael

2 1

Hoogle Advice
by Neil Mitchell 28 Jan '11

28 Jan '11

Hi, I am the author of Hoogle (http://haskell.org/hoogle/) a Haskell API search engine, that is usually used through a web interface. I'm not an expert on website programming in Haskell, and I'm not a server admin by any means. Currently the web side of Hoogle can run in two modes - as a CGI binary (as deployed on haskell.org) or with "hoogle server" which starts its own server. I wrote both the CGI interface (see http://code.haskell.org/hoogle/src/General/Web.hs) and the server (see http://code.haskell.org/hoogle/src/Web/Server.hs) myself. I imagine there is something in the web devel community that I should be using instead? If so, what is it, and how do I use it? I have a few requirements: * I have to be able to integrate it with haskell.org (which probably runs Apache?). * I also want to be able to run it as a server where the hoogle binary runs the server. * If I could get log/email messages whenever serving a page caused an exception, or took longer than a certain threshold, that would be great - but isn't necessary. * I'd like to log all queries run, and in particular how long they take, so I can investigate performance issues. Thanks for any advice you may be able to give, Neil

7 37