August 2012 - web-devel

Re: [web-devel] new field for Network.HTTP.Types.Status
by Kazu Yamamoto 31 Aug '12

31 Aug '12

Hello, > What if you try just the bang pattern and no unboxing? Here it is: bang for three fields: 41346.8 req/s bang and unpack for Int only: 41761.7 req/s So, the current merged code is the best. --Kazu

2 4

Conduit/Wai proxy, HOWTO?
by Jason Dusek 29 Aug '12

29 Aug '12

Please find below my signature a sketch of a WAI proxy server, which always redirects every request to the same URL. In the sketch, the function `magic' takes care of translating from a Network.HTTP.Conduit.Response body to Network.Wai.ResponseSource body. However, I am a at a loss as to how to write `magic' safely. The body of a Network.HTTP.Conduit.Response is a ResumableSource but the body of Network.Wai.ResponseSource is a plain source and the conversion from one to the other would seem to involve one of: * Connecting the ResumableSource to a Sink, thus regrettably forcing it all into memory, * Unwrapping the resumable source and trying to figure out what to do with the finalizer. I was originally attracted to Wai/Warp/Conduit by the prospect of building a high quality proxy server. I am building an web service that talks to AWS and, for some tasks, must proxy results back to the user. I would be very grateful if someone could explain to me how best to put together a simple, efficient proxy server with Wai and Conduit. -- Jason Dusek pgp // solidsnack // C1EBC57DC55144F35460C8DF1FD4C6C1FED18A2B > {-# LANGUAGE OverloadedStrings > , RecordWildCards > #-} > > import Control.Applicative > import Control.Monad > import Data.ByteString (ByteString) > import qualified Data.ByteString.Char8 as Bytes > > import qualified Blaze.ByteString.Builder as Blaze > import qualified Blaze.ByteString.Builder.Char.Utf8 as Blaze > import Control.Monad.Trans > import qualified Data.Conduit as Conduit > import qualified Data.Conduit.List as Conduit > import qualified Network.Wai as Wai > import qualified Network.HTTP.Conduit as Conduit > import qualified Network.HTTP.Types as HTTP > > wai :: String -> Wai.Application > wai s req@Wai.Request{..} = Conduit.withManager go > where > magic = undefined > go manager = do > request <- liftIO $ Conduit.parseUrl s > Conduit.Response s _ h src <- Conduit.http request manager > return $ Wai.ResponseSource s h (magic src)

1 1

Re: [web-devel] new field for Network.HTTP.Types.Status
by Kazu Yamamoto 29 Aug '12

29 Aug '12

Hello, > I believe unboxing is not possible for ByteString, but it is for the Int > field. I don't really know how much performance gains this brings, not really > a Haskell performance guru. I tried this: data Status = Status { statusCode :: {-# UNPACK #-} !Int , statusAsciiCode :: {-# UNPACK #-} !B.ByteString , statusMessage :: {-# UNPACK #-} !B.ByteString } deriving (Show) Since ByteString has just one constructor, UNPACK for ByteString should work according to GHC manual. This results in 41345.3 req/s while no bangs and no UNPACK results in 43098.6. So, let's not add bangs and UNPACKs. Please merge the current pull request. Thanks in advance. --Kazu

1 0

Re: [web-devel] new field for Network.HTTP.Types.Status
by Kazu Yamamoto 28 Aug '12

28 Aug '12

Hello, > Looks good. Could you please test the performance impact of also making the > fields strict, and of making them strict and unboxed? I think we can make Int unboxed. But what about ByteString? Does the unboxed pragma make ByteString different? --Kazu

1 0

new field for Network.HTTP.Types.Status
by Kazu Yamamoto 28 Aug '12

28 Aug '12

Hello, I'm now tuning warp. https://github.com/kazu-yamamoto/wai/tree/refactoring/warp One of bottlenecks is the code to compose HTTP header. I have already rewritten the code using direct memory copy instead of BlazeBuilder. https://github.com/kazu-yamamoto/wai/blob/refactoring/warp/Network/Wai/Hand… This code gain around 2,000 req/s on my machine. In this code, you can find two 'divMod' and one 'mod'. These is to convert numeric HTTP status code to ByteString. It appeared this conversion is slow. Note that due to a bug of GHC, five (not three) divisions are carried out. So, I redefined Network.HTTP.Types.Status to have reformatted ByteString for status code (the statusAsciiCode field): https://github.com/kazu-yamamoto/http-types/blob/status-ascii-code/Network/… This hack gains also around 2,000 req/s. This change breaks compatibilities. I found that warp, http-conduit, wai-app-file-cgi are affected. So, my question: is this hack acceptable? If not, I will define a large array of ByteString in warp. But it seems to me that this is nonsense. --Kazu

2 3

Re: [web-devel] [Yesod] Re: Abstracting permissions with Yesod
by Felipe Almeida Lessa 27 Aug '12

27 Aug '12

On Mon, Aug 27, 2012 at 5:52 AM, Arthur Clemens <arthurclemens(a)gmail.com> wrote: > Did something change in Yesod? > This part doesn't seem to work: > > isAuthorized route isWrite = do > mauth <- maybeAuth > runDB $ mauth `isAuthorizedTo` permissionsRequiredFor route isWrite > > Returns > > Foundation.hs:109:20: > Couldn't match expected type `(UserId, User)' > with actual type `Entity val0' To fix this bug you'll need to change all '(UserId, User)'s into 'Entity User's. For example: hasPermissionTo :: Entity User -> Permission -> YesodDB sub Blog AuthResult (Entity _ user) `hasPermissionTo` Post = ... etc. I'm not sure if there's anything else missing. Cheers, =) -- Felipe.

1 0

yesod persistent mysql
by Asafe Ribeiro 24 Aug '12

24 Aug '12

Are there any persistent-mysql examples ? I can't get anything from http://www.yesodweb.com/book/persistent to compile, let alone with mysql.

2 5

new warp
by Kazu Yamamoto 22 Aug '12

22 Aug '12

Hello, In the last week, with Michael's permission, I have refactored warp so that we can tune it up more easily: https://github.com/kazu-yamamoto/wai/tree/refactoring/warp This version maintains the backward compatibility to the current WAI and passed all tests. I'm now testing it in the real world. If you are interested in the code, please take a look. Comments are highly appreciated. If you volunteer, please test it and tell me whether or not it works well. Thanks in advance. --Kazu

2 1

Re: [web-devel] new field for Network.HTTP.Types.Status
by Kazu Yamamoto 18 Aug '12

18 Aug '12

Hello Aristid, > So, two questions: > > 1. Is this a bottleneck just in your small benchmark, or do you think this > would affect other use-cases too? To see how worth it is to break > compatibility. This effects if we want to implement a really fast HTTP server like nginx. Otherwise, this does not matter. > 2. Have you tried caching the rendered status codes in an array or so? That > might be another way of speeding this up without changing the Status type. I will try the array approach in the next week. If that is fast enough and http-types includes the array, it would be very appreciated. --Kazu

1 0

WAI Routing
by Stephen Paul Weber 09 Aug '12

09 Aug '12

I'm back! I've walked in the desert for a bit longer than last time I was on this list, and have a much better handle on Haskell base concepts, which is useful. So, I re-looked-at yesod-routes as it exists now, and Yesod.Routes.Dispatch is very nice for doing the sort of thing I want. Doesn't even use any language extensions itself! (Oh, almost, it uses bang patterns. Or is that not an extension when used in a datatype? Anyway...) Unfortunately, using Yesod.Routes.Dispatch by itself is a bit verbose. Of course, yesod-routes has all kinds of nice utilities to help with that, but they are very TH-heavy, so I set out to build a different way: code generation! The result is: <http://hackage.haskell.org/package/route-generator> which takes in a simple text file format (attoparsec FTW!) and outputs a module that can be compiled into any application using yesod-routes and provide a list of routes based on the text file. The types of the captured URI segments are inferred from the types of the actual Haskell code that handles the route. The generator assumes a single module that it will import exports names for all of the handlers, but I don't think this is too bad since you can always have a module that just re-exports all the handlers from wherever you define them. A simple example is in the git repository. I have also pushed up <http://hackage.haskell.org/package/wai-dispatch>, which is a simple wrapper around Yesod.Routes.Dispatch for use in WAI apps to convert a "default" (likely 404 or static file serving) Application, and a list of Route for yesod-route, into an Application that does the routing. It's very little code, but I immidiately identified it as code that would end up appearing in every single one of my web apps, so I abstracted it out into a tiny library. -- Stephen Paul Weber, @singpolyma See <http://singpolyma.net> for how I prefer to be contacted edition right joseph

2 2