July 2013 - web-devel

Next version of WAI: cleaning house
by Michael Snoyman 12 Aug '13

12 Aug '13

Some of you may have seen the brief interchange between Kazu and myself regarding ResourceT in WAI. Let me give a very brief synopsis of the situation: - In order to allow WAI applications to acquire scarce resources like file handles in an exception-safe way, WAI actions all live in a `ResourceT IO` monad, instead of just `IO`. - Kazu has mentioned to me in the past that, based on his profiling, monadic bind for ResourceT may currently be a bottleneck in WAI. - I made an experimental change to WAI so that, instead of actions living in `ResourceT IO`, the WAI Request value contains a ResourceT InternalState value, which can be used to accomplish the exact same thing as living in ResourceT (see [1]). - Kazu checked out the performance difference of this new branch, and it allows for some better optimizations. If this was purely a performance optimization, I would probably just optimize Warp and not bother changing WAI. However, I think that this change makes WAI itself better as well, and therefore am in favor of making this as a breaking change, releasing it as WAI 2.0[2]. Firstly, are there any objections to this move? So as long as we're making a breaking change, the question arises: what other changes should we be making to WAI? I know Kazu had mentioned adding fields to avoid the need for lookups, can you clarify that request a bit? Here are some other changes I can think of: - Remove the deprecated isSecure field from the Request data type. - Remove the misleading fields serverName and serverPort fields as well. - Possibly: hide the Request constructor so that new fields can be added without breaking backwards compatibility. If anyone else has some ideas, please bring them up. [1] http://haddocks.fpcomplete.com/fp/7.4.2/20130704-120/resourcet/Control-Mona… [2] For the Yesod community: the change is minor enough that Yesod 1.2 can have some conditional compilation to support both the current WAI 1.4 and WAI 2.0, so this shouldn't affect most Yesod users at all.

9 31

Re: [web-devel] [Yesod] Next version of WAI: cleaning house
by Greg Weber 25 Jul '13

25 Jul '13

When a request hits a handler, it will already have gone through the middleware, so it only seems possible to alter how the response is handled at that point. If you want different middleware for different routes, that is something that must be handled at the routing level, so it seems out of the scope of WAI. But if you explain your use case on the Yesod mail list we can try and come up with a more convenient solution. I do have a WAI middleware that checks routes that might benefit from this. On Thu, Jul 25, 2013 at 10:20 AM, Robert Lee <robert.lee(a)chicago.vc> wrote: > Is it possible in the present design to change middleware modules at the > level of a handler? > > On Tue, 2013-07-23 at 13:04 +0300, Michael Snoyman wrote: > > Some of you may have seen the brief interchange between Kazu and > > myself regarding ResourceT in WAI. Let me give a very brief synopsis > > of the situation: > > * In order to allow WAI applications to acquire scarce resources > > like file handles in an exception-safe way, WAI actions all > > live > > > -- > You received this message because you are subscribed to the Google Groups > "Yesod Web Framework" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to yesodweb+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > >

1 0

Making throughput much better
by Kazu Yamamoto 22 Jul '13

22 Jul '13

Hi all, Cc: Simon Marlow I think I found a way to make the throughput of web servers much better. In short, put "Control.Concurrent.yield" after sending something. This enables good schedule shaping. Typical code for web servers is like this: loop = do keepAlive <- receiveRequest sendResponse when keepAlive loop With this code, a typical sequence of system calls is like this: recvfrom(13, ) -- thread A sendto(13, ) -- thread A recvfrom(13, ) = -1 EAGAIN -- thread A epoll_ctl(3, ) -- IO manager recvfrom(14, ) -- thread B sendto(14, ) -- thread B recvfrom(14, ) = -1 EAGAIN -- thread B epoll_ctl(3, ) -- IO manager Since each thread calls recvfrom() immediately after sendto(), the possibility that recvfrom() can receive a request is low. This involves the IO manager. To make the possibility higher, I put "yield" after sendResponse: loop = do keepAlive <- receiveRequest sendResponse yield when keepAlive loop Yield pushes its Haskell thread onto the end of thread queue. So, another thread can work. During the work of other threads, a request message would arrive. recvfrom(13, ) -- thread A sendto(13, ) -- thread A recvfrom(14, ) -- thread B sendto(14, ) -- thread B recvfrom(13, ) -- thread A sendto(13, ) -- thread A I tested this idea on SimpleServer and confirmed that its throughput is doubled: No yield: 58,152 req/s (https://gist.github.com/AndreasVoellmy/4184520#file-simpleserver-hs) Yield: 113,048 req/s (https://gist.github.com/kazu-yamamoto/6051118#file-simpleserver2-hs) Unfortunately, yield does not work for Warp as expected. It does not pass control to another thread and behaves as used to be. So, my question: why doesn't "yield" work in Warp? ResourceT is doing a special thing? My code change is here: https://github.com/yesodweb/wai/commit/4e9c6316ad59cd87be13000d15df8e6fd7c3… I'm using GHC head (with multicore IO manager). I'm sure that the following patch to fix the bug of yield is applied: https://github.com/ghc/ghc/commit/66839957a376dbe85608822c1820eb6c99210883 P.S. It would be appreciated if someone tries this idea on other web servers. --Kazu

3 4

Automatic error traces
by Alberto G. Corona 18 Jul '13

18 Jul '13

It is important to have execution traces in case of error. specially in server applications that run 24/7 such are web applications Thanks to the wonderful package monadloc by Pepe Iborra, now MFlow can generate a complete execution trace in case of error. The control-monad-exception uses monadLoc to generate stack traces, but MFlow makes use of his backtracking mechanism to generate a complete execution trace. Here I explain what and how: http://haskell-web.blogspot.com.es/2013/07/automatic-error-trace-generation… The MFlow version that implements this is in gitHub. Not in hackage yet. https://github.com/agocorona/MFlow I´m quite proud of it since it is one of the things closest to magic that I have done. Feedback? I do not want to keep MFlow as a single person development. I think that it has many unique and nice features not available in other languages and frameworks, and it can be raised to a serious exploitation level by the Haskell community. This will attract people working in Industrial Web development to Haskell thanks to the edge in expressiveness and safety necessary for creating industrial solutions that Haskell has over other languages. It uses most other Haskell web developments blaze, wai, hamlet etc and there are many other things to come. monadLoc : http://hackage.haskell.org/package/monadloc -- Alberto.

2 3

Re: [web-devel] [Haskell-cafe] ANNOUNCE: MFlow 3.0
by Alberto G. Corona 13 Jul '13

13 Jul '13

My plan is to synchronize MFlow servers using cloud Haskell since the state serialization is small. I´m working on it. However continuation based frameworks can not synchronize state. There is "swarm" in scala that generate portable continuations but this is not used in the context of web application since the state in a continuation is big and can not be synchronized fast enough. In other languages these states are not even portable between machines. 2013/7/10 Alexander Kjeldaas <alexander.kjeldaas(a)gmail.com> > Here are some common-lisp web frameworks using continuations: > > http://common-lisp.net/project/cl-weblocks/ > http://common-lisp.net/project/ucw/features.html > > What always worried me with these frameworks is how they could be made > robust in case of failures. Storing all state in a database backend often > makes it possible to isolate failures. However, it seems to me that it is > be possible to solve this in Haskell where state can be serialized and > synchronized between multiple machines using Cloud Haskell, something that > is error-prone or impossible in other languages. But that step has never > been taken. > > Alexander > > > On Wed, Jul 10, 2013 at 4:48 PM, Alberto G. Corona <agocorona(a)gmail.com>wrote: > >> Thanks Adrian. The racket people where pioneers in this idea I think. >> >> There is another web framework in Ocaml, Osigen that it is also >> continuation based. MFlow is not continuation-based but it also define the >> navigation as a sequence. But only Seaside (and now MFlow) supports many >> flows in the same page. See for example this: >> >> [PDF] *Seaside* – A *Multiple* Control *Flow* Web Application Framework<http://www.google.es/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=3&cad=rja…> >> >> There is also other: Apache Coccoon that run in a special kind of >> JavaScript. The continuation-based frameworks have the reputation of >> storing a lot of application state and to be non scalable. MFlow >> uses backtracking and It does not have these problems. >> >> >> 2013/7/10 Adrian May <adrian.alexander.may(a)gmail.com> >> >>> Oh how nice! >>> >>> I have been looking at MFlow a lot lately and I think it's got something >>> quite special that Yesod, Happstack, etc don't seem to have, at least, not >>> as far as I know. I mean, look at this: >>> >>> sumWidget= pageFlow "sum" $ do >>> >>> n1 <- p << "Enter first number" ++> getInt Nothing <** submitButton "enter" <++ br >>> >>> >>> >>> >>> >>> n2 <- p << "Enter second number" ++> getInt Nothing <** submitButton "enter" <++ br >>> >>> >>> >>> >>> >>> n3 <- p << "Enter third number" ++> getInt Nothing <** submitButton "enter" <++ br >>> >>> >>> >>> >>> >>> p << ("The result is: "++show (n1 + n2 + n3)) ++> wlink () << b << " menu" >>> >>> >>> >>> >>> >>> <++ p << "you can change the numbers in the boxes to see how the result changes" >>> >>> Is that pretty or what? That's the code for this: >>> >>> http://mflowdemo.herokuapp.com/noscript/fviewmonad >>> >>> To me that's a real technological step over and above the usual servlets >>> paradigm and I'd love to see more people getting involved. It seems like >>> Yesod and Happstack have a lot more manpower behind them, but unless I've >>> missed something, MFlow is going somewhere new and should be helped along. >>> >>> Adrian. >>> >>> PS. Besides Seaside, Racket is playing with the same ideas. They (Jay >>> McCarthy) have something to say about performance but I didn't quite >>> understand it. >>> >>> >>> >>> On 10 July 2013 06:41, Alberto G. Corona <agocorona(a)gmail.com> wrote: >>> >>>> The third version of MFlow is out. >>>> >>>> http://hackage.haskell.org/package/MFlow >>>> >>>> MFlow is an all-heterodox web application framework, but very >>>> haskellish. >>>> >>>> Now MFlow support restful URLs. It is the first stateful web framework >>>> to my knowledge that supports it. The type safe routes are implicitly >>>> expressed as normal monadic code within a navigation monad. The application >>>> look as a normal imperative console application, but the navigation monad >>>> goes back and forth to match the path of the URL. The user has control of >>>> the state, that can roll-back or not when the navigation goes back >>>> depending on the application needs. The state is in the form of normal >>>> Haskell variables In a monadic computation, with the weird addition of >>>> backtracking. >>>> >>>> The menu of the application below is implemented as an imperative-like >>>> syntax, but the application navigate forward and backward to synchronize >>>> with the requests of the web browser: >>>> http://mflowdemo.herokuapp.com/ >>>> >>>> This version support in-page flows. >>>> What is that? look at this example: >>>> >>>> http://mflowdemo.herokuapp.com/noscript/fviewmonad >>>> >>>> These flows are implemented as formlets with a monad instance, and >>>> callbacks which change the look. I call them "widgets": >>>> >>>> >>>> http://haskell-web.blogspot.com.es/2013/06/the-promising-land-of-monadic-fo… >>>> >>>> >>>> Each page may have many of these active widgets, each one running >>>> their own flow. These widgets refresh themselves trough Ajax if they are >>>> enclosed in the primitive "autoRefresh". If there is no Ajax or JavaScript >>>> available, they gracefully degrade by refreshing the entire page: >>>> >>>> http://mflowdemo.herokuapp.com/noscript/combination >>>> >>>> >>>> http://haskell-web.blogspot.com.es/2013/06/and-finally-widget-auto-refreshi… >>>> >>>> The page flows and the multiflow idea was inspired in Seaside<http://www.seaside.st/>, >>>> a great Smalltalk web framework and adapted to the pure recursive nature of >>>> Haskell and the formlets. >>>> >>>> It also support some JQuery widgets integrated: modal and not modal >>>> dialogs, datePicker and other active widgets that handle other widgets. >>>> >>>> It also support the older features: persistent state, WAI, blaze-html >>>> and others integration, server process timeouts, Ajax, requirements, >>>> content management, caching of widget rendering and all the other >>>> previous stuff. >>>> >>>> I wish to thank some people for their feedback. Specially Adrian May >>>> for his feedback and interest >>>> >>>> >>>> -- >>>> Alberto. >>>> >>>> _______________________________________________ >>>> Haskell-Cafe mailing list >>>> Haskell-Cafe(a)haskell.org >>>> http://www.haskell.org/mailman/listinfo/haskell-cafe >>>> >>>> >>> >>> _______________________________________________ >>> Haskell-Cafe mailing list >>> Haskell-Cafe(a)haskell.org >>> http://www.haskell.org/mailman/listinfo/haskell-cafe >>> >>> >> >> >> -- >> Alberto. >> >> _______________________________________________ >> Haskell-Cafe mailing list >> Haskell-Cafe(a)haskell.org >> http://www.haskell.org/mailman/listinfo/haskell-cafe >> >> > -- Alberto.

1 1

Re: [web-devel] [Haskell-cafe] ANNOUNCE: MFlow 3.0
by Alberto G. Corona 13 Jul '13

13 Jul '13

I factored out the submit buttons. Now the three text boxes appear in succession above a unique button below. http://mflowdemo.herokuapp.com/noscript/fviewmonad This is the new code: sumWidget= pageFlow "sum" $ do n ← (do n1 ← p << "Enter first number" ++> getInt Nothing <++ br n2 ← p << "Enter second number" ++> getInt Nothing <++ br n3 ← p << "Enter third number" ++> getInt Nothing <++ br return (n1+ n2 + n3)) <** pageFlow "button" (submitButton "submit") p << ("The result is: "++show n) ++> wlink () << b << " menu" <++ p << "you can change the numbers in the boxes to see how the result changes" 2013/7/10 Adrian May <adrian.alexander.may(a)gmail.com> > Oh how nice! > > I have been looking at MFlow a lot lately and I think it's got something > quite special that Yesod, Happstack, etc don't seem to have, at least, not > as far as I know. I mean, look at this: > > sumWidget= pageFlow "sum" $ do > > n1 <- p << "Enter first number" ++> getInt Nothing <** submitButton "enter" <++ br > > n2 <- p << "Enter second number" ++> getInt Nothing <** submitButton "enter" <++ br > > n3 <- p << "Enter third number" ++> getInt Nothing <** submitButton "enter" <++ br > > p << ("The result is: "++show (n1 + n2 + n3)) ++> wlink () << b << " menu" > > <++ p << "you can change the numbers in the boxes to see how the result changes" > > Is that pretty or what? That's the code for this: > > http://mflowdemo.herokuapp.com/noscript/fviewmonad > > To me that's a real technological step over and above the usual servlets > paradigm and I'd love to see more people getting involved. It seems like > Yesod and Happstack have a lot more manpower behind them, but unless I've > missed something, MFlow is going somewhere new and should be helped along. > > Adrian. > > PS. Besides Seaside, Racket is playing with the same ideas. They (Jay > McCarthy) have something to say about performance but I didn't quite > understand it. > > > > On 10 July 2013 06:41, Alberto G. Corona <agocorona(a)gmail.com> wrote: > >> The third version of MFlow is out. >> >> http://hackage.haskell.org/package/MFlow >> >> MFlow is an all-heterodox web application framework, but very haskellish. >> >> Now MFlow support restful URLs. It is the first stateful web framework >> to my knowledge that supports it. The type safe routes are implicitly >> expressed as normal monadic code within a navigation monad. The application >> look as a normal imperative console application, but the navigation monad >> goes back and forth to match the path of the URL. The user has control of >> the state, that can roll-back or not when the navigation goes back >> depending on the application needs. The state is in the form of normal >> Haskell variables In a monadic computation, with the weird addition of >> backtracking. >> >> The menu of the application below is implemented as an imperative-like >> syntax, but the application navigate forward and backward to synchronize >> with the requests of the web browser: >> http://mflowdemo.herokuapp.com/ >> >> This version support in-page flows. >> What is that? look at this example: >> >> http://mflowdemo.herokuapp.com/noscript/fviewmonad >> >> These flows are implemented as formlets with a monad instance, and >> callbacks which change the look. I call them "widgets": >> >> >> http://haskell-web.blogspot.com.es/2013/06/the-promising-land-of-monadic-fo… >> >> >> Each page may have many of these active widgets, each one running their >> own flow. These widgets refresh themselves trough Ajax if they are enclosed >> in the primitive "autoRefresh". If there is no Ajax or JavaScript >> available, they gracefully degrade by refreshing the entire page: >> >> http://mflowdemo.herokuapp.com/noscript/combination >> >> >> http://haskell-web.blogspot.com.es/2013/06/and-finally-widget-auto-refreshi… >> >> The page flows and the multiflow idea was inspired in Seaside<http://www.seaside.st/>, >> a great Smalltalk web framework and adapted to the pure recursive nature of >> Haskell and the formlets. >> >> It also support some JQuery widgets integrated: modal and not modal >> dialogs, datePicker and other active widgets that handle other widgets. >> >> It also support the older features: persistent state, WAI, blaze-html and >> others integration, server process timeouts, Ajax, requirements, >> content management, caching of widget rendering and all the other >> previous stuff. >> >> I wish to thank some people for their feedback. Specially Adrian May for >> his feedback and interest >> >> >> -- >> Alberto. >> >> _______________________________________________ >> Haskell-Cafe mailing list >> Haskell-Cafe(a)haskell.org >> http://www.haskell.org/mailman/listinfo/haskell-cafe >> >> > > _______________________________________________ > Haskell-Cafe mailing list > Haskell-Cafe(a)haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe > > -- Alberto.

1 0

Re: [web-devel] [Haskell-cafe] ANNOUNCE: MFlow 3.0
by Alberto G. Corona 10 Jul '13

10 Jul '13

Thanks Adrian. The racket people where pioneers in this idea I think. There is another web framework in Ocaml, Osigen that it is also continuation based. MFlow is not continuation-based but it also define the navigation as a sequence. But only Seaside (and now MFlow) supports many flows in the same page. See for example this: [PDF] *Seaside* – A *Multiple* Control *Flow* Web Application Framework<http://www.google.es/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=3&cad=rja…> There is also other: Apache Coccoon that run in a special kind of JavaScript. The continuation-based frameworks have the reputation of storing a lot of application state and to be non scalable. MFlow uses backtracking and It does not have these problems. 2013/7/10 Adrian May <adrian.alexander.may(a)gmail.com> > Oh how nice! > > I have been looking at MFlow a lot lately and I think it's got something > quite special that Yesod, Happstack, etc don't seem to have, at least, not > as far as I know. I mean, look at this: > > sumWidget= pageFlow "sum" $ do > > n1 <- p << "Enter first number" ++> getInt Nothing <** submitButton "enter" <++ br > > n2 <- p << "Enter second number" ++> getInt Nothing <** submitButton "enter" <++ br > > n3 <- p << "Enter third number" ++> getInt Nothing <** submitButton "enter" <++ br > > p << ("The result is: "++show (n1 + n2 + n3)) ++> wlink () << b << " menu" > > <++ p << "you can change the numbers in the boxes to see how the result changes" > > Is that pretty or what? That's the code for this: > > http://mflowdemo.herokuapp.com/noscript/fviewmonad > > To me that's a real technological step over and above the usual servlets > paradigm and I'd love to see more people getting involved. It seems like > Yesod and Happstack have a lot more manpower behind them, but unless I've > missed something, MFlow is going somewhere new and should be helped along. > > Adrian. > > PS. Besides Seaside, Racket is playing with the same ideas. They (Jay > McCarthy) have something to say about performance but I didn't quite > understand it. > > > > On 10 July 2013 06:41, Alberto G. Corona <agocorona(a)gmail.com> wrote: > >> The third version of MFlow is out. >> >> http://hackage.haskell.org/package/MFlow >> >> MFlow is an all-heterodox web application framework, but very haskellish. >> >> Now MFlow support restful URLs. It is the first stateful web framework >> to my knowledge that supports it. The type safe routes are implicitly >> expressed as normal monadic code within a navigation monad. The application >> look as a normal imperative console application, but the navigation monad >> goes back and forth to match the path of the URL. The user has control of >> the state, that can roll-back or not when the navigation goes back >> depending on the application needs. The state is in the form of normal >> Haskell variables In a monadic computation, with the weird addition of >> backtracking. >> >> The menu of the application below is implemented as an imperative-like >> syntax, but the application navigate forward and backward to synchronize >> with the requests of the web browser: >> http://mflowdemo.herokuapp.com/ >> >> This version support in-page flows. >> What is that? look at this example: >> >> http://mflowdemo.herokuapp.com/noscript/fviewmonad >> >> These flows are implemented as formlets with a monad instance, and >> callbacks which change the look. I call them "widgets": >> >> >> http://haskell-web.blogspot.com.es/2013/06/the-promising-land-of-monadic-fo… >> >> >> Each page may have many of these active widgets, each one running their >> own flow. These widgets refresh themselves trough Ajax if they are enclosed >> in the primitive "autoRefresh". If there is no Ajax or JavaScript >> available, they gracefully degrade by refreshing the entire page: >> >> http://mflowdemo.herokuapp.com/noscript/combination >> >> >> http://haskell-web.blogspot.com.es/2013/06/and-finally-widget-auto-refreshi… >> >> The page flows and the multiflow idea was inspired in Seaside<http://www.seaside.st/>, >> a great Smalltalk web framework and adapted to the pure recursive nature of >> Haskell and the formlets. >> >> It also support some JQuery widgets integrated: modal and not modal >> dialogs, datePicker and other active widgets that handle other widgets. >> >> It also support the older features: persistent state, WAI, blaze-html and >> others integration, server process timeouts, Ajax, requirements, >> content management, caching of widget rendering and all the other >> previous stuff. >> >> I wish to thank some people for their feedback. Specially Adrian May for >> his feedback and interest >> >> >> -- >> Alberto. >> >> _______________________________________________ >> Haskell-Cafe mailing list >> Haskell-Cafe(a)haskell.org >> http://www.haskell.org/mailman/listinfo/haskell-cafe >> >> > > _______________________________________________ > Haskell-Cafe mailing list > Haskell-Cafe(a)haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe > > -- Alberto.

1 0

ANNOUNCE: MFlow 3.0
by Alberto G. Corona 09 Jul '13

09 Jul '13

The third version of MFlow is out. http://hackage.haskell.org/package/MFlow MFlow is an all-heterodox web application framework, but very haskellish. Now MFlow support restful URLs. It is the first stateful web framework to my knowledge that supports it. The type safe routes are implicitly expressed as normal monadic code within a navigation monad. The application look as a normal imperative console application, but the navigation monad goes back and forth to match the path of the URL. The user has control of the state, that can roll-back or not when the navigation goes back depending on the application needs. The state is in the form of normal Haskell variables In a monadic computation, with the weird addition of backtracking. The menu of the application below is implemented as an imperative-like syntax, but the application navigate forward and backward to synchronize with the requests of the web browser: http://mflowdemo.herokuapp.com/ This version support in-page flows. What is that? look at this example: http://mflowdemo.herokuapp.com/noscript/fviewmonad These flows are implemented as formlets with a monad instance, and callbacks which change the look. I call them "widgets": http://haskell-web.blogspot.com.es/2013/06/the-promising-land-of-monadic-fo… Each page may have many of these active widgets, each one running their own flow. These widgets refresh themselves trough Ajax if they are enclosed in the primitive "autoRefresh". If there is no Ajax or JavaScript available, they gracefully degrade by refreshing the entire page: http://mflowdemo.herokuapp.com/noscript/combination http://haskell-web.blogspot.com.es/2013/06/and-finally-widget-auto-refreshi… The page flows and the multiflow idea was inspired in Seaside<http://www.seaside.st/>, a great Smalltalk web framework and adapted to the pure recursive nature of Haskell and the formlets. It also support some JQuery widgets integrated: modal and not modal dialogs, datePicker and other active widgets that handle other widgets. It also support the older features: persistent state, WAI, blaze-html and others integration, server process timeouts, Ajax, requirements, content management, caching of widget rendering and all the other previous stuff. I wish to thank some people for their feedback. Specially Adrian May for his feedback and interest -- Alberto.

1 0