- web-devel - Haskell.org

gitit performance: happstack vs yesod
by John MacFarlane 07 Jul '12

07 Jul '12

I've been working on a port of gitit to Yesod, which is currently called gitit2 (https://github.com/jgm/gitit2) At this point I have implemented most of the basic wiki functionality. I've really been enjoying some of Yesod's features -- especially the type-safe URLs, form support, internationalization, and widgets -- and the code is much cleaner and shorter than the original gitit code. But today I thought I'd look at performance, and I was surprised to find that gitit2 is considerably slower than the original gitit (built on happstack). For example, on one simple test page, siege recorded 9 transactions per second for gitit2, 18 for the original gitit. (This is with caching turned off, so each hit involves getting content from the git repository, parsing it, and creating an HTML wiki page. Of course most of that work is done by filestore and pandoc, which are used by both gitit and gitit2.) I'm really curious why gitit2 is so much slower, and I wonder whether I'm making some basic mistake. At this point most of the code is in one module, https://github.com/jgm/gitit2/blob/master/Network/Gitit2.hs . This defines a Yesod subsite which is wrapped by a Yesod site in the main executable, https://github.com/jgm/gitit2/blob/master/src/gitit2.hs . The benchmark I did just involved fetching the front page, so most of the handlers are irrelevant. If anyone has any useful ideas, I'd be grateful. John

1 2

Re: [web-devel] http-types: Request for comments for major release
by Kazu Yamamoto 06 Jul '12

06 Jul '12

Hello Aristid, > I have decided to deprecate the current header functions, and expose new > constants like hContentLength = "Content-Length" instead. I guess this is a best solution. > I'm very open to pull requests for more headers though! So if you often need a > header, please just add it to the file and send me a pull request. The only > headers I won't add are non-standard headers. I have tried to use http-types v0.7. Here are missing headers for me: https://github.com/kazu-yamamoto/wai-app-file-cgi/blob/master/Network/Wai/A… I guess that Server: and Location: should be defined in http-types since they are used often. (This is not a strong opinion.) --Kazu

1 0

Happstack Irregular News Issue #2
by Jeremy Shaw 02 Jul '12

02 Jul '12

Hello, I pleased to announce that Happstack Irregular News Issue #2 is now available. You can view it online at: http://www.happstack.com/C/ViewPage/12 Or continue reading the rest of this message. Happstack Irregular News Issue #2 ================================= Hello! It is I, your editor Jeremy Shaw. I am pleased to bring you Happstack Irregular News Issue #2. Some exciting things have happened since the last issue! clckwrks -------- The biggest news since the last issue is the release of clckwrks: [clckwrks](http://www.clckwrks.com/) clckwrks is a Haskell-based blog and CMS framework with support for editing pages via the browser plus downloadable themes and plugins. clckwrks is now powering happstack.com and clckwrks.com. We are currently focusing on making the clckwrks blogging portion solid. We have moved the official Happstack blog to clckwrks in order to encourage us to make it better :) If you want to help out, you can [browse our bug list](http://www.clckwrks.com/B/Timeline) and find something to take action on. We are more than happy to provide guidance and other assistance. reform ------ The other new big release was [reform](http://www.happstack.com/C/ViewPage/11). `reform` is a form generation library that continues in the footsteps of `formlets` and `digestive-functors <= 0.2`. `digestive-functors 0.3` has gone off to explore a different direction, and we wanted to continue pushing the development in this direction. There are still many ideas we can share between the two libraries. Two changes we want to make in the next release include: 1. switch to `Bifunctors` package instead of homebrewed `IndexedApplicative` (thanks to Leonid Onokhov for pointing that out). (Another alternative might be `index-core`, though it does not yet export the `Applicative` instances). 2. consider using a `Free Applicative` / `Operational Applicative` for implementing the `reform` applicative instances. `digestive functors 0.3` does something like this and Jasper Van der Jeugt said it was very beneficial and we should try it in `reform` as well. happstack-yui ------------- Dag Odenhall has released `happstack-yui`, which makes it easy to use YUI with Happstack. According the YUI website: "YUI is a free, open source JavaScript and CSS framework for building richly interactive web applications." [happstack-yui](http://www.happstack.com/docs/happstack-yui-7351.4.1/doc/htm… [YUI website](http://yuilibrary.com/) HSX --- Niklas Broberg and I (Jeremy Shaw) did some work on HSX. It now builds with GHC 7.4 and we also fixed some hidden bugs in `HSX.Transform`. One thing we have been experimenting with is a `QuasiQuoter` for HSX. A demo version can be found here: darcs get http://src.seereason.com/hsx-qq/ The QQ provides an alternative to the `trhsx` preprocessor and allows you to write things like: html :: (XMLGenerator m) => XMLGenT m (XMLType m) html = [hsx| <p class="foo"><% map toUpper "hello, world!" %></p> |] This should be included in the next release of HSX. The next release of HSX will also contain a major refactoring of the packages. Mostly we are just planning to move modules into different packages and divide things up differently. One major benefit of the new arrangement is that you will no longer be required to install `HJavaScript` and `HJScript` even though you probably never use them. Other Minor Fixes ----------------- * changed types in `happstack-lite` so that `serveFile` and `asContentType` work better together, and added `guessContentType`, `MimeMap`, `mimeType` * patched `happstack-jmacro` to work with older versions of `template haskell` * tweaks to `ixset.cabal` so that it does not require the latest `Cabal` to build acid-state and hackage2 ----------------------- I have started research into why hackage2 requires so much RAM to run. I will be blogging about that separately. I do expect that we can substantially reduce that amount of RAM it requires. So far I have uncovered two minor issues: 1. it turns out that `mapM Lazy.readFile fileList` returns the file contents lazily but opens all the files immediately. This means you can run out of file descriptors if you have a lot of checkpoints or event files. A patch has been submitted for `acid-state` and it will be fixed in the next release. 2. `acid-state` reads the entire checkpoint file into RAM before decoding it. There are a couple places in the code that cause this to happen. The first place is in `cereal`. The `getLazyByteString` function does return a lazy `ByteString`.. but it does it by first reading a strict `ByteString` of the required length and then converting it into a lazy `ByteString`. Changing the behavior of `getLazyByteString` is actually quite difficult, as `cereal` was designed to allow for value-level error handling, instead of throwing async exceptions. We can probably work around this by using `runGetState` to get one-chunk at a time and build the lazy `ByteString` that way. That might actually be a lot less hackish than it sounds at first, because it allows us to explicity detect and handle failure cases and control how much and when things are read into RAM. Though, at that point, it starts to feel a bit like enumerators/iteratee/etc. Perhaps we will switch to `pipes` at some point in time. `pipes` provides streaming for pure (non-IO) values -- which is probably what we want here. ELM --- Evan Czaplicki has been doing a ton of work on ELM recently. As described on the [ELM Language Homepage](http://elm-lang.org/): "Elm is a type-safe, functional reactive language that compiles to HTML, CSS, and JavaScript." It is easy to use ELM with Happstack -- no special support is required. (i.e., we do not need `happstack-elm`). Vincent Ambo has created a simple demo here: [elm-happstack demo](https://github.com/tazjin/Elm/tree/master/Examples/elm-happstack) web-routes + Hamlet ------------------- Vincent also wrote a nice blog post showing how to combine `web-routes` (type-safe URL routing) with `Hamlet` (a `QuasiQuoter` for generating `blaze-html` from HTML-like syntax): [Best of both worlds: Using Hamlet and web-routes with Happstack](http://tazj.in/en/1335123720) quasi-quoter for language-css ----------------------------- `JMacro` is great for creating JavaScript, but we still have a hole when it comes to generating CSS. The [language-css](http://hackage.haskell.org/package/language-css) library already contains combinators and a syntax ADT for CSS3. If it had a parser, then we could also create a syntax-checking `[css| |]` `QuasiQuoter`. I have discussed the idea with Anton Kholomiov, and he is interested -- but we could use some one else to help write the parser. If you love writing parsers, this should be a fun little project. happstack.com theme ------------------- Finally, if you could suggest one thing that would make the happstack.com website nicer that would be awesome. There are four things we already plan to change: 1. use black on white text instead of gray on white 2. fix the paragraph width so that paragraphs are around 45em wide. 3. fix the grid alignment so that things are properly aligned to the grid 4. add more dates to the pages so that it clear that the site and project is still active If you have other suggestions, we would love to hear them! If you want to hack on the theme directly, that is even better! Until next time, happy hacking. - Jeremy Shaw

3 2

http-types: Request for comments for major release
by Aristid Breitkreuz 02 Jul '12

02 Jul '12

Hello Haskellers, I would like to prepare a new major release of http-types, for the following reasons: - Removing functions that have been deprecated for months. - I have added a new module QueryLike, which should make it much easier to create Query objects (it is modelled after recurring patterns in my usage). - Possibly: An opportunity to split the big module Network.HTTP.Types into smaller modules (of course still exposing all functionality at the well-known Network.HTTP.Types module). - Add new functionality that the community (you!) wants to see. So the first thing I'd like to ask of you is to please remove deprecated functionality and/or create strict upper bounds. The upcoming version will be 0.7, and other than removing deprecated functions, I plan to keep the API fully compatible. The other thing is: Are there things you'd like to see added? Does http-types as it is have deficiencies that you'd like to see addressed? And to the Yesod maintainers: As Yesod is right now the single biggest user of http-types (with a relatively long release cycle), would it be OK for you to relax the bounds in the upcoming release to http-types >=0.6.11 && <0.8 (assuming you don't use deprecated functions), even when http-types 0.7 is not released yet? Cheers, Aristid

3 2

Re: [web-devel] http-types: Request for comments for major release
by Michael Snoyman 01 Jul '12

01 Jul '12

On Sun, Jul 1, 2012 at 5:23 AM, Aristid Breitkreuz <aristidb(a)gmail.com> wrote: > Hi, > > I'd like to deprecate the Ascii type, and use ByteString directly, to > simplify the module structure. Any objections? Well, Ascii is just a type synonym, so all you'd really be doing is changing around the type signatures. From a compatibility standpoint, a library could easily be compliant with both the old and new versions (either by switching to using ByteString everywhere, or if you still export the Ascii type synonym, using there). In other words, no objections. Michael > > 2012/6/30 Michael Snoyman <michael(a)snoyman.com> >> >> >> What's the timeframe on the http-types release? It might be that it >> fits in perfectly with the upcoming wai release (probably in the next >> two weeks or so). Either way, it's not a problem to bump max versions >> mid-cycle. If the code can work unmodified with either version, that's >> a really easy change. But even breaking changes can be accomodated >> (with a bit of pain) using conditional compiling; that's what we ended >> up doing for blaze-html 0.5. > > > > It depends on whether anybody has a feature request. If not, it might be > done in less than 2 weeks. > > >> >> I think that splitting up into a few submodules and keeping a master >> module re-exporting everything is a good move. And of course, removing >> deprecated functions is a good thing too. Otherwise, I don't have much >> of an opinion here, http-types has been one of those "it works and I >> don't have to think much about it" packages :). > > > Already changed in the Github. Also glad to hear that http-types "just > works" for you. :) > > > Aristid

1 0

mongodb test issue and nested models
by Richard Plangger 26 Jun '12

26 Jun '12

Hi there, i'm having 2 problems, maybe someone can help me: 1) first I would like to write some tests for my web proj. but 'yesod test' does not run due to a compile time error. i get the same error like on this issue but i got no clue how to fix it: https://github.com/yesodweb/yesod/issues/349 2) second I dunno if my approach is correct. I'm using mongodb and i have entities like so: A name Text B name Text as [A] As it compiles like a charm and i can add A types to B entities, these A entities do not have an _id field. This seems logical to me due to the type A does not have this _id property generated, like described in the book and i do not 'insert' the A type but simply update a B entity. I would like to identify (not by index) these nested A entities as i'm going to delete/update them from time to time. Can you point out the way how I would do that? I also searched alot for more docu on the mongoDB persistent api, is there any? -- richard

2 1

FilePath in WAI
by Kazu Yamamoto 25 Jun '12

25 Jun '12

Hello, To implement high-performance web servers, I would like to remove overhead of String and want to use ByteString instead. Unfortunately, WAI is using FilePath (String) in ResponseFile. ResponseFile Status ResponseHeaders FilePath (Maybe FilePart) To my experience, the third argument is created from rawPathInfo (ByteString) in many cases. So, using FilePath is overhead. Since new Haskell Platform is released, I think I can implement new simple-sendfile which provides Network.Sendfile.ByteString: sendfile :: Socket -> RawFilePath -> FileRange -> IO () -> IO () Note that RawFilePath is a type synonym of ByteString. What do you think of this? A1) It's too late. A2) We can change FilePath to RawFilePath in ReponseFile. A3) We can provide a new constructor: ReponseRawFile A4) Others --Kazu

7 13

simple-sendfile v0.2.4
by Kazu Yamamoto 25 Jun '12

25 Jun '12

Hello, I have released simple-sendfile v0.2.4. In short, this version is 100+ times faster than v0.2.3 in some cases. I always did benchmark mighttpd with high concurrency. That is, I always make multiple connections at the same time. It gives me a good result. However, when I set the number of concurrency to 1, I found mighttpd is unbearablely slow on Linux and FreeBSD. I realized that this is because that warp uses sendMany (the writev() syscall) for a header and sendfile (the sendfile () syscall) for a body (ie. a file). They are sent in separate TCP packets. simple-sendfile v0.2.4 provides a new function called sendfileWithHeader. On Linux, it uses the send() syscall with the MSG_MORE flag to *store* a header and the sendfile() syscall to send both the stored header and a file. On FreeBSD, it uses the sendfile() syscall with the header arguments to send both a header and a file. This trick ensures that both a header and a body is sent in a single TCP packet. On Mac, v0.2.3 sends separate packets but is fast. I don't know why. v0.2.4 uses the same logic of FreeBSD. Unfortunately, the header and the body is also sent separately but it is still fast. Again, I don't know why. Here is the result of benchmark: OLD+pronk NEW+pronk OLD+httperf NEW+httperf Mac 3678.74 3415.76 6642.5 6891.8 FreeBSD 10.0010 1309.43 10.0 3174.8 # this machine is slow Linux 24.9875 2558.36 25.0 8284.2 OLD means v0.2.3. NEW means v0.2.4. pronk is "pronk -c 1 -n 1000 -r 10000 -b http://127.0.0.1:8000/" httperf is "httperf --hog --num-conns 1 --num-calls 1000 --rate 10000 --server localhost --port 8000 --uri /" I have already sent a pull request to warp to use sendfileWithHeader: https://github.com/yesodweb/wai/pull/86 P.S. I also found that the network package cannot find IPV6_V6ONLY macro on FreeBSD because the test code does not include "netinet/in.h". I will sent a pull request to fix this later. --Kazu

3 5

Re: [web-devel] FilePath in WAI
by Vagif Verdi 15 Jun '12

15 Jun '12

Switching to ByteString does not break windows applications. WAI can be perfectly used on windows for development, testing, even production use unless you are building next Facebook :) In order to see considerable degradation in performance from switching to ByteString on windows, your web application really need to have a huge traffic. But then again why would you host it on windows in this case ? On Friday, June 15, 2012 03:03:06 PM Thomas Hartman wrote: > I don't go anywhere near windows, but I disagree anyway. > > Making it possible for windows users to at least test this out is a > potentially significant factor in adoption. > > The other frameworks play nicely with windows. WAI should too. > > On Fri, Jun 15, 2012 at 3:00 PM, Vagif Verdi <vagif.verdi(a)gmail.com> wrote: > > Considering that we are talking about the web server, it would be safe to > > assume that most people would not run it on windows for production use. > > > > So i say WAI should switch to ByteString, and let the windows users be > > penalized just a little bit. You can't please everyone. > > > > On Friday, June 15, 2012 02:47:15 PM Johan Tibell wrote: > >> On Fri, Jun 15, 2012 at 2:41 PM, Greg Weber <greg(a)gregweber.info> wrote: > >> > The ticket would be to switch the standard libraries to use > >> > system-filepath, which probably requires a mailing list discussion > >> > somewhere, a lot of boring changes of code, and a more inconvenient > >> > API for end users. I will wait for Text to become standardized first > >> > > >> > :) > >> > >> The FilePath issue has been discussed at length before. The problem is > >> that different OS:es define file paths to be different things (i.e. > >> Unicode or bytes.) The FilePath type somehow has to bridge that, that > >> will be difficult. My main grip with FilePath right now is that it's > >> not an abstract type. > >> > >> -- Johan > >> > >> _______________________________________________ > >> web-devel mailing list > >> web-devel(a)haskell.org > >> http://www.haskell.org/mailman/listinfo/web-devel > > > > _______________________________________________ > > web-devel mailing list > > web-devel(a)haskell.org > > http://www.haskell.org/mailman/listinfo/web-devel

1 0

ANN: clckwrks - a Haskell blogging and CMS framework
by Jeremy Shaw 12 Jun '12

12 Jun '12

Hello! I am pleased to announce the launch of clckwrks.com. clckwrks (pronounced, 'clockworks') is a new Haskell-based content management (CMS) and blogging framework: http://www.clckwrks.com/ It is freely available under the BSD3 license. clckwrks is still in early alpha development. We are looking for developers who want to contribute or just like to try bleeding edge software. clckwrks is already being used to power happstack.com and clckwrks.com -- so it is already useful. There are a lot of things we are not happy with yet ranging from code formatting to design issues. But, at the same time, there are a lot of things we are very happy with. If you would like to help get us to clckwrks 1.0, we'd love to have your help! For more information, check out the website and consider joining the mailing list. Thanks! - jeremy

1 0