May 2010 - web-devel - Haskell.org

Google Summer of Code: BlazeHTML RFC
by Jasper Van der Jeugt 03 Jun '10

03 Jun '10

Dear all, BlazeHtml started out on ZuriHac 2010. Now, Jasper Van der Jeugt is working on it as a student to Google Summer of Code for haskell.org. His mentors are Simon Meier and Johan Tibell. The goal is to create a high-performance HTML generation library. In the past few weeks, we have been exploring the performance and design of different drafts of this library. Now, the time has come to ask some questions to the Haskell community — more specifically the future users of BlazeHtml as well as current users of other HTML generation libraries. We have written an RFC to gather feedback from the community: HTML version: http://jaspervdj.be/posts/2010-05-27-blazehtml-rfc.html Plain version: http://github.com/jaspervdj/BlazeHtml/raw/develop/doc/RFC.lhs The easiest way of sending us feedback, comments or criticism is replying to the haskell-cafe thread here. Alternatively, drop a comment at the bottom of the HTML version or at reddit. Looking forward to your feedback, Kind regards, Simon Meier Jasper Van der Jeugt

9 14

Re: [Haskell-cafe] Re: [web-devel] Google Summer of Code: BlazeHTML RFC
by Jeremy Shaw 01 Jun '10

01 Jun '10

Hello, HSP does not use xhtml or any other library internally. The trhsx pre-processor turns this: bigTable :: [[Int]] -> String bigTable t = renderAsHTML $ evalIdentity $ <table> <% mapM (\r -> <tr><% mapM (\d -> <td><% show d %></td>) r %></tr>) t %> </table> into: bigTable :: [[Int]] -> String {-# LINE 38 "hsp-blaze.hs" #-} bigTable t = L.unpack $ B.renderHtml $ evalBlaze $ (genElement (Nothing, "table") [] [asChild ((asChild (mapM (\ r -> (genElement (Nothing, "tr") [] [asChild ((asChild (mapM (\ d -> (genElement (Nothing, "td") [] [asChild ((asChild (show d)))])) r)))])) t)))]) Basically, it just calls genElement to create the elements, and asChild is a class to turn things into things that can be children on an element. The question remains, though, what is the type that genElement is producing? The type is monad specific. genElement comes from this class, -- | Generate XML values in some XMLGenerator monad. class Monad m => XMLGen m where type XML m data Child m data Attribute m genElement :: Name -> [XMLGenT m [Attribute m]] -> [XMLGenT m [Child m]] -> XMLGenT m (XML m) genEElement :: Name -> [XMLGenT m [Attribute m]] -> XMLGenT m (XML m) genEElement n ats = genElement n ats [] xmlToChild :: XML m -> Child m pcdataToChild :: String -> Child m Most of the stuff in the HSX/HSP family now produces values of the type: data XML = Element Name Attributes Children | CDATA Bool String and you can use renderXML or renderAsHTML to convert that to a String depending on what rendering rules you want in effect. Anyway, on to the exciting stuff! The benchmarks: As I test I have implemented the big-table benchmark for four cases: 1. the existing html test in benchmarkes/bigtable 2. plain-old blaze 3. hsp using the identity monad, the XML type, and String 4. hsp using blaze html: mean: 15.86650 ms, lb 15.68054 ms, ub 16.22545 ms, ci 0.950 blaze: mean: 28.86730 ms, lb 28.45495 ms, ub 29.63139 ms, ci 0.950 hsp+identity: mean: 50.05748 ms, lb 49.45395 ms, ub 50.97315 ms, ci 0.950 hsp+blaze: mean: 166.4717 ms, lb 161.7957 ms, ub 174.1437 ms, ci 0.950 As a chart: http://chart.apis.google.com/chart?cht=bvg&chs=200x250&chd=t:29,50,166.5&ch… I am not entirely sure I did these tests correctly. So, we should not really trust these numbers at all for the moment. Also, the 'nf' thing did not have a lazy ByteString instance, so for the blaze stuff, I used L.unpack to turn the output back into a String -- which is not really 'fair'. (And would also be bad if the tests actually contained utf-8, and not just an ascii subset of it). That said, my first attempts at using blaze to speed up HSP do not seem to have worked :p I think that 'speeding up' HSX is going to be somewhat dependent on what you are trying to do with it. If you are just writing templates by hand, and the data you are adding in is String data, then I think the current HSX stuff is probably doing a descent job. Everything starts as String, and ends as a String. Then if you want you can convert that String to a ByteString, etc. The default instances for HSX would probably not perform as well if you had a lot of data you wanted to splice in that was already pre-encoded utf-8 bytestrings. If that is your case, though, you could just implement an alternative XML data type and matching monad, and use those with hsx instead. Though, you still have to be careful. It is not enough to have just utf-8 encoded bytestrings, they would also need to already have any special html characters escaped (such as &). You can't just read utf-8 values out of database and stick them straight into the html output. Another option might be to implement a instance of XMLGen that does not have an intermediate type -- it just goes directly to String, Text, ByteString, or whatever you want. I am not sure how much overhead the intermediate type is really causing though. One thing that slows the hsx html rendering down is that it has to check each tag and see if it is one of the special tags that has no close tag. (Such as meta, input, img, etc). Or if it is one of the tags where the content is rendered as as CDATA instead of PCDATA, such as script and style. And the only way to do that is to pattern match on the tag name. I have attached all three files I used to benchmark to this message. Certainly making hsx/hsp faster would be nice (though it doesn't seem especially slow to start with). And it would also be nice if you could use use a mixture of hsx+blaze combinators. - jeremy ps. I also added, ghc-options: -O2 to my BlazeHtml.cabal file, but that didn't seem to change anything On Sun, May 30, 2010 at 10:58 AM, Thomas Hartman <tphyahoo(a)gmail.com> wrote: > I'm a heavy hsp user. Could hsp benefit from this project by using > blaze as a back end instead of whatever it's using now? > > IIUC, Hsp uses hsx (via the preprocessor program trhsx) to convert > xml-containing hybrid hsp/xml/html files into compilable haskell. > > I expected hsx uses XHTML (which iiuc is what blaze would replace) on > the backend but I don't see it listed in the dependencies at > > http://hackage.haskell.org/package/hsx-0.7.0 > > so I guess it doesn't and uses something internal rather than xhtml. > (Right?) > > My question as regards BlazeHTML is if there could be any performance > win/tie in for the hsp/hsx toolchain. > > FWIW, wrt to blaze sclv commented on reddit "The idea is rather that > this would be a replacement for the html combinator library, as > distinct from templating (hamlet, hstringtemplate, bravo, chunks, > press, & co) and as distinct from the *sp model of inlined code (hsp). > > Edit: Ideally, and generally for Haskell libs, the choices of > persistence layer, html generation library, dispatch model, and server > layer are largely orthogonal. Strong typing makes any ad-hoc plumbing > a breeze." > > > http://www.reddit.com/r/haskell/comments/bxa0a/blazehtml_a_blazingly_fast_h… > > thomas. > > 2010/5/30 Tom Lokhorst <tom(a)lokhorst.eu>: > > +1 for HTML5. > > > > Also, I suggest focussing on the html serialization of HTML5. > > > > The xml serialization (XHTML5) is only useful in an XML environment. > > For such environments pure xml libraries are more appropriate. > > > > Besides, I like html syntax better. > > > > On 30 May 2010 16:27, Jochem Berndsen <jochem(a)functor.nl> wrote: > >> On 05/29/2010 08:05 PM, Gregory Collins wrote: > >>> > >>> Matt Parker<moonmaster9000(a)gmail.com> writes: > >>> > >>>> Q3: Which HTML version would you preferably use? > >>>> > >>>> HTML 5. google summer of code should be about pushing the new and > >>>> exciting. > >>> > >>> Yes, definitely, this should be the default IMO. > >> > >> +1 > >> > >> > >> -- > >> Jochem Berndsen | jochem(a)functor.nl > >> _______________________________________________ > >> Haskell-Cafe mailing list > >> Haskell-Cafe(a)haskell.org > >> http://www.haskell.org/mailman/listinfo/haskell-cafe > >> > > _______________________________________________ > > Haskell-Cafe mailing list > > Haskell-Cafe(a)haskell.org > > http://www.haskell.org/mailman/listinfo/haskell-cafe > > > _______________________________________________ > Haskell-Cafe mailing list > Haskell-Cafe(a)haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe >

1 1

Re: [Haskell-cafe] Re: [web-devel] Google Summer of Code: BlazeHTML RFC
by Jasper Van der Jeugt 31 May '10

31 May '10

Hey Thomas, Yes, hsx/hsp could get a performance gain by using BlazeHtml as a backend. However, I'm not sure if it is possible to change the backend without changing the user API (but I do hope so). Kind regards, Jasper Van der Jeugt On Sun, May 30, 2010 at 5:58 PM, Thomas Hartman <tphyahoo(a)gmail.com> wrote: > I'm a heavy hsp user. Could hsp benefit from this project by using > blaze as a back end instead of whatever it's using now? > > IIUC, Hsp uses hsx (via the preprocessor program trhsx) to convert > xml-containing hybrid hsp/xml/html files into compilable haskell. > > I expected hsx uses XHTML (which iiuc is what blaze would replace) on > the backend but I don't see it listed in the dependencies at > > http://hackage.haskell.org/package/hsx-0.7.0 > > so I guess it doesn't and uses something internal rather than xhtml. (Right?) > > My question as regards BlazeHTML is if there could be any performance > win/tie in for the hsp/hsx toolchain. > > FWIW, wrt to blaze sclv commented on reddit "The idea is rather that > this would be a replacement for the html combinator library, as > distinct from templating (hamlet, hstringtemplate, bravo, chunks, > press, & co) and as distinct from the *sp model of inlined code (hsp). > > Edit: Ideally, and generally for Haskell libs, the choices of > persistence layer, html generation library, dispatch model, and server > layer are largely orthogonal. Strong typing makes any ad-hoc plumbing > a breeze." > > http://www.reddit.com/r/haskell/comments/bxa0a/blazehtml_a_blazingly_fast_h… > > thomas. > > 2010/5/30 Tom Lokhorst <tom(a)lokhorst.eu>: >> +1 for HTML5. >> >> Also, I suggest focussing on the html serialization of HTML5. >> >> The xml serialization (XHTML5) is only useful in an XML environment. >> For such environments pure xml libraries are more appropriate. >> >> Besides, I like html syntax better. >> >> On 30 May 2010 16:27, Jochem Berndsen <jochem(a)functor.nl> wrote: >>> On 05/29/2010 08:05 PM, Gregory Collins wrote: >>>> >>>> Matt Parker<moonmaster9000(a)gmail.com> writes: >>>> >>>>> Q3: Which HTML version would you preferably use? >>>>> >>>>> HTML 5. google summer of code should be about pushing the new and >>>>> exciting. >>>> >>>> Yes, definitely, this should be the default IMO. >>> >>> +1 >>> >>> >>> -- >>> Jochem Berndsen | jochem(a)functor.nl >>> _______________________________________________ >>> Haskell-Cafe mailing list >>> Haskell-Cafe(a)haskell.org >>> http://www.haskell.org/mailman/listinfo/haskell-cafe >>> >> _______________________________________________ >> Haskell-Cafe mailing list >> Haskell-Cafe(a)haskell.org >> http://www.haskell.org/mailman/listinfo/haskell-cafe >> > _______________________________________________ > Haskell-Cafe mailing list > Haskell-Cafe(a)haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe >

1 0

Call for comments: Yesod 0.2
by Michael Snoyman 28 May '10

28 May '10

Hi all, I believe my 0.2 branch of Yesod[1] is feature-complete for this release, and it seems to be fairly stable. The documentation on it[2] give a good introduction, and hopefully the haddocks will fill in the gaps. At this point, I wanted to ask the community if they have any changes they'd like to see in this release. I'm planning on releasing some time next week. The main feature which is still prominently missing for this release is a persistence layer; hopefully that will be included in the next major release. If anyone has any ideas on this, I'm all ears. Thanks, Michael [1] http://github.com/snoyberg/yesod/tree/ver0.2 [2] http://docs.yesodweb.com/yesod/

7 10

Re: [Haskell-cafe] Google Summer of Code: BlazeHTML RFC
by Michael Snoyman 27 May '10

27 May '10

On Thu, May 27, 2010 at 11:40 AM, Ivan Miljenovic <ivan.miljenovic(a)gmail.com > wrote: > On 27 May 2010 18:33, Michael Snoyman <michael(a)snoyman.com> wrote: > > I don't do any string concatenation (look closely), I was very careful to > > avoid it. I tried with lazy text as well: it was slower. This isn't > > surprising, since lazy text- under the surface- is just a list of strict > > text. And the benchmark itself already has a lazy list of strict text. > Using > > lazy text would just be adding a layer of wrapping. > > I don't know what you mean by "explicitly using Text values"; you mean > > calling pack manually? That's really all that OverloadedStrings does. > > You can try out lots of different variants on that benchmark. I did that > > already, and found this to be the fastest version. > > Fair enough. Now that I think about it, I recall once trying to have > pretty generate Text values rather than String for graphviz (by using > fullRender, so it was still using String under the hood until it came > time to render) and it too was much slower than String (unfortunately, > I didn't record a patch with these changes so I can't just go back and > play with it anymore as I reverted them all :s). > > Maybe Bryan can chime in with some best-practices for using Text? > > Here's my guess at an explanation for what's happening in my benchmark: text will clearly beat String in memory usage, that's what it's designed for. However, the compiler is still generating String values which are being encoded to Text as runtime. Now, this is the same process for bytestrings. However, bytestrings never have to be decoded: the IO routines simply read the character buffer. In the case of text, however, the encoded data must be decoded again to a bytestring. In other words, here's what I think the three different benchmarks are really doing: * String: generates a list of Strings, passes each String to a relatively inefficient IO routine. * ByteString: encodes Strings one by one into ByteStrings, generates a list of these ByteStrings, and passes each ByteString to a very efficient IO routine. : Text: encodes Strings one by one into Texts, generates a list of these Texts, calls a UTF-8 decoding function to decode each Text into a ByteString, and passes each resulting ByteString to a very efficient IO routine. In the case of ASCII data to be output as UTF-8, uses the Data.ByteString.Char8.pack function will most likely always be the most efficient choice, and thus it seems like something BlazeHtml should support. I'm considering releasing a Hamlet 0.3 based entirely on UTF-8 encoded ByteStrings, but I'd also like to hear from Bryan about this. Michael

2 1

Re: [Haskell-cafe] Google Summer of Code: BlazeHTML RFC
by Michael Snoyman 27 May '10

27 May '10

On Thu, May 27, 2010 at 11:16 AM, Ivan Miljenovic <ivan.miljenovic(a)gmail.com > wrote: > On 27 May 2010 17:55, Michael Snoyman <michael(a)snoyman.com> wrote: > > Two comments: > > * The exclamation point seems good enough for attributes. I copied that > for > > Hamlet as well. > > * If you're standardizing on UTF-8, why not support bytestrings? I'm > aware > > that a user could shoot him/herself in the foot by passing in non-UTF8 > data, > > but I would imagine the performance gains would outweigh this. My recent > > benchmarks on the BigTable benchmark[1] imply a huge performance gap > between > > ByteStrings and other contenders. > > Wow, I find it rather surprising that String out-performs Text; any > idea why that is? I wonder if you're just using it wrong... > > Could be, I'd be very happy if that were the case. All of the benchmarks are available on Github, and the bytestring[1], text[2] and string[3] versions are all rather short. Michael [1] http://github.com/snoyberg/benchmarks/blob/master/bigtable/cgi/bytestring.hs [2] http://github.com/snoyberg/benchmarks/blob/master/bigtable/cgi/text.hs [3] http://github.com/snoyberg/benchmarks/blob/master/bigtable/cgi/string.hs

2 2

Re: [Haskell-cafe] Google Summer of Code: BlazeHTML RFC
by Michael Snoyman 27 May '10

27 May '10

On Thu, May 27, 2010 at 11:28 AM, Ivan Miljenovic <ivan.miljenovic(a)gmail.com > wrote: > On 27 May 2010 18:23, Michael Snoyman <michael(a)snoyman.com> wrote: > > > > > > On Thu, May 27, 2010 at 11:16 AM, Ivan Miljenovic > > <ivan.miljenovic(a)gmail.com> wrote: > >> > >> On 27 May 2010 17:55, Michael Snoyman <michael(a)snoyman.com> wrote: > >> > Two comments: > >> > * The exclamation point seems good enough for attributes. I copied > that > >> > for > >> > Hamlet as well. > >> > * If you're standardizing on UTF-8, why not support bytestrings? I'm > >> > aware > >> > that a user could shoot him/herself in the foot by passing in non-UTF8 > >> > data, > >> > but I would imagine the performance gains would outweigh this. My > recent > >> > benchmarks on the BigTable benchmark[1] imply a huge performance gap > >> > between > >> > ByteStrings and other contenders. > >> > >> Wow, I find it rather surprising that String out-performs Text; any > >> idea why that is? I wonder if you're just using it wrong... > >> > > Could be, I'd be very happy if that were the case. All of the benchmarks > are > > available on Github, and the bytestring[1], text[2] and string[3] > versions > > are all rather short. > > Does using lazy Text values improve this? I find it a little strange > that you concatenate so many individual Strings that much. Also, how > about explicitly using Text values rather than OverloadedStrings? > > I don't do any string concatenation (look closely), I was very careful to avoid it. I tried with lazy text as well: it was slower. This isn't surprising, since lazy text- under the surface- is just a list of strict text. And the benchmark itself already has a lazy list of strict text. Using lazy text would just be adding a layer of wrapping. I don't know what you mean by "explicitly using Text values"; you mean calling pack manually? That's really all that OverloadedStrings does. You can try out lots of different variants on that benchmark. I did that already, and found this to be the fastest version. Michael

1 0

ANNOUNCE: yesod 0.2.0
by Michael Snoyman 25 May '10

25 May '10

Hi all, I'm very happy to announce the release of yesod 0.2.0. Yesod is a Haskell web framework for type-safe, RESTful web applications. I won't bore you all with the full release announcement here, please see the blog[1]. Installing yesod should be as simple as: cabal update && cabal install yesod. Please note that due to usage of type families in Template Haskell code, this release requires GHC 6.12. To get started with Yesod, I recommend going through the tutorials on the documentation site[2]. Please let me know if anything is unclear, or if you find any bugs. The next feature I'm targeting for Yesod is persistence. If anyone has any thoughts to share on this topic, please send me an e-mail. Michael [1] http://www.snoyman.com/blog/entry/yesod-0-2-0-released/ [2] http://docs.yesodweb.com/yesod/

8 20

Web.Encodings + Hack - Not sure what I'm doing wrong.
by Clint Moore 18 May '10

18 May '10

The project that I'm working on requires user image galleries. The idea is to allow them to upload images via a form, do some rudimentary processing on the uploaded data and then send it directly to Amazon's S3 service. The problem that I'm running in to is that I can't seem to get the binary data from the post request correctly. When written out to disk with withBinaryFile (hopefully thereby bypassing the new encoding pitfalls in ghc 6.12.x), file(1) says it's 'data'. This seems to be happening if I push it to a new object on S3 as well. I'm secretly hoping I'm doing something obviously wrong here. The code in question is at http://gist.github.com/404636 . Hopefully it's obvious that it's way more frantic what-the-hell-is-going-on-here code than some crackpot's idea of production ready code :) Thanks, web-devel!

3 10

ANNOUNCE: hamlet 0.2.0
by Michael Snoyman 11 May '10

11 May '10

Hi all, I'm happy to announce the second major release of Hamlet[1]. Hamlet is a HTML templating library which works via quasi-quoting, giving you compile-time assurances, type safety and very efficient HTML generation. I wrote a blog post[2] describing the changes in this version of Hamlet. The two most major are reversing of identifier order in references and dropping the template argument. Combined, these changes make a Hamlet template fit in better with Haskell code. All the other changes are noted in that same blog post. The Hamlet documentation[3] has been updated to reflect all changes. Please let me know if the documentation needs clarification on any points. Cheers, Michael [1] http://hackage.haskell.org/package/hamlet-0.2.0 [2] http://www.snoyman.com/blog/entry/hamlet-version-0-2/ [3] http://docs.yesodweb.com/hamlet/

2 1