On the state of Haskell web frameworks

newer
Getting the http request header in...

older
Proposed changes to web-routes:...

Christopher Done

19 Sep 2010 19 Sep '10

10:49 a.m.

For what it's worth, I thought I'd throw a simple web app into the mix, hpaste.org: * Address http://hpaste.org/ * Source http://github.com/chrisdone/amelie The notable thing about this application is it's easy to see all the requirements ("small libraries"): 1. CGI/FastCGI: I needed a way to talk to the web, I chose CGI/FastCGI. The CGI library is quite general and small. 2. highlighting-kate: I needed to do syntax highlighting. Notice how it's just a few lines in an isolated module: http://github.com/chrisdone/amelie/blob/master/src/Amelie/Highlight.hs 3. Takusen: I needed to talk to a PostgreSQL database to save/read pastes, just a few lines (for setup) in an isolated module: http://github.com/chrisdone/amelie/blob/master/src/Amelie/DB.hs 4. ConfigFile: I needed a way to store the service's settings in a configuration file. Again, just a few lines in an isolated module: http://github.com/chrisdone/amelie/blob/master/src/Amelie/Config.hs 5. blaze-html/xhtml: I needed a way to build HTML inside my program. 6. formlets: I needed a way to generate and validate forms (e.g. the paste form). 5 and 6 are used in the HTML module: http://github.com/chrisdone/amelie/blob/master/src/Amelie/HTML.hs All of these libraries are perfect, they do one thing and do it well. I just imported them and they did the work for me. You can see also how they are modules in themselves, not interdependent as part of some grand framework. It seems that with good libraries, the architecture of your web app just designs itself. Personally I've always thought the "MVC" approach isn't particularly novel and follows naturally from good design. I'm not saying mine is good design, but without putting much conscious thought towards it, I've ended up with a DB module (model), a Pages module (controller) and a HTML module (view). To make this into the MVC approach, I'd just rename them to Model.hs, Controller.hs and View.hs. The things that I had to implement myself are as follows: 7. URI routing: http://github.com/chrisdone/amelie/blob/master/src/Amelie/Routes.hs How to receive requests and map that into a page + parameters to that page (e.g. paste/pid/30196 -> pastePage [("pid","30196")]), how to *generate* a URL with parameters. And how to generate a *pretty* URL: http://github.com/chrisdone/amelie/blob/master/src/Amelie/Links.hs A URL that maps back to a normal URL (e.g. http://hpaste.org/30196/half_a_proof is actually rewritten by nginx to http://hpaste.org/paste/id/30196). I think there probably are quite a few URI routing libraries out there but I only gave it a cursory glance. 8. Simple template writing. http://github.com/chrisdone/amelie/blob/master/src/Amelie/Templates.hs In fact I found that tibbe already wrote something to do this: http://hackage.haskell.org/package/template but I didn't end up using it for some reason. Maybe because everything has to be Text and I couldn't be bothered littering my code with string conversions all over the place. Turned out I had to do that anyway. Converting between String and ByteString and lazy ByteString and now Text is a real pain but you have to do it because all these different libraries decided to use a particular type. (At least some let you implement a Stringable class.) I will probably eventually just replace the templating code with tibbe's or go the full hog and use HStringTemplate. It's all the same idea, just more or less engineered. I'm interested in your thoughts and ideas. If you have any hpaste-size web applications that you have available online to read that you can maybe summarise and evaluate in the same way I have above, maybe that can spark some interesting discussion. Cheers!

Show replies by date

Jinjing Wang

19 Sep 19 Sep

1:05 p.m.

great start! my example: `maid` is a standalone web server i sometimes use as a remote browser ><

...

cabal install maid maid

there's only one source file

...

http://github.com/nfjinjing/maid/blob/master/src/maid.hs

a bit different approach though, as it uses middlewares extensively. I like to write apps pretty much the same way chris mentioned, i.e. using a lot of libraries under a very flat architecture. happy hacking :) -- jinjing

Jinjing Wang

1:20 p.m.

here's added comparable information

> 1. CGI/FastCGI:
maid uses hack as an interface, whatever, it kind of works for me :)

> 2. highlighting-kate:
-

> 3. Takusen:
raw file IO

> 4. ConfigFile:
-

> 5. blaze-html/xhtml:
moe as an html combinator library, i like the syntax :)

http://github.com/nfjinjing/moe


> 6. formlets:
-

> 7. URI routing:
the included cascade middleware.

> 8. Simple template writing.
the $here macro

-- 
jinjing

Nils Schweinsberg

4:50 p.m.

About one year ago, I started hacking on web projects with Haskell. One of my first web application was, just like hpaste, a simple paste bin: Website: http://npaste.de/ Source: http://github.com/mcmaniac/npaste.de npaste.de has currently over 3000 pastes and 13 registered users. You can use custom/random paste IDs, hide your paste or modify/delete them (if you have a user account). I did a lot of experiments in there and the source of npaste is therefor probably not very beautiful/easy to read. As my second "big" project I started just recently to turn my current "website" (a static index.html on n-sch.de) into a dynamic Haskell powered website with integrated blog, comment system etc. It's still in its early days, but the code can be found on: http://github.com/mcmaniac/n-sch.de Since I'm using happstack in all my projects (seems to be the most "complete" web framework to me) I released a small library "happstack-auth" on last friday, which is a high level library for user authentication/session managment in happstack applications. I wrote a very simple example website on how to use this library. It can be found on: Website: http://n-sch.de/happstack-auth Source: http://github.com/mcmaniac/happstack-auth/tree/master/demo/ My general approach for a new web project is:

...

1. CGI/FastCGI

Happstack for me. Setting up the happstack server is a simple "simpleHTTP appConf myWebapp" call, and happstack itself offers a lot of very nice functions to

...

2. highlighting-kate

Used it for npaste.de aswell, in combination with pandoc (markdown rendering).

...

3. Takusen

Since I'm using happstack, I'm using happstack-state as my primary "database". Setting up the data types for the state and creating the functions to manipulate them is sometimes a bit of a mess (since it involves some template haskell etc), but I think it's worth it: You can use real haskell data types instead of having to think about a database representation. In my latest "website-blog-project" I completly separated the state and the actual code of the website. Only a small, very nice API is returned: http://github.com/mcmaniac/n-sch.de/blob/master/src-lib/Blog.hs

...

4. ConfigFile

I usually just add a few lines to my Main.hs, so there is no big need for a config: http://github.com/mcmaniac/n-sch.de/blob/master/src/Main.hs#L24

...

5. blaze-html/xhtml

I used HSP for a while, but I think Blaze is way more beautiful to use: http://github.com/mcmaniac/happstack-auth/blob/master/demo/Templates.hs

...

6. formlets

Didn't know about this lib. :) Gotta try it one day.

...

7. URI routing

Again, happstack is doing this very nicely: http://github.com/mcmaniac/happstack-auth/blob/master/demo/Route.hs

...

8. Simple template writing

No template experience so far. I'd say, once you *know* which libraries to use, it's very easy and uncomplicated to create a new website with haskell. But since there is no complete framework, the learning curve is a bit steep at the beginning and you'll have to do a lot of experiments before you realize what libraries are actually usefull. By the way, did anyone use javascript on their websites? I wonder if there are any nice ways to combine those two worlds?

Marc Weber

20 Sep 20 Sep

2:50 a.m.

Excerpts from Nils Schweinsberg's message of Sun Sep 19 18:50:39 +0200 2010:

...

By the way, did anyone use javascript on their websites? I wonder if there are any nice ways to combine those two worlds?

Haskell: - http://github.com/sviperll/ghcjs (There is work in progress): JS backend for GHC. You have to find a way to compile the base packages. It was originally written for ghc-6.8.2 - YHC (york haskell compiler) has a JS backend - of course those DSL's you can find on hackage My guess is that Haskells laziness adds both: a runtime penalty and a code size penalty. I don't have proofs for that. But be aware that there is some competition (non Haskell): - http://www.impredicative.com/ur/ has some features of WASH. The urls have nicer names though. - http://www.itu.dk/people/mael/smltojs/ (ml to js) [ - HaXe.org (not such a strong type system - but it you can even create .swf files!) ] Marc Weber

Christopher Done

6:37 a.m.

On 20 September 2010 04:50, Marc Weber wrote:

...

Excerpts from Nils Schweinsberg's message of Sun Sep 19 18:50:39 +0200 2010:

...
By the way, did anyone use javascript on their websites? I wonder if there are any nice ways to combine those two worlds?

Haskell:

- http://github.com/sviperll/ghcjs (There is work in progress): JS backend for GHC. You have to find a way to compile the base packages. It was originally written for ghc-6.8.2

- YHC (york haskell compiler) has a JS backend

- of course those DSL's you can find on hackage

My guess is that Haskells laziness adds both: a runtime penalty and a code size penalty. I don't have proofs for that.

I've been thinking maybe a nicer route is to have a DSL which generates JavaScript or jQuery or some higher-level JS library. We don't need to compile *Haskell* code to JavaScript, I don't think -- we can merely have a good declarative description language in something like Applicative or an Arrow. I haven't tried it yet but I'll let you know how that goes.

Michael Snoyman

7:04 a.m.

On Mon, Sep 20, 2010 at 8:37 AM, Christopher Done wrote:

...

On 20 September 2010 04:50, Marc Weber wrote:

...
Excerpts from Nils Schweinsberg's message of Sun Sep 19 18:50:39 +0200 2010:

...
By the way, did anyone use javascript on their websites? I wonder if there are any nice ways to combine those two worlds?

Haskell:

- http://github.com/sviperll/ghcjs (There is work in progress): JS backend for GHC. You have to find a way to compile the base packages. It was originally written for ghc-6.8.2

- YHC (york haskell compiler) has a JS backend

- of course those DSL's you can find on hackage

My guess is that Haskells laziness adds both: a runtime penalty and a code size penalty. I don't have proofs for that.

I've been thinking maybe a nicer route is to have a DSL which generates JavaScript or jQuery or some higher-level JS library. We don't need to compile *Haskell* code to JavaScript, I don't think -- we can merely have a good declarative description language in something like Applicative or an Arrow. I haven't tried it yet but I'll let you know how that goes.

That sounds like a much better approach. I would be interested in such a library as well, so please keep us updated. Michael

Jeremy Shaw

1:29 p.m.

On Sep 20, 2010, at 1:37 AM, Christopher Done wrote:

...

I've been thinking maybe a nicer route is to have a DSL which generates JavaScript or jQuery or some higher-level JS library. We don't need to compile *Haskell* code to JavaScript, I don't think -- we can merely have a good declarative description language in something like Applicative or an Arrow. I haven't tried it yet but I'll let you know how that goes.

Have you looked at HJScript or jmacro? Any idea what you would do differently from those? This reminds me of another possible solution I thought of. Maybe the nicest solution (and a lot of work) would be a new language with a compiler that targets javascript but is designed to also interact with Haskell and be tolerable to Haskellers. Like HaXe, but different.. - jeremy

Christopher Done

1:59 p.m.

On 20 September 2010 15:29, Jeremy Shaw wrote:

...

Have you looked at HJScript or jmacro? Any idea what you would do differently from those?

I looked at HJScript a while ago. They're imitations of JavaScript, too low level, I don't want to write JavaScript at all. I could build ontop of HJScript, though, for a standard library. But I really would like to forget about JavaScript entirely a la GWT's Java and Cappuchino's Objective-C and work in some embedded interface description language.

...

This reminds me of another possible solution I thought of. Maybe the nicest solution (and a lot of work) would be a new language with a compiler that targets javascript but is designed to also interact with Haskell and be tolerable to Haskellers. Like HaXe, but different..

Personally I'd rather stay within Haskell. Anyway talk is cheap, I'll chat more about this particular idea once I've actually something substantial.

Jeremy Shaw

2:25 p.m.

Nice. There is definitely a gapping hole to fill at the moment. - jeremy On Sep 20, 2010, at 8:59 AM, Christopher Done wrote:

...

On 20 September 2010 15:29, Jeremy Shaw wrote:

...
Have you looked at HJScript or jmacro? Any idea what you would do differently from those?

I looked at HJScript a while ago. They're imitations of JavaScript, too low level, I don't want to write JavaScript at all. I could build ontop of HJScript, though, for a standard library. But I really would like to forget about JavaScript entirely a la GWT's Java and Cappuchino's Objective-C and work in some embedded interface description language.

...
This reminds me of another possible solution I thought of. Maybe the nicest solution (and a lot of work) would be a new language with a compiler that targets javascript but is designed to also interact with Haskell and be tolerable to Haskellers. Like HaXe, but different..

Personally I'd rather stay within Haskell.

Anyway talk is cheap, I'll chat more about this particular idea once I've actually something substantial.

Chris Eidhof

22 Sep 22 Sep

10:27 a.m.

Hi Chris, First of all: this is an excellent writeup, and very useful for future reference. Thanks a lot for sharing. On 19 sep 2010, at 12:49, Christopher Done wrote:

...

3. Takusen: I needed to talk to a PostgreSQL database to save/read pastes, just a few lines (for setup) in an isolated module: http://github.com/chrisdone/amelie/blob/master/src/Amelie/DB.hs

Would you use this for larger projects? I always feel that this is too low-level (coming from Rails). There is Michael's work on his persistence library, Kevin's work, and I'm also playing around in this area.

...

All of these libraries are perfect, they do one thing and do it well. I just imported them and they did the work for me. You can see also how they are modules in themselves, not interdependent as part of some grand framework. It seems that with good libraries, the architecture of your web app just designs itself. Personally I've always thought the "MVC" approach isn't particularly novel and follows naturally from good design. I'm not saying mine is good design, but without putting much conscious thought towards it, I've ended up with a DB module (model), a Pages module (controller) and a HTML module (view). To make this into the MVC approach, I'd just rename them to Model.hs, Controller.hs and View.hs.

That is great. Maybe, in larger projects the boundaries might not be so clear. I think design patterns are overrated for small projects, but become really beneficial once you scale up.

...

7. URI routing: http://github.com/chrisdone/amelie/blob/master/src/Amelie/Routes.hs How to receive requests and map that into a page + parameters to that page (e.g. paste/pid/30196 -> pastePage [("pid","30196")]), how to *generate* a URL with parameters.

And how to generate a *pretty* URL: http://github.com/chrisdone/amelie/blob/master/src/Amelie/Links.hs A URL that maps back to a normal URL (e.g. http://hpaste.org/30196/half_a_proof is actually rewritten by nginx to http://hpaste.org/paste/id/30196). I think there probably are quite a few URI routing libraries out there but I only gave it a cursory glance.

Have you looked at the web-routes package? Again, this might only be useful for more complex applications. The idea is that you use datatypes to encode all possible URLs, and linking is then done by building a value of that datatype: it is not possibly anymore to build invalid URLs. -chris

Christopher Done

11:58 a.m.

On 22 September 2010 12:27, Chris Eidhof wrote:

...

...
3. Takusen: I needed to talk to a PostgreSQL database to save/read pastes, just a few lines (for setup) in an isolated module: http://github.com/chrisdone/amelie/blob/master/src/Amelie/DB.hs

Would you use this for larger projects? I always feel that this is too low-level (coming from Rails). There is Michael's work on his persistence library, Kevin's work, and I'm also playing around in this area.

I'm using HaskellDB at work which automatically derives your tables into types. That's quite nice but it needs some work to be more usable as it's kind of cumbersome to program in (mostly with writing type annotations). I need to use PostgreSQL at work because other people need to be able to use my database in different languages and applications, I used it for Amelie because I'm used to it. Looks like the Persistence library has a PostgreSQL backend so I should try that out. It's pros and cons with Takusen: 1. I like that you have flexibility over implementing the left fold for Takusen, but also don't like that I have to do that. 2. I like that you have the flexibility to construct a value however you want, doing transformations and things, , tags = maybe [] (splitWhen (==',')) tags' but I also don't like having to write out every field I want in order to construct a value: makePaste pid' title' content' tags' author' lang' chan' created' an_of expires xs = DB.result' (paste:xs) where paste = Paste { pid = pid', ... This is also annoying: updatePaste :: Paste -> DBM mark Session () updatePaste Paste{..} = DB.execDDL (DB.cmdbind stmt params) where stmt = unwords ["update paste set" ,fieldSpec ,"where id = " ++ show pid] fieldSpec = intercalate "," $ map spec fields where spec (key,_) = key ++ " = ?" params = map snd fields fields = [("title",DB.bindP title) ,("content",DB.bindP content) ,("tags",DB.bindP $ intercalate "," tags) ,("author",DB.bindP author) ,("language",DB.bindP $ fmap lid language) ,("channel",DB.bindP $ fmap cid channel) ,("annotation_of",DB.bindP annotation_of)] It's both manual and not type checked. So what I'd want from a DB library is the ability to retrieve from the database type-correct value automatically, and write them or update them back, but be able to transform certain fields which don't make sense in the DB but make sense in Haskell. Likewise, for JSON, I have problems with my data types not making any sense for JSON, i.e., when I send a value from JavaScript to a Haskell server as JSON, I don't want to have to send the `id' field -- but this `id' is in my local data type because it's useful for the Haskell code. Likewise when rendering it to JSON it outputs the `id' field or some $foo field which shouldn't be shown. How do you remove old and add new fields to and from JSON/DB/XML/whatever without having to implement *everything* yourself? I also don't want to have to sacrifice the clarity of my Haskell data types in order to adhere to a particular representation of it (JSON, DB, etc.) just because I'm using deriving (Typeable,Data). How is your work going?

...

That is great. Maybe, in larger projects the boundaries might not be so clear. I think design patterns are overrated for small projects, but become really beneficial once you scale up.

That's true, as the project sizes increases so does the need for design consistency. I'm going to add an RSS feed to amelie, and this isn't a page. It's a another type of resource. In this way I should abstract Pages into Resources, but it's nice to have a common vocabulary and way of doing things so I'll update it to Controllers, Models, Views, etc. Formlets are an interesting one, because technically they combine the controller and the view for forms. In the end I decided it was a view and put it in the HTML module, because I have separate logic in the Pages module for getting the value of the form and then talking to the DB, etc. I found myself encoding some non-view-ish properties in the formlet, though. I suppose this is one of those cases where you just do your best to separate the view and the controller without sacrificing clarity and concision. What do you think about how formlets fit into the MVC pattern?

...

Have you looked at the web-routes package? Again, this might only be useful for more complex applications. The idea is that you use datatypes to encode all possible URLs, and linking is then done by building a value of that datatype: it is not possibly anymore to build invalid URLs.

This looks basically perfect as a replacement for my Routes and Links modules if a little involved. I'll look into it when I get home. Looks like I should be able to drop my custom modules entirely. I'll let you know how I get on with it. Cheers!

Chris Eidhof

1:17 p.m.

...

...
Have you looked at the web-routes package? Again, this might only be useful for more complex applications. The idea is that you use datatypes to encode all possible URLs, and linking is then done by building a value of that datatype: it is not possibly anymore to build invalid URLs.

This looks basically perfect as a replacement for my Routes and Links modules if a little involved. I'll look into it when I get home. Looks like I should be able to drop my custom modules entirely. I'll let you know how I get on with it.

There are also these two gists, which you may find interesting: http://gist.github.com/348425 http://gist.github.com/333769 These are the two different approaches I've played around with. -chris

5399

Age (days ago)

5402

Last active (days ago)

List overview

Download

12 comments

7 participants

participants (7)

Chris Eidhof
Christopher Done
Jeremy Shaw
Jinjing Wang
Marc Weber
Michael Snoyman
Nils Schweinsberg