On Sat, Mar 26, 2011 at 1:32 PM, Michael Snoyman wrote:

Hi all,

JP Moresmau recently sent me an email asking for recommendations on hosting a Yesod app. I think a number of you have heard me mention the idea of setting up a WAI web hosting system (code named Lambda Engine). I think it's time to start hitting this idea hard, and I'd like community involvement as much as possible.

I think this is an interesting idea but hosting Haskell apps is basically just providing an OS with open ports on which they can run their http or fcgi servers. ... At the other extreme, we could simply provide a set of shell scripts

for setting up a vanilla Ubuntu (or Debian, or Cent, or FreeBSD...) install to support compiling a WAI application, provide an nginx setup, etc. The user would be responsible- and have the freedom to- set up his/her own server however he/she wants.

This seems like a better idea. IMO, it doesn't matter where we host, the OS images are what matter. I'm a security guy by trade, so my app servers will never have any type of compiler on them when I host in production, and I would prefer all of my apps to be statically linked. The thing that people new to web hosting suffer from is that there are approximately 1 billion ways to configure and host. Some ways are better than others, so I think we could provide: * A hardened OS image that understands our required configuration and provides no other services. My recommendation would be Turn Key Linux [3] [4] * A deployment script (private key auth based) which would push out new binaries, swap and bounce them. * A set of *-server.hs that understand our set of server configurations** If someone had these items, they could take their game to any hosting company in town that supported VMs on custom ISOs. We could probably even come up with a Ubuntu or RedHat hardening script for those that don't want to use VMs/ISOs. There are lots of in-between points on that spectrum. For example, I

was thinking of perhaps providing a central compilation server that everyone could use. In my experience, Haskell applications take very few resources to run, but actually building the things take a while. Compiling on a local system and then uploading is painful because (1) you have to deal with architecture issues and (2) you might have a slow upload speed.

I see no reason why a person couldn't build on their own machine, but if they don't have the resources, we could certainly set up a micro share on EC2 for that purpose. Making a secure build environment where they send over the code and it gets turned into a binary could be done.. but there is a *huge* amount of mutual trust going on. I wouldn't want to be involved at either end. ... Anyway, I'm very interested to hear what other people think about all

this. I'll enumerate a bunch of random thoughts on this topic:

* Should this be a for-profit venture, bankrolled by a few members, or a community, non-profit approach? (I'm favoring the latter.)

Using my recommended approach, the cost would be nominal.

* If we decide to provide some serious hosting power, we might be able to make a deal with a hosting provider, stating that we're representing the "entire Haskell web community".

I'm sure some hosting providers would love to play ball if we could guarantee some long term customers if they did.

* I don't really think this needs to be WAI specific: assuming we just use reverse HTTP proxies, this will work for any app that uses

standalone hosting. We could also support FastCGI if we wanted.

Yep.

* Remember that Amazon is now providing a free tier, which provides a micro EC2 instance (available as either 32 or 64 bit). I think the micro instance should be sufficient for running a few web apps, nginx and PostgreSQL.

This is a great idea, certainly a decent proving ground.

* Since starting up and shutting down instances is so easy with Amazon, we could provide both a "web host" image and a "Haskell compiler" image. The web host could automatically start up a compiler instance as needed.

Yep.

* Are we going to provide email sending support, with all of the complications of dealing with spam lists, or simply offload to external services (like Amazon SES[2])?

Not gonna touch that one.

* Does anyone have a good recommendation on who to use as a host?

If we really want to go down this path, I have some contacts we could talk to, but don't want to bother them unless we have a solid plan.

** By server configurations, I mean a 1 tier approach, where the front-end http server, the applications, and the database are on on the same machine, through the 3 tier approach. [3] http://en.wikipedia.org/wiki/TurnKey_Linux_Virtual_Appliance_Library [4] http://www.turnkeylinux.org/

On Sat, 2011-03-26 at 19:32 +0200, Michael Snoyman wrote:

Hi all,

JP Moresmau recently sent me an email asking for recommendations on hosting a Yesod app. I think a number of you have heard me mention the idea of setting up a WAI web hosting system (code named Lambda Engine). I think it's time to start hitting this idea hard, and I'd like community involvement as much as possible.

This is something that I feel would be *very* useful; perhaps even more useful than the Yesod installer. I am very excited about seeing this happen.

I think we have two extremes we can approach this from: at the one end, we can provide a complete hosting solution, similar to Heroku[1]. People would simply "cabal sdist" their project, type "lambda-engine upload mytarball.tar.gz" and their site would be live. Lambda Engine would automatically handle database server configuration, DNS setup, and so on.

I currently use something similar to the 'vhost' middleware for Wai (Network.Wai.Middleware.Vhost) and intended to expand it to a more powerful proxy server; and I like the idea of easily uploading new / updating current applications. However, I would probably prefer to upload a binary rather than the source. None of my servers have any compiler support (for reasons similar to those stated by Rick Richardson). I would love to be able to "lambda-engine upload server mysite" (which would upload "mysite" to a server at "server") and immediately see the result. Oh my...

At the other extreme, we could simply provide a set of shell scripts for setting up a vanilla Ubuntu (or Debian, or Cent, or FreeBSD...) install to support compiling a WAI application, provide an nginx setup, etc. The user would be responsible- and have the freedom to- set up his/her own server however he/she wants.

I would prefer to be able to choose the Linux distro I used, so a set of shell scripts would have to be reasonably generic. However I feel this would take a lot of effort to realise. IMO better to be just a program.

There are lots of in-between points on that spectrum. For example, I was thinking of perhaps providing a central compilation server that everyone could use. In my experience, Haskell applications take very few resources to run, but actually building the things take a while. Compiling on a local system and then uploading is painful because (1) you have to deal with architecture issues and (2) you might have a slow upload speed.

If these were problems I suffered from, I would be more comfortable if this service was commercial.

Anyway, I'm very interested to hear what other people think about all this. I'll enumerate a bunch of random thoughts on this topic:

* Should this be a for-profit venture, bankrolled by a few members, or a community, non-profit approach? (I'm favoring the latter.)

Perhaps both? For single users the community approach would be wonderful. For us, we would rather pay for a service that would offer certain guarantees.

* If we decide to provide some serious hosting power, we might be able to make a deal with a hosting provider, stating that we're representing the "entire Haskell web community".

I believe that Rick's point about providing some guarantee of long-term customers would certainly get some hosts on-board, but I fear that the guarantee would be hard to make.

* I don't really think this needs to be WAI specific: assuming we just use reverse HTTP proxies, this will work for any app that uses standalone hosting. We could also support FastCGI if we wanted.

Useful.

* Remember that Amazon is now providing a free tier, which provides a micro EC2 instance (available as either 32 or 64 bit). I think the micro instance should be sufficient for running a few web apps, nginx and PostgreSQL.

I'm not sure about that one.

* Since starting up and shutting down instances is so easy with Amazon, we could provide both a "web host" image and a "Haskell compiler" image. The web host could automatically start up a compiler instance as needed.

Sure.

* Are we going to provide email sending support, with all of the complications of dealing with spam lists, or simply offload to external services (like Amazon SES[2])?

Would be very useful. ----------------- * Load balancing? Install lambda-engine on a number of servers and allow load-balancing between them cross-roads style (XR)? * ... and be able to provide custom balancing algorithm in Haskell?! * Upload binary to one machine and it mirrors it to all? * How to handle per-application configuration (host, and so on)? * Daemonized lambda-engine? I'm very excited about this idea; I would love to be able to ditch lighttpd/apache in favour of a WAI specific server (with optional FastCGI support). - Blake xx

Michael

[1] http://heroku.com/ [2] http://aws.amazon.com/ses/

_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel