[web-devel] Embedded Files for Yesod.Static (was: Content-Length on sendFile)

Changed the subject since we're pretty far from the OP. I think embedding files is a fantastic idea. Also, this would make deployment a bit easier since an entire app can be packaged into a single executable. max On Jun 15, 2011, at 11:20 AM, Michael Snoyman wrote:

On Wed, Jun 15, 2011 at 5:57 AM, Greg Weber wrote:

...

On Tue, Jun 14, 2011 at 7:29 PM, Michael Snoyman wrote:

...

Let me point out one other distinction: sendFile versus yesod-static. The former is a function you would call from a normal handler, while yesod-static is the "magical" package which would actually know all of the stuff about your files at compile time. For the sendFile case, the only options for getting the file size are (1) the programmer manually adding the header and (2) Yesod automatically doing a system call to get it.

As for the behavior of Warp... while I agree Kazu that we should reduce system call overhead, it might make sense for Warp to perform the system call to get file size *if* no content-length header is present.

And I'm still very uncomfortable setting the content-length header for static files based on compile-time information. In this case, having the wrong value (e.g., someone modified a CSS file after compiling) will completely break things and corrupt an open HTTP connection.

Then it should probably not be the default. Lets add a big scary warning to such a setting and tell users the files should be set to read-only permission. The most efficient technique for file serving static files that could be changed would actually be to setup a file notifier (that uses an efficient OS listener, like inotify on linux) that listens to static assets and knows when they are changed or a new one is added and would stat them just when changed. But there would be a race condition if you had to wait for the notification, so you would actually have to have the old file revision on hand- perhaps having a symlink convention for adding new files, but meaning the system could still get screwed up by someone careless.

I'm really starting to think that the correct solution for this use case really is embedded files[1]. It avoids any kind of system call overhead, doesn't touch the hard drive at all, and doesn't allow for modification of the files after compile time. It would be interesting to see if it ends up faster than sendfile.

Michael

[1] Using Template Haskell to store the entire static file inside the executable during compilation.

...

...

Michael

On Wed, Jun 15, 2011 at 5:22 AM, Kazu Yamamoto wrote:

...

Greg,

...

I apologize for the confusing terminology. I am not differentiating between sending a static file with sendfile and a streaming response. I am differentiating between 2 different use cases for sending static files (with sendfile). For all of my web applications, I know what all the static files are and they will never change until I deploy another web application. That means I can stat the files once when the application is deployed and keep that information in memory. So I already have the file length information to include in the header, even though I don't do a file stat when the file is requested. wai-app-static and yesod-static supports these techniques.

Thanks. I think I understand. :)

So, do you support to *not* change the API (apps should add CL: by themselves)?

--Kazu

_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel

_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel

_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel