Yes, that is all very clear. My contention is that one should not have overlapping routes, particularly when mounting an Application. If an Application is mounted at /route, then it should be clear that /route* should belong to the Application. For example, with Yesod, Yesod "stands in front", but one often mounts wai-app-static at /static and Yesod will statically enforce that you don't create another static/* route anywhere else. What is the use case for having one framework "stand in front" such that it requires overlapping, or what is the use case for overlapping otherwise? On Fri, Oct 23, 2015 at 4:27 PM, Julian Arni wrote:

So let's say we have:

/auth ---> application mounted here POST /auth/subauth --> normal handler POST /auth ---> normal handler

If a GET to "auth/subauth" arrives, it gets dispatched to the Application. If the Application doesn't have a subauth endpoint, it 404s. But the response we want is 405, since there is something that matches the URL (but not the method). If the Application responded with essentially any other error besides 404, however, or with a successful response, we don't want to override it with a 405.

Moreover, if a POST to /auth arrives, where do we send it to - the normal handler or the Application? The handler just because we know that matches? Or the Application because it showed up first in the code, even though it might not match? Essentially, we just don't know how to route, given that everything inside Application is opaque.

We can try sending things to the Application and checking the first line of the response. However, not all 404s are created equal. If the handler decides to 404 because the request body contains an ID for a User in the database that happens to not exist, rerouting would probably be misleading (returning a 405 would definitely be). Also, that solution just generally feels very hacky!

To some extent we can just warn in the documentation against handlers that might overlap with the mounted Application (or even do static checks to prevent that), but this isn't very nice when e.g. one web framework is meant to "stand in front" of another, in the sense that anything that doesn't get matched by the one framework should fall through into the other (the mounted Application one), which I imagine being a decently common use-case. (Because the problem also exists in that direction, where the Application is the last tried thing - if we reroute to the Application because the best we got was a 405, the Application might respond with a 404, confusing everybody...) It also doesn't really allow for one framework handling a GET, and another (via application) a POST on the same URL, say, without the same issues. Anyhow, these things seems like something to consider if integrating between frameworks via Application becomes a more heavily used feature.

Not sure if this makes it any clearer - let me know if not!

On Sat, Oct 24, 2015 at 12:54 AM, Greg Weber wrote:

...

On Fri, Oct 23, 2015 at 8:00 AM, Julian Arni wrote:

...

Thanks again for starting the discussion!

As I mentioned, for Servant I think it's unlikely we'd completely move away from our current handlers, since we don't want to lose all the type information we have, and don't want to impose such information on others. So I'm hoping the next best thing might be achievable - allowing the new-style handlers to be run directly, and having conversion functions from our current handlers to the new style. I think the first is easy, and the second possible. Since there isn't enough information at the value-level to establish where it is our handlers are getting their arguments from (query string, headers, request body, etc.), we'd need a Proxy argument for the relevant part of the api. Something like:

convert :: (CanConvert api x) => Proxy api -> x -> NewHanderM ()

Where 'convert' would need to walk down the arguments of x and pull arguments from the appropriate places. E.g., if

x ~ Int -> Person -> ExceptT ServantErr IO (), and api ~ Header "H" Int :> ReqBody '[JSON] Person -> Post '[] ()

we'd have 'convert' conceptually doing: convertExceptT $ liftM2 x (parseHeader <$> getHeader "H") (decode <$> getBody), where getHeader and getBody are functions in the new handler monad (/class). (This isn't quite right, since decoding failures need to be handled).

Though as I mentioned before, since for servant at least in one direction it'd be a conversion rather than direct usage of the new handler, and since we can already convert any servant handler plus appropriate Proxy into an Application, and can use any Application as a handler, for us it'd be a lot easier to just use Application. As far as I can tell, this is also true in a couple of other frameworks. Indeed, one of the great things about the Application signature, in my opinion, is that it works well both as the signature for an entire Application and for a handler (that gets a modified request). The only problem we've come across with that idea is that the type of failure of the handler is opaque to the outer framework - so e.g., there's no easy way to tell whether re-routing needs to be triggered because the handler failed to match. This, however, would still be a problem in the model mentioned above. Having something like Application, but with return type IO (Either ReroutePlz ResponseReceived), would likely suffice (as would settling on an exception type).

I think that for an Application to be shared it needs to be mountable to an arbitrary route (e.g. "/auth"). And any request that matches the mounted route should be dispatched to just the mounted Application. Inside that mounted route the Application can then do additional routing. But if it does not match, it should return a 404, and that should be sent all the way to the client. Or is this too restrictive? What exactly is the need to interpret a 404 as meaning "try a different route"?

...

On Thu, Oct 22, 2015 at 2:00 PM, wrote:

...

Send web-devel mailing list submissions to web-devel@haskell.org

To subscribe or unsubscribe via the World Wide Web, visit http://mail.haskell.org/cgi-bin/mailman/listinfo/web-devel or, via email, send a message with subject or body 'help' to web-devel-request@haskell.org

You can reach the person managing the list at web-devel-owner@haskell.org

When replying, please edit your Subject line so it is more specific than "Re: Contents of web-devel digest..."

Today's Topics:

1. More collaboration in web development (Michael Snoyman)

----------------------------------------------------------------------

Message: 1 Date: Thu, 22 Oct 2015 08:13:31 +0300 From: Michael Snoyman To: web-devel , haskell-wai@googlegroups.com, "yesodweb@googlegroups.com" Subject: [web-devel] More collaboration in web development Message-ID: Content-Type: text/plain; charset="utf-8"

I originally kicked this off as an email to the maintainers of some of the other WAI web frameworks. After some initial brainstorming and some broad consensus to find some more shared common ground, we decided to open this up to a broader discussion.

I think the primary forum for this discussion will likely be the web-devel mailing list, though I'm cross-posting to haskell-wai and yesodweb as well for those interested. Please feel free to share with other mailing lists also. (We debated whether to use web-devel or haskell-wai for this discussion and didn't come to a decision, I'm fine moving it elsewhere.)

Based on the discussion, it seems like our best next step will be putting together a package containing some form of a Handler monad that could be shared by multiple web frameworks, hopefully making it easier to write code that can move between various frameworks. By contrast, we also seem to be in agreement that the high level routing code will end up being framework-specific, though we may be able to share some common low-level Trie-based dispatch code.

I'll leave it to other participants to restate points they made in the initial private discussion.

* * *

Hi all,

When I was in San Francisco a few months ago, Greg and I discussed the state of Haskell web development. It's clear that there's a lot of great stuff going on. Both of us have a strong desire to find as much common ground for all web framework authors to work on as possible, without detracting from meaningful distinctions between the various frameworks.

Greg and I have discussed this point off and on for the past few months, and haven't come up with very many concrete ideas. The recent work on http-api-data[1] would be one possible example of collaboration. Other possible ideas that Greg and I brainstormed were some kind of shared library for efficient dispatch (that could be used under the surface for the various high-level interface different libraries already provide), or some kind of a common Handler-like monad (which from what I've seen many people seem to reimplement all over the place). Some concrete benefits we can see from this:

* Previously, Persistent users were complaining that their types did not serialize in servant. http-api-data addresses this. * A common Handler would allow a higher layer abstraction for web apps, much like WAI provides at a low level. With something like that, sharing code like authentication logic becomes a reality * We already have precedence for other such shared libraries: client-session, server-session, cookie, and authenticate, as a few examples.

These are just ideas to kick off some thoughts, I'm sure some discussion could bring out some better ideas. Overall, it seems like we have a lot of opportunity to improve the situation for everyone if we focus on such collaboration. Having a shared low-level HTTP interface (WAI) has already helped this out considerably, I'm hoping there's something else we can do in this vein as well.

Michael

[1] http://hackage.haskell.org/package/http-api-data

On Mon, Oct 26, 2015 at 6:30 AM, Julian Arni wrote:

(Sorry for the delay, was away for the weekend.)

On Sat, Oct 24, 2015 at 2:05 AM, Greg Weber wrote:

...

Yes, that is all very clear. My contention is that one should not have overlapping routes, particularly when mounting an Application. If an Application is mounted at /route, then it should be clear that /route* should belong to the Application.

For example, with Yesod, Yesod "stands in front", but one often mounts wai-app-static at /static and Yesod will statically enforce that you don't create another static/* route anywhere else.

What is the use case for having one framework "stand in front" such that it requires overlapping, or what is the use case for overlapping otherwise?

Well, my thought was that Application is, modulo this one problem, already a way of sharing handlers (or sets of handlers) between frameworks. If overlaps are handled nicely, one can combine different web frameworks very neatly. This seems to achieve the original aim, but maybe I'm missing some of the motivation for a common handler?

This is the aim, the question is just whether overlapping routes are useful.

As for a use case for standing-in-front, it's sort of a nice way to have something like a reverse proxy that captures some endpoints rather than proxying them to the underlying service, without the actual overhead of external requests. I imagine there are other use-cases.

It seems like a 404 response would be sufficient for something behaving like a proxy.

Finally, it seems like if we had a good story about Applications working anywhere - in particular, converting one or more endpoints to Application and then mounting them in the same place not changing the behaviour of the overall Application - it would be quite easy for all web frameworks to apply Middleware to a subset of endpoints rather than all of them.

It should be easy enough to apply a middleware to selected applications. If they are already combined with routing one can still use a Routed Middleware: https://github.com/yesodweb/wai/blob/master/wai-extra/Network/Wai/Middleware... So I am still looking for a very concrete (code) example where overlapping routing is desirable. I can point you to the yesod-scaffold where we explicitly avoid overlapping routes and have applications at /auth and /static https://github.com/yesodweb/yesod-scaffold/blob/postgres/config/routes