Hi Myles,
These sound like two solid features, and I'd be happy to merge in code to
support it. Some comments below.
On Sat, Jan 21, 2012 at 8:38 AM, Myles C. Maxfield wrote: To: Michael Snoyman, author and maintainer of http-conduit
CC: haskell-cafe Hello! I am interested in contributing to the http-conduit library. I've been
using it for a little while and reading through its source, but have felt
that it could be improved with two features: - Allowing the caller to know the final URL that ultimately resulted
in the HTTP Source. Because httpRaw is not exported, the caller can't even
re-implement the redirect-following code themselves. Ideally, the caller
would be able to know not only the final URL, but also the entire chain of
URLs that led to the final request. I was thinking that it would be even
cooler if the caller could be notified of these redirects as they happen in
another thread. There are a couple ways to implement this that I have been
thinking about:
- A straightforward way would be to add a [W.Ascii] to the type of
Response, and getResponse can fill in this extra field. getResponse already
knows about the Request so it can tell if the response should be gunzipped. What would be in the [W.Ascii], a list of all paths redirected to? Also,
I'm not sure what gunzipping has to do with here, can you clarify? - It would be nice for the caller to be able to know in real time what
URLs the request is being redirected to. A possible way to do this would be
for the 'http' function to take an extra argument of type (Maybe
(Control.Concurrent.Chan W.Ascii)) which httpRaw can push URLs into. If the
caller doesn't want to use this variable, they can simply pass Nothing.
Otherwise, the caller can create an IO thread which reads the Chan until
some termination condition is met (Perhaps this will change the type of the
extra argument to (Maybe (Chan (Maybe W.Ascii)))). I like this solution,
though I can see how it could be considered too heavyweight. I do think it's too heavyweight. I think if people really want lower-level
control of the redirects, they should turn off automatic redirect and allow
3xx responses. - Making the redirection aware of cookies. There are redirects around
the web where the first URL returns a Set-Cookie header and a 3xx code
which redirects to another site that expects the cookie that the first HTTP
transaction set. I propose to add an (IORef to a Data.Set of Cookies) to
the Manager datatype, letting the Manager act as a cookie store as well as
a repository of available TCP connections. httpRaw could deal with the
cookie store. Network.HTTP.Types does not declare a Cookie datatype, so I
would probably be adding one. I would probably take it directly from
Network.HTTP.Cookie. Actually, we already have the cookie package for this. I'm not sure if putting the cookie store in the manager is necessarily the right approach,
since I can imagine wanting to have separate sessions while reusing the
same connections. A different approach could be adding a list of Cookies to
both the Request and Response. I'd be happy to do both of these things, but I'm hoping for your input on
how to go about this endeavor. Are these features even good to be pursuing?
Should I be going about this entirely differently? Thanks,
Myles C. Maxfield P.S. I'm curious about the lack of Network.URI throughout
Network.HTTP.Conduit. Is there a particular design decision that led you to
use raw ascii strings? Because there are plenty of URIs that are valid that we don't handle at
all, e.g., ftp.
Michael