http-enumerator connecting via a http proxy
Hi all, Has anyone managed to get http-enumerator connecting via a http proxy? I've tried constructing a custom 'Request m' object and submitting that the httpLbsRedirect but all I've managed to get is: connect: does not exist (Connection refused) Clues? Would this need a new http-enumerator function something like: httpProxyLbsRedirect :: (MonadIO m, Failure HttpException m) => String -> Int -> Request m -> Manager -> m Response where the first two parameters are the proxy hostname and port? Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
On Tue, May 10, 2011 at 6:04 AM, Erik de Castro Lopo
Hi all,
Has anyone managed to get http-enumerator connecting via a http proxy?
I've tried constructing a custom 'Request m' object and submitting that the httpLbsRedirect but all I've managed to get is:
connect: does not exist (Connection refused)
Clues? Would this need a new http-enumerator function something like:
httpProxyLbsRedirect :: (MonadIO m, Failure HttpException m) => String -> Int -> Request m -> Manager -> m Response
where the first two parameters are the proxy hostname and port?
Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel
I've never tried http-enumerator with a proxy, but I'm almost certain it won't work without modifying the code to handle it. I'll accept patches/provide help to write this, but I won't be able to write it myself in the near future. Michael
Michael Snoyman wrote:
I've never tried http-enumerator with a proxy, but I'm almost certain it won't work without modifying the code to handle it.
It doesn't :-).
I'll accept patches/provide help to write this
That is sufficent. Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
Erik de Castro Lopo wrote:
Clues? Would this need a new http-enumerator function something like:
This is mainly an FYI. I've been doing a little hacking on this. I've defined a Proxy data type as: data Proxy = Proxy { proxyHost :: W.Ascii -- ^ The host name of the HTTP proxy. , proxyPort :: Int -- ^ The port numner of the HTTP proxy. } deriving (Show) I then copied the http function to produce a function proxyHttp: proxyHttp :: MonadIO m => Proxy -> Request m -> (W.Status -> W.ResponseHeaders -> Iteratee S.ByteString m a) -> Manager -> Iteratee S.ByteString m a which can now do a GET via a Squid proxy to HTTP server. I'm now working on the CONNECT method for connecting to HTTPS. Once I get CONNECT working as well, I'm going to try and share as much code between the proxy and non-proxy versions as possible. I'm currently doing this in a local branch so I can merge upstream and rebase as needed. Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
Erik de Castro Lopo wrote:
Erik de Castro Lopo wrote:
Clues? Would this need a new http-enumerator function something like:
This is mainly an FYI. I've been doing a little hacking on this. I've defined a Proxy data type as:
data Proxy = Proxy { proxyHost :: W.Ascii -- ^ The host name of the HTTP proxy. , proxyPort :: Int -- ^ The port numner of the HTTP proxy. } deriving (Show)
I then copied the http function to produce a function proxyHttp:
proxyHttp :: MonadIO m => Proxy -> Request m -> (W.Status -> W.ResponseHeaders -> Iteratee S.ByteString m a) -> Manager -> Iteratee S.ByteString m a
which can now do a GET via a Squid proxy to HTTP server.
I've implemented four functions; proxyHttp, proxyHttpLbs, proxyHttpRedirect and proxyHttpLbsRedirect but it occurs to me that if we added a field of type 'Maybe Proxy' to the 'Request m' data type, the fucntionality of the four functions I have hacked up could be merged with the original versions of these. Does that sound like a sane approach? Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
On Thu, May 12, 2011 at 6:08 AM, Erik de Castro Lopo
Erik de Castro Lopo wrote:
Erik de Castro Lopo wrote:
Clues? Would this need a new http-enumerator function something like:
This is mainly an FYI. I've been doing a little hacking on this. I've defined a Proxy data type as:
data Proxy = Proxy { proxyHost :: W.Ascii -- ^ The host name of the HTTP proxy. , proxyPort :: Int -- ^ The port numner of the HTTP proxy. } deriving (Show)
I then copied the http function to produce a function proxyHttp:
proxyHttp :: MonadIO m => Proxy -> Request m -> (W.Status -> W.ResponseHeaders -> Iteratee S.ByteString m a) -> Manager -> Iteratee S.ByteString m a
which can now do a GET via a Squid proxy to HTTP server.
I've implemented four functions; proxyHttp, proxyHttpLbs, proxyHttpRedirect and proxyHttpLbsRedirect but it occurs to me that if we added a field of type 'Maybe Proxy' to the 'Request m' data type, the fucntionality of the four functions I have hacked up could be merged with the original versions of these.
Does that sound like a sane approach?
Yes, I think that sounds good. Just let me know when I should look at the code; I'm impressed how quickly you're getting this done!
Michael
Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel
Michael Snoyman wrote:
Does that sound like a sane approach?
Yes, I think that sounds good. Just let me know when I should look at the code;
Will do.
I'm impressed how quickly you're getting this done!
Well so far its been really easy; cut, paste, tweak and repeat. Doing the CONNECT method for HTTPS is going to be a lot trickier. It also helps that I've done this stuff before in C, C++ and Ocaml. I'll probably also have a crack at doing basic auth as thats something I'll need in the near future (and I've done it in C++ and Ocaml before). Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
Michael Snoyman wrote:
I've implemented four functions; proxyHttp, proxyHttpLbs, proxyHttpRedirect and proxyHttpLbsRedirect but it occurs to me that if we added a field of type 'Maybe Proxy' to the 'Request m' data type, the fucntionality of the four functions I have hacked up could be merged with the original versions of these.
Does that sound like a sane approach?
Yes, I think that sounds good. Just let me know when I should look at the code; I'm impressed how quickly you're getting this done!
Ok, I've submitted a github pull request that modifies the existing http function to proxy HTTP requests. This was a relatively simple matter of modifying hosts, ports, and headers. I'm now looking at doing HTTPS and I'm a little lost on how to proceed. Basically proxying of HTTPS works as follows: a) Wants to connect to https://encrypted.google.com/ via HTTP proxy called squid listening on port 3128. b) Client opens an un-encrypted connection to squid:3128 and sends a request: CONNECT encrypted.google.com:443 HTTP/1.1 c) Squid proxy connects to encrypted.google.com port 443 and gets back a response of: HTTP/1.1 200 Connection established which it sends the client. d) The squid proxy then blindly transfers bytes from the client to encrypted.google.com and bytes from encrypted.google.com to the client. e) The client does TLS negotiation over the bi-directional pipe established and maintained by the proxy. I can send the CONNECT and get back the HTTP 200 OK, but I'm not sure how to proceed. Clues? Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
Michael Snoyman wrote:
I've implemented four functions; proxyHttp, proxyHttpLbs, proxyHttpRedirect and proxyHttpLbsRedirect but it occurs to me that if we added a field of type 'Maybe Proxy' to the 'Request m' data type, the fucntionality of the four functions I have hacked up could be merged with the original versions of these.
Does that sound like a sane approach?
Yes, I think that sounds good. Just let me know when I should look at the code; I'm impressed how quickly you're getting this done!
Ok, I've submitted a github pull request that modifies the existing http function to proxy HTTP requests. This was a relatively simple matter of modifying hosts, ports, and headers.
I'm now looking at doing HTTPS and I'm a little lost on how to proceed. Basically proxying of HTTPS works as follows:
a) Wants to connect to https://encrypted.google.com/ via HTTP proxy called squid listening on port 3128.
b) Client opens an un-encrypted connection to squid:3128 and sends a request:
CONNECT encrypted.google.com:443 HTTP/1.1
c) Squid proxy connects to encrypted.google.com port 443 and gets back a response of:
HTTP/1.1 200 Connection established
which it sends the client.
d) The squid proxy then blindly transfers bytes from the client to encrypted.google.com and bytes from encrypted.google.com to the client.
e) The client does TLS negotiation over the bi-directional pipe established and maintained by the proxy.
I can send the CONNECT and get back the HTTP 200 OK, but I'm not sure how to proceed.
Clues?
Hmm... this *does* seem problematic. If I understand correctly, we need to first accept unencrypted data over the socket, and then encrypted. The first
On Sat, May 14, 2011 at 2:55 PM, Erik de Castro Lopo
Michael Snoyman wrote:
If I were to take a stab at the best approach, it would be to modify withSslConn to (optionally) read in the HTTP response line from the proxy before handing control off to TLS. I'm not sure I entirely understand the issue, but does that seem like a plausible approach?
Yes it does. I'll investigate it. Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
Erik de Castro Lopo wrote:
Michael Snoyman wrote:
If I were to take a stab at the best approach, it would be to modify withSslConn to (optionally) read in the HTTP response line from the proxy before handing control off to TLS. I'm not sure I entirely understand the issue, but does that seem like a plausible approach?
Yes it does. I'll investigate it.
Ok, after a long break, I'm back working on this. Starting with the example of the withSslConn function I am working on the following function for the case of an SSL connection through a proxy: withSslProxyConn :: MonadIO m => ([X509] -> IO TLS.TLSCertificateUsage) -> S8.ByteString -- ^ Target host -> Int -- ^ Target port -> Manager -> String -- ^ Proxy host -> Int -- ^ Proxy port -> Enumerator Blaze.Builder m () -- ^ request -> Enumerator S.ByteString m a -- ^ response withSslProxyConn checkCert thost tport man phost pport = withManagedConn man (phost, pport, True) $ doConnect >>= TLS.sslClientConn checkCert where doConnect = do h <- connectTo phost (PortNumber $ fromIntegral pport) -- Need to send connectRequest and get a HTTP 200 response. return h connectRequest = Blaze.fromByteString "CONNECT " `mappend` Blaze.fromByteString thost `mappend` Blaze.fromByteString (S8.pack (':' : show tport)) `mappend` Blaze.fromByteString " HTTP/1.1\r\n\r\n" The problem is that I'm new to iteratee based coding techniques and I figure out how to complete the doConnect function. Basically it should a) send the connectRequest b) check the HTTP repsonse c) if the response is 200 return the handle, otherwise throw an error Can someone please give me a nudge in the right direction? Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
Erik de Castro Lopo wrote:
doConnect = do h <- connectTo phost (PortNumber $ fromIntegral pport) -- Need to send connectRequest and get a HTTP 200 response. return h
For this problem I do have the following solution: doConnect = do h <- connectTo phost (PortNumber $ fromIntegral pport) S8.hPutStr h $ toByteString connectRequest hFlush h din <- S.hGetSome h 1024 return h This does actually work correctly during my simplistic testing but this code doesn't really deserve to go into a package called http-enumerator :-). Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
participants (2)
-
Erik de Castro Lopo -
Michael Snoyman