Network.Curl POST body not showing up

Hi all, I was hoping one of you was pretty proficient with Network.Curl to help me out, as I am not. I need to post JSON to an HTTPS endpoint using HTTP Basic Authentication. The endpoint will not accept the request unless the content-type is set to application/json. If I were using the curl utility, I'd do something like curl -H "Content-Type: application/json" -d "{blah:42}" -u 'myuser:mypass' https://example.com/api.json I'd really like to use curl's curlGetString because the return type is perfect, the code and the body. I've tried curlGetString, curlPost and curlMultiPost and all combinations of CurlOption, but I can either get it to send the correct content type header with empty content or send the post body with the content type defaulted to urlencoded or multipart/form-data. My code right now looks something like: doPost :: (ToJSON a) => T.Text -> T.Text -> T.Text -> a -> CampfireM (CurlCode, String) doPost key sub path pay = liftIO $ curlGetString (url path sub) opts where opts = method_POST ++ curlOpts key ++ [CurlVerbose True, post, ct] post = CurlPostFields [T.unpack encPay] ct = CurlHttpHeaders ["Content-Type: application/json"] encPath = encodePath path encPay encPay = encodePayload pay The post fields will not be in the form of "key=value", but rather a JSCODE: * Server auth using Basic with user 'SECRET_KEY'
POST /rooms/287132/speak.json HTTP/1.1 Authorization: Basic SECRET_KEY Host: example.com Accept: */* Content-Type: application/json Content-Length: 103
* The requested URL returned error: 500 * Closing connection #0 * HTTP response code said error It would seem to me from this output that no actual content was sent. What am I doing wrong? I'd really like to use curlGetString instead of some of the more complex curl functions. Thanks! -- Michael Xavier http://www.michaelxavier.net -- Michael Xavier http://www.michaelxavier.net

I would be interested in this as well. I'm trying to use Network.Curl for a personal project, also using a method_post, and am not able to get a response. Though I know it's connecting. I _think_ that I need to use CurlWriteFunction, but I'm still new to Haskell & not sure how to work with that option. Bryce On 05/10/2011 10:25 PM, Michael Xavier wrote:
Hi all,
I was hoping one of you was pretty proficient with Network.Curl to help me out, as I am not. I need to post JSON to an HTTPS endpoint using HTTP Basic Authentication. The endpoint will not accept the request unless the content-type is set to application/json.
If I were using the curl utility, I'd do something like
curl -H "Content-Type: application/json" -d "{blah:42}" -u 'myuser:mypass' https://example.com/api.json
I'd really like to use curl's curlGetString because the return type is perfect, the code and the body.
I've tried curlGetString, curlPost and curlMultiPost and all combinations of CurlOption, but I can either get it to send the correct content type header with empty content or send the post body with the content type defaulted to urlencoded or multipart/form-data. My code right now looks something like:
doPost :: (ToJSON a) => T.Text -> T.Text -> T.Text -> a -> CampfireM (CurlCode, String) doPost key sub path pay = liftIO $ curlGetString (url path sub) opts where opts = method_POST ++ curlOpts key ++ [CurlVerbose True, post, ct] post = CurlPostFields [T.unpack encPay] ct = CurlHttpHeaders ["Content-Type: application/json"] encPath = encodePath path encPay encPay = encodePayload pay
The post fields will not be in the form of "key=value", but rather a JSCODE:
* Server auth using Basic with user 'SECRET_KEY'
POST /rooms/287132/speak.json HTTP/1.1 Authorization: Basic SECRET_KEY Host: example.com http://example.com Accept: */* Content-Type: application/json Content-Length: 103
* The requested URL returned error: 500 * Closing connection #0 * HTTP response code said error
It would seem to me from this output that no actual content was sent. What am I doing wrong? I'd really like to use curlGetString instead of some of the more complex curl functions.
Thanks!
-- Michael Xavier http://www.michaelxavier.net
-- Michael Xavier http://www.michaelxavier.net
_______________________________________________ Beginners mailing list Beginners@haskell.org http://www.haskell.org/mailman/listinfo/beginners

Bryce Verdier wrote:
I would be interested in this as well. I'm trying to use Network.Curl for a personal project, also using a method_post, and am not able to get a response. Though I know it's connecting.
I _think_ that I need to use CurlWriteFunction, but I'm still new to Haskell & not sure how to work with that option.
I tried a number of times to do things with Network.Curl and found it to be a painful and frustrating experience. The API is highly irregular and often remains too close to the underlying C API. More recently I've been using the http-enumerator package which is conceptually a little more complicated but works really, really well. It does both HTTP and HTTPS, GET and POST. It currently doesn't work via a proxy, but the http-enumerator author has said he will take patches and I'm working on it. Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/

I've tried 3 different HTTP libraries for this project: Network.HTTP (which
doesn't support SSL), Network.Curl, and http-enumerator. This project
requires SSL, HTTP Basic Auth and Post. Network.HTTP does not support SSL
and http-enumerator does not support HTTP basic authentication. The
maintainer said he'd welcome a patch to it but I don't really have the time
or skill at this stage to implement something like that.
So I'm stuck with trying to coerce curl into doing what I need it to do.
Hopefully someone on this list has been able to successfully POST a string
body with Network.Curl
On Wed, May 11, 2011 at 6:07 PM, Erik de Castro Lopo
Bryce Verdier wrote:
I would be interested in this as well. I'm trying to use Network.Curl for a personal project, also using a method_post, and am not able to get a response. Though I know it's connecting.
I _think_ that I need to use CurlWriteFunction, but I'm still new to Haskell & not sure how to work with that option.
I tried a number of times to do things with Network.Curl and found it to be a painful and frustrating experience. The API is highly irregular and often remains too close to the underlying C API.
More recently I've been using the http-enumerator package which is conceptually a little more complicated but works really, really well. It does both HTTP and HTTPS, GET and POST. It currently doesn't work via a proxy, but the http-enumerator author has said he will take patches and I'm working on it.
Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
_______________________________________________ Beginners mailing list Beginners@haskell.org http://www.haskell.org/mailman/listinfo/beginners
-- Michael Xavier http://www.michaelxavier.net

Hi Micheal,
You cannot use curlGetString to POST since it always uses GET.
Here is a code snippet which I use. Although I don't use basic auth,
you can add another opt to use it.
{-# LANGUAGE ScopedTypeVariables #-}
import Control.Exception (IOException, handle)
import Control.Monad (liftM)
import qualified Data.ByteString as BSS
import qualified Data.ByteString.Lazy as BS
import qualified Data.ByteString.Lazy.Char8 as BS8
import Data.IORef
import qualified Network.Curl as Curl
import Network.URI (URI)
post :: URI -> BS.ByteString -> String -> IO (Maybe BS.ByteString)
post uri body contentType = handleIOException (const $ return Nothing)
$ Curl.withCurlDo $ do
bodyRef <- newIORef []
h <- Curl.initialize
mapM_ (Curl.setopt h) $ [Curl.CurlURL $ show uri,
Curl.CurlNoBody False,
Curl.CurlFollowLocation False,
Curl.CurlMaxRedirs 0,
Curl.CurlAutoReferer False,
Curl.CurlUserAgent "Mozilla/5.0",
Curl.CurlNoSignal True,
-- Curl.CurlVerbose True,
Curl.CurlPostFields [BS8.unpack body],
Curl.CurlHttpHeaders ["Content-Type:
" ++ contentType],
Curl.CurlWriteFunction $ bodyFunction bodyRef]
code <- Curl.perform h
if code /= Curl.CurlOK
then return Nothing
else liftM (Just . BS.fromChunks . reverse) $ readIORef bodyRef
bodyFunction :: IORef [BSS.ByteString] -> Curl.WriteFunction
bodyFunction r = Curl.gatherOutput_ $ \s -> do
bs <- BSS.packCStringLen s
modifyIORef r (bs:)
handleIOException :: (IOException -> IO a) -> IO a -> IO a
handleIOException handler action = handle (\(e :: IOException) ->
handler e) action
--
Satoshi Nakamura
I've tried 3 different HTTP libraries for this project: Network.HTTP (which doesn't support SSL), Network.Curl, and http-enumerator. This project requires SSL, HTTP Basic Auth and Post. Network.HTTP does not support SSL and http-enumerator does not support HTTP basic authentication. The maintainer said he'd welcome a patch to it but I don't really have the time or skill at this stage to implement something like that. So I'm stuck with trying to coerce curl into doing what I need it to do. Hopefully someone on this list has been able to successfully POST a string body with Network.Curl
On Wed, May 11, 2011 at 6:07 PM, Erik de Castro Lopo
wrote: Bryce Verdier wrote:
I would be interested in this as well. I'm trying to use Network.Curl for a personal project, also using a method_post, and am not able to get a response. Though I know it's connecting.
I _think_ that I need to use CurlWriteFunction, but I'm still new to Haskell & not sure how to work with that option.
I tried a number of times to do things with Network.Curl and found it to be a painful and frustrating experience. The API is highly irregular and often remains too close to the underlying C API.
More recently I've been using the http-enumerator package which is conceptually a little more complicated but works really, really well. It does both HTTP and HTTPS, GET and POST. It currently doesn't work via a proxy, but the http-enumerator author has said he will take patches and I'm working on it.
Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
_______________________________________________ Beginners mailing list Beginners@haskell.org http://www.haskell.org/mailman/listinfo/beginners
-- Michael Xavier http://www.michaelxavier.net
_______________________________________________ Beginners mailing list Beginners@haskell.org http://www.haskell.org/mailman/listinfo/beginners

Michael Xavier wrote:
I've tried 3 different HTTP libraries for this project: Network.HTTP (which doesn't support SSL), Network.Curl, and http-enumerator. This project requires SSL, HTTP Basic Auth and Post. Network.HTTP does not support SSL and http-enumerator does not support HTTP basic authentication. The maintainer said he'd welcome a patch to it but I don't really have the time or skill at this stage to implement something like that.
Nakamura-san has provided you with some code that gets you part way to where you need to be. Basic auth is something I'm going to need in http-enumerator pretty soon so I'll probably have a stab at it over the next week or two. If you're interested in that and you haven't heard back from me in say two weeks, feel free to ping me. Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/

Basic auth is something I'm going to need in http-enumerator pretty
soon so I'll probably have a stab at it over the next week or two. If you're interested in that and you haven't heard back from me in say two weeks, feel free to ping me.
I have an active issue with the author of the project on github discussing this very thing: https://github.com/snoyberg/http-enumerator/issues/14#comment_1086615 I'm not really up to implementing auth quite yet but it may help you in your endeavors. Definitely keep me posted if you decide to tackle this issue. I'd definitely like to swap out the backend on my library for http-enumerator which i now greatly prefer to Network.Curl Thanks so much for the code sample Satoshi. It is a bit daunting to me as a Haskell novice but I was able to massage it into what I needed. It's unfortunate at how much more convoluted the Curl api is when you want to POST instead of GET but I suspect as others have suggested that this may have a lot to do with the underlying C API. -- Michael Xavier http://www.michaelxavier.net

On Fri, May 13, 2011 at 9:19 AM, Michael Xavier
Basic auth is something I'm going to need in http-enumerator pretty
soon so I'll probably have a stab at it over the next week or two. If you're interested in that and you haven't heard back from me in say two weeks, feel free to ping me.
I have an active issue with the author of the project on github discussing this very thing:
https://github.com/snoyberg/http-enumerator/issues/14#comment_1086615
I'm not really up to implementing auth quite yet but it may help you in your endeavors. Definitely keep me posted if you decide to tackle this issue. I'd definitely like to swap out the backend on my library for http-enumerator which i now greatly prefer to Network.Curl
Thanks so much for the code sample Satoshi. It is a bit daunting to me as a Haskell novice but I was able to massage it into what I needed. It's unfortunate at how much more convoluted the Curl api is when you want to POST instead of GET but I suspect as others have suggested that this may have a lot to do with the underlying C API.
Actually, Erik already submitted a patch adding basic auth support, and I've released it on Hackage. Michael
-- Michael Xavier http://www.michaelxavier.net
_______________________________________________ Beginners mailing list Beginners@haskell.org http://www.haskell.org/mailman/listinfo/beginners

Michael Xavier wrote:
I have an active issue with the author of the project on github discussing this very thing:
https://github.com/snoyberg/http-enumerator/issues/14#comment_1086615
I'm not really up to implementing auth quite yet but it may help you in your endeavors. Definitely keep me posted if you decide to tackle this issue.
I did. https://github.com/snoyberg/http-enumerator/commit/346c16607cad805e393e666e5... It should be in version 6.5 of httpenumerator which is already on Hackage. Please test it and if it works for you close the github issue. Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/

Wow, you guys work fast! Thanks for all your help. I will probably be switching back to http-enumerator in the next few days. -- Michael Xavier http://www.michaelxavier.net
participants (5)
-
Bryce Verdier
-
Erik de Castro Lopo
-
Michael Snoyman
-
Michael Xavier
-
Satoshi Nakamura