Problem with multipart request on yesod/wai

Hi guys, I'm developing a webservice in Yesod that needs to accept multipart POST's with image data in them. This webservice will be used mainly by mobile devices. My handler that takes care of receiving the image works correctly when receiveng a POST crafted by google chrome (from a multipart/form-data encoded form), but when receiving a multipart post from a mobile device it fails to parse the multipart request arguments. I've setup a ruby on rails application with a similar handler, and both request from my mobile device and from a browser work, so I'm leaning towards thinking this is a yesod/wai issue. This is an excerpt from my handler, that I use to debug this: myHandler = do (params, files) <- runRequestBody liftIO $ putStrLn $ "params " ++ show params When requesting from the mobile device it just shows an empty list. I have dumps of both multipart requests, they seem pretty similar with the main differences being: * the multipart boundaries are longer in the browser request. * the request from the mobile device adds a trailing semicolon to the content-type of each part (except for the image) I attached the request that's not parsing correctly Any help will be greatly appreciated, thanks in advance! --nubis :)

It might be getting confused by the request headers. Could you send
those over as well? Also, what version of Yesod and wai-extra are you
running?
Michael
On Fri, Feb 25, 2011 at 4:34 AM, Nubis
Hi guys, I'm developing a webservice in Yesod that needs to accept multipart POST's with image data in them. This webservice will be used mainly by mobile devices. My handler that takes care of receiving the image works correctly when receiveng a POST crafted by google chrome (from a multipart/form-data encoded form), but when receiving a multipart post from a mobile device it fails to parse the multipart request arguments. I've setup a ruby on rails application with a similar handler, and both request from my mobile device and from a browser work, so I'm leaning towards thinking this is a yesod/wai issue.
This is an excerpt from my handler, that I use to debug this: myHandler = do (params, files) <- runRequestBody liftIO $ putStrLn $ "params " ++ show params
When requesting from the mobile device it just shows an empty list.
I have dumps of both multipart requests, they seem pretty similar with the main differences being: * the multipart boundaries are longer in the browser request. * the request from the mobile device adds a trailing semicolon to the content-type of each part (except for the image)
I attached the request that's not parsing correctly
Any help will be greatly appreciated, thanks in advance!
--nubis :)
_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel

On Fri, Feb 25, 2011 at 7:34 AM, Michael Snoyman
It might be getting confused by the request headers. Could you send those over as well? Also, what version of Yesod and wai-extra are you running?
Michael
Hi Michael, thanks for your reply and help I'm running yesod 0.7.0, wai-0.3.1 and wai-extra-0.3.2 Please let me know if there's anything else I could do to try to diagnose this problem. I could compile a custom version of any library with more debugging info, for now the stepping debugger approach exceeds me. Again, thanks a lot! ----nubis :) This is the reelevant environment info for each request (got it from ruby though, I'm looking for a way to dump the whole request from wai): -------------------- dalvik CONTENT_LENGTH:12098 CONTENT_TYPE:multipart/form-data;boundary=***** GATEWAY_INTERFACE:CGI/1.1 PATH_INFO:/ QUERY_STRING: REMOTE_ADDR:192.168.1.115 REMOTE_HOST:ganjizza REQUEST_URI:http://192.168.1.115:3000/ REQUEST_METHOD:POST HTTP_CONNECTION:Keep-Alive HTTP_COOKIE:_SESSION=fgUGM5J/k6mGAAW+MMXIJZCJHobw/oEbb6T17KQN0p9yNqiXn/m/ACrsnRjiCEgqtG4fogMUDI+jikoFGcwmPjvuD5d+MDz32iXvDdDJsFdsFMfivuey2H+n6IF6yFGD HTTP_USER_AGENT:Dalvik/1.1.0 (Linux; U; Android 2.1-update1; sdk Build/ECLAIR) HTTP_HOST:192.168.1.115:3000 HTTP_ACCEPT:*, */* HTTP_VERSION:HTTP/1.1 REQUEST_PATH:/ -------------------- chrome: CONTENT_LENGTH:1169352 CONTENT_TYPE:multipart/form-data; boundary=----WebKitFormBoundaryEAoeasrCzOBAAP5s GATEWAY_INTERFACE:CGI/1.1 PATH_INFO:/ QUERY_STRING: REMOTE_ADDR:192.168.1.115 REMOTE_HOST:ganjizza REQUEST_METHOD:POST REQUEST_URI:http://192.168.1.115:3000/ HTTP_HOST:192.168.1.115:3000 HTTP_CONNECTION:keep-alive HTTP_REFERER:http://192.168.1.115:3000/static/client/index.html HTTP_CACHE_CONTROL:max-age=0 HTTP_ORIGIN:http://192.168.1.115:3000 HTTP_ACCEPT:application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 HTTP_USER_AGENT:Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.237 Safari/534.10 HTTP_ACCEPT_ENCODING:gzip,deflate,sdch HTTP_ACCEPT_LANGUAGE:en-US,en;q=0.8 HTTP_ACCEPT_CHARSET:ISO-8859-1,utf-8;q=0.7,*;q=0.3 HTTP_COOKIE:_SESSION=AoJJW9pxuycN7/bDU0+ho4auufmHtnZM0IiCp04EgXhyNqiXn/m/ACrsnRjiCEgqtG4fogMUDI+jikoFGcwmPhxQjE4k4AICWrp934EzZ53IyUdx5OqY9KhjYT8xSyfa; _recetas_session=BAh7B0kiD3Nlc3Npb25faWQGOgZFRiIlMjEyMTBlZjEzMThiOGNhZDU4Njk0NTdjZmNhYTM0MTJJIhBfY3NyZl90b2tlbgY7AEZJIjFkTVB0NGdPMFJVNHgzMk5QVC9sMXBWV3FVSXNNSStCZ2xqc0RxOXFVUk9zPQY7AEY%3D--b71e4e03f9b47a7b62bc977d45e9502123e5510a HTTP_VERSION:HTTP/1.1 REQUEST_PATH:/

*
Hi all!
"...parseRequest
Parsing the request headers involves taking all of the header lines (every
line until a blank line) and then parsing those via parseRequest'.
takeHeaders (a function I will not explain here) goes ahead and reads in all
of the header lines until a blank. A special thanks to Gregory Collins for
pointing out a security hole in the initial versions of Warp: takeHeaders
now ensures that no header is longer than 1024 bytes, and there are at most
30 headers to avoid a DOS attack..." (
http://docs.yesodweb.com/blog/announcing-warp)*
May be it is the reason ?
2011/2/25 Nubis
On Fri, Feb 25, 2011 at 7:34 AM, Michael Snoyman
wrote: It might be getting confused by the request headers. Could you send those over as well? Also, what version of Yesod and wai-extra are you running?
Michael
Hi Michael, thanks for your reply and help
I'm running yesod 0.7.0, wai-0.3.1 and wai-extra-0.3.2
Please let me know if there's anything else I could do to try to diagnose this problem. I could compile a custom version of any library with more debugging info, for now the stepping debugger approach exceeds me.
Again, thanks a lot!
----nubis :)
This is the reelevant environment info for each request (got it from ruby though, I'm looking for a way to dump the whole request from wai): -------------------- dalvik
CONTENT_LENGTH:12098 CONTENT_TYPE:multipart/form-data;boundary=***** GATEWAY_INTERFACE:CGI/1.1 PATH_INFO:/ QUERY_STRING: REMOTE_ADDR:192.168.1.115 REMOTE_HOST:ganjizza REQUEST_URI:http://192.168.1.115:3000/ REQUEST_METHOD:POST HTTP_CONNECTION:Keep-Alive
HTTP_COOKIE:_SESSION=fgUGM5J/k6mGAAW+MMXIJZCJHobw/oEbb6T17KQN0p9yNqiXn/m/ACrsnRjiCEgqtG4fogMUDI+jikoFGcwmPjvuD5d+MDz32iXvDdDJsFdsFMfivuey2H+n6IF6yFGD HTTP_USER_AGENT:Dalvik/1.1.0 (Linux; U; Android 2.1-update1; sdk Build/ECLAIR) HTTP_HOST:192.168.1.115:3000 HTTP_ACCEPT:*, */* HTTP_VERSION:HTTP/1.1 REQUEST_PATH:/
-------------------- chrome:
CONTENT_LENGTH:1169352 CONTENT_TYPE:multipart/form-data; boundary=----WebKitFormBoundaryEAoeasrCzOBAAP5s GATEWAY_INTERFACE:CGI/1.1 PATH_INFO:/ QUERY_STRING: REMOTE_ADDR:192.168.1.115 REMOTE_HOST:ganjizza REQUEST_METHOD:POST REQUEST_URI:http://192.168.1.115:3000/ HTTP_HOST:192.168.1.115:3000 HTTP_CONNECTION:keep-alive HTTP_REFERER:http://192.168.1.115:3000/static/client/index.html HTTP_CACHE_CONTROL:max-age=0 HTTP_ORIGIN:http://192.168.1.115:3000
HTTP_ACCEPT:application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 HTTP_USER_AGENT:Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.237 Safari/534.10 HTTP_ACCEPT_ENCODING:gzip,deflate,sdch HTTP_ACCEPT_LANGUAGE:en-US,en;q=0.8 HTTP_ACCEPT_CHARSET:ISO-8859-1,utf-8;q=0.7,*;q=0.3 HTTP_COOKIE:_SESSION=AoJJW9pxuycN7/bDU0+ho4auufmHtnZM0IiCp04EgXhyNqiXn/m/ACrsnRjiCEgqtG4fogMUDI+jikoFGcwmPhxQjE4k4AICWrp934EzZ53IyUdx5OqY9KhjYT8xSyfa; _recetas_session=BAh7B0kiD3Nlc3Npb25faWQGOgZFRiIlMjEyMTBlZjEzMThiOGNhZDU4Njk0NTdjZmNhYTM0MTJJIhBfY3NyZl90b2tlbgY7AEZJIjFkTVB0NGdPMFJVNHgzMk5QVC9sMXBWV3FVSXNNSStCZ2xqc0RxOXFVUk9zPQY7AEY%3D--b71e4e03f9b47a7b62bc977d45e9502123e5510a HTTP_VERSION:HTTP/1.1 REQUEST_PATH:/
_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel
-- Best regards, Cheshkov Anton Phone: +7 909 005 18 82 Skype: cheshkov_anton

2011/2/26 Антон Чешков
Hi all! "...parseRequest Parsing the request headers involves taking all of the header lines (every line until a blank line) and then parsing those via parseRequest'. takeHeaders (a function I will not explain here) goes ahead and reads in all of the header lines until a blank. A special thanks to Gregory Collins for pointing out a security hole in the initial versions of Warp: takeHeaders now ensures that no header is longer than 1024 bytes, and there are at most 30 headers to avoid a DOS attack..." (http://docs.yesodweb.com/blog/announcing-warp)
Neither of the supplied requests matches these criteria.
G
--
Gregory Collins

Thanks for the headers, that made the bug apparent. parseRequestBody
assumes there will be a space between the semi-colon and boundary=. I
can push a fix later today (assuming my internet still works...)
Michael
On Fri, Feb 25, 2011 at 3:53 PM, Nubis
On Fri, Feb 25, 2011 at 7:34 AM, Michael Snoyman
wrote: It might be getting confused by the request headers. Could you send those over as well? Also, what version of Yesod and wai-extra are you running?
Michael
Hi Michael, thanks for your reply and help
I'm running yesod 0.7.0, wai-0.3.1 and wai-extra-0.3.2
Please let me know if there's anything else I could do to try to diagnose this problem. I could compile a custom version of any library with more debugging info, for now the stepping debugger approach exceeds me.
Again, thanks a lot!
----nubis :)
This is the reelevant environment info for each request (got it from ruby though, I'm looking for a way to dump the whole request from wai): -------------------- dalvik
CONTENT_LENGTH:12098 CONTENT_TYPE:multipart/form-data;boundary=***** GATEWAY_INTERFACE:CGI/1.1 PATH_INFO:/ QUERY_STRING: REMOTE_ADDR:192.168.1.115 REMOTE_HOST:ganjizza REQUEST_URI:http://192.168.1.115:3000/ REQUEST_METHOD:POST HTTP_CONNECTION:Keep-Alive HTTP_COOKIE:_SESSION=fgUGM5J/k6mGAAW+MMXIJZCJHobw/oEbb6T17KQN0p9yNqiXn/m/ACrsnRjiCEgqtG4fogMUDI+jikoFGcwmPjvuD5d+MDz32iXvDdDJsFdsFMfivuey2H+n6IF6yFGD HTTP_USER_AGENT:Dalvik/1.1.0 (Linux; U; Android 2.1-update1; sdk Build/ECLAIR) HTTP_HOST:192.168.1.115:3000 HTTP_ACCEPT:*, */* HTTP_VERSION:HTTP/1.1 REQUEST_PATH:/
-------------------- chrome:
CONTENT_LENGTH:1169352 CONTENT_TYPE:multipart/form-data; boundary=----WebKitFormBoundaryEAoeasrCzOBAAP5s GATEWAY_INTERFACE:CGI/1.1 PATH_INFO:/ QUERY_STRING: REMOTE_ADDR:192.168.1.115 REMOTE_HOST:ganjizza REQUEST_METHOD:POST REQUEST_URI:http://192.168.1.115:3000/ HTTP_HOST:192.168.1.115:3000 HTTP_CONNECTION:keep-alive HTTP_REFERER:http://192.168.1.115:3000/static/client/index.html HTTP_CACHE_CONTROL:max-age=0 HTTP_ORIGIN:http://192.168.1.115:3000 HTTP_ACCEPT:application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 HTTP_USER_AGENT:Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.237 Safari/534.10 HTTP_ACCEPT_ENCODING:gzip,deflate,sdch HTTP_ACCEPT_LANGUAGE:en-US,en;q=0.8 HTTP_ACCEPT_CHARSET:ISO-8859-1,utf-8;q=0.7,*;q=0.3 HTTP_COOKIE:_SESSION=AoJJW9pxuycN7/bDU0+ho4auufmHtnZM0IiCp04EgXhyNqiXn/m/ACrsnRjiCEgqtG4fogMUDI+jikoFGcwmPhxQjE4k4AICWrp934EzZ53IyUdx5OqY9KhjYT8xSyfa; _recetas_session=BAh7B0kiD3Nlc3Npb25faWQGOgZFRiIlMjEyMTBlZjEzMThiOGNhZDU4Njk0NTdjZmNhYTM0MTJJIhBfY3NyZl90b2tlbgY7AEZJIjFkTVB0NGdPMFJVNHgzMk5QVC9sMXBWV3FVSXNNSStCZ2xqc0RxOXFVUk9zPQY7AEY%3D--b71e4e03f9b47a7b62bc977d45e9502123e5510a HTTP_VERSION:HTTP/1.1 REQUEST_PATH:/
participants (4)
-
Gregory Collins
-
Michael Snoyman
-
Nubis
-
Антон Чешков