Re: [Haskell-cafe] upload file with ftp

Hi Brandon,
The ftp-client I use (total commander) does not use passive mode. I changed the host to another server. And now I do not get an exception, no error messages, but nothing is uploaded.
Unfortunately is ‘working in passive mode’ not so easy (at least not to me). After login I send the ‘pasv’ command and I get back a IP address and a port (I assume). Do I have to create a new connection with a handle to this new port for the stor command? I don’t know how to do that in network.ftp.client. A hint or better a working example would be nice.
Kees
Van: Brandon Allbery [mailto:allbery.b@gmail.com]
Verzonden: dinsdag 5 januari 2021 13:49
Aan: Kees Bleijenberg

Am 05.01.21 um 16:54 schrieb Kees Bleijenberg:
Unfortunately is ‘working in passive mode’ not so easy (at least not to me). After login I send the ‘pasv’ command and I get back a IP address and a port (I assume). Do I have to create a new connection with a handle to this new port for the stor command?
Yes.
I don’t know how to do that in network.ftp.client.
If network.ftp.client does not support that out of the box, you'll either have to find another library or add PASV support to it. My knowledge of the FTP protocol is pretty rusty, but I believe you the model after PASV is that you continue using normal commands but the data transfers happen over the other connection. I'd watch out for how the protocol marks the beginning and end of a data stream (it's not a single connection anymore so you lose the sequencing guarantees of TCP packets; I'd assume they either send a special packet on the data connection, or they transmit the file sizes via the normal channel). However, I can't avoid wondering why anybody would want to use the FTP protocol at all. It's so unsafe, so bottleneck-endangered, the only reason to do that would be interfacing with existing FTP servers, and even these are being phased out in favor of HTTPS, with POST for uploads. (I'm not questioning the validity of your use case, just wondering how such a use case can come up in today's world.) Regards, Jo

In normal mode the server opens a connection to the FTP client; in PASV
mode the client instead opens a connection to the server, so it'll work
through a firewall.
On Tue, Jan 5, 2021 at 2:09 PM Joachim Durchholz
Am 05.01.21 um 16:54 schrieb Kees Bleijenberg:
Unfortunately is ‘working in passive mode’ not so easy (at least not to me). After login I send the ‘pasv’ command and I get back a IP address and a port (I assume). Do I have to create a new connection with a handle to this new port for the stor command?
Yes.
I don’t know how to do that in network.ftp.client.
If network.ftp.client does not support that out of the box, you'll either have to find another library or add PASV support to it.
My knowledge of the FTP protocol is pretty rusty, but I believe you the model after PASV is that you continue using normal commands but the data transfers happen over the other connection. I'd watch out for how the protocol marks the beginning and end of a data stream (it's not a single connection anymore so you lose the sequencing guarantees of TCP packets; I'd assume they either send a special packet on the data connection, or they transmit the file sizes via the normal channel).
However, I can't avoid wondering why anybody would want to use the FTP protocol at all. It's so unsafe, so bottleneck-endangered, the only reason to do that would be interfacing with existing FTP servers, and even these are being phased out in favor of HTTPS, with POST for uploads. (I'm not questioning the validity of your use case, just wondering how such a use case can come up in today's world.)
Regards, Jo _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
-- brandon s allbery kf8nh allbery.b@gmail.com

Joachim, Brandon, After investigating the source for the store command in Network.FTP.Client (https://hackage.haskell.org/package/ftp-client-0.5.1.4/docs/Network-FTP-Clie...) I found that the stor command already sets the connection in passive mode. I tried my original code on Linux and everything works fine. So probably the problem is (as always) Windows. A few years ago I tried some Haskell ftp client libs. On Windows they all raised access violations as soon as I connected to a server. Now, a few years later the Network library has improved (not sure) and I tried it again. Big improvements: Login, get a list of files, change directory works fine but stor does not. So there is still no working FTP-client lib for Haskell (I think). I know FTP is not secure but it is easy to use. Almost everybody uses it for uploading files to a website. Writing code for uploading a file with http(s)in Haskell is a lot more work than ftp and I wonder whether the https stuff will work on Windows. And with http(s) you have to write an extra server side program (php) to save the uploaded file, return error messages... I just want a simple way to upload a generated file to my private website. Until now I upload a file by running a program within my Haskell program. That ftp-program sends a file with ftp to a server. The ftp-program is written in Delphi/Lazarus. Thanks for your help. Kees -----Oorspronkelijk bericht----- Van: Haskell-Cafe [mailto:haskell-cafe-bounces@haskell.org] Namens Joachim Durchholz Verzonden: dinsdag 5 januari 2021 20:09 Aan: haskell-cafe@haskell.org Onderwerp: Re: [Haskell-cafe] upload file with ftp Am 05.01.21 um 16:54 schrieb Kees Bleijenberg:
Unfortunately is ‘working in passive mode’ not so easy (at least not to me). After login I send the ‘pasv’ command and I get back a IP address and a port (I assume). Do I have to create a new connection with a handle to this new port for the stor command?
Yes.
I don’t know how to do that in network.ftp.client.
If network.ftp.client does not support that out of the box, you'll either have to find another library or add PASV support to it. My knowledge of the FTP protocol is pretty rusty, but I believe you the model after PASV is that you continue using normal commands but the data transfers happen over the other connection. I'd watch out for how the protocol marks the beginning and end of a data stream (it's not a single connection anymore so you lose the sequencing guarantees of TCP packets; I'd assume they either send a special packet on the data connection, or they transmit the file sizes via the normal channel). However, I can't avoid wondering why anybody would want to use the FTP protocol at all. It's so unsafe, so bottleneck-endangered, the only reason to do that would be interfacing with existing FTP servers, and even these are being phased out in favor of HTTPS, with POST for uploads. (I'm not questioning the validity of your use case, just wondering how such a use case can come up in today's world.) Regards, Jo _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Am 06.01.21 um 10:42 schrieb Kees Bleijenberg:
Joachim, Brandon,
After investigating the source for the store command in Network.FTP.Client (https://hackage.haskell.org/package/ftp-client-0.5.1.4/docs/Network-FTP-Clie...) I found that the stor command already sets the connection in passive mode. I tried my original code on Linux and everything works fine. So probably the problem is (as always) Windows.
Unlikely it's Windows' network stack - there is nothing special about an outgoing FTP connection.
A few years ago I tried some Haskell ftp client libs. On Windows they all raised access violations as soon as I connected to a server. Now, a few years later the Network library has improved (not sure) and I tried it again. Big improvements: Login, get a list of files, change directory works fine but stor does not. So there is still no working FTP-client lib for Haskell (I think).
Or the difference is that you have a different firewall setup. Possibly caused by Windows itself, or maybe some antivirus product. You should be able to test this hypothesis by using telnet or some other program that allows you to open any port. Access violations are typically the result of opening a reserved port without proper permissions.
I know FTP is not secure but it is easy to use.
It is not, actually. You have to worry about: * passive vs. normal mode, * text vs. binary mode (text mode is obsolete since more than a decade), * directory listings being misrepresented because * encoding issues * some FTP server with its own idea of how to represent these (manageable with command-line clients, GUI clients may fall over) * flipped bits (TCP does only so much here), * aborted transfers (supported but not with client/server mismatch) * security: your FTP password is visible to anybody in the same network segment (including malware they might have caught). rsync, sftp and scp eliminate every single one of these worries.
Almost everybody uses it for uploading files to a website. Actually the vast majority of file uploads go through HTTP POST nowadays. It's been over a decade since I have even seen something with an FTP upload, though I guess very old services might still stick to it just because upgrading is a pretty big step for all parties involved. My knee-jerk reaction to an FTP upload site: Big Red Flag. Admins who ignore basic security advice, like not offering a server site that forces you to send passwords in the clear. (sftp is NOT affected by this. Protocol-wise, it's a totally different technology.)
Writing code for uploading a file with http(s)in Haskell is a lot more work than ftp and I wonder whether the https stuff will work on Windows.
Between Linux and Windows, HTTPS does not have any significant differences. Hmm... well, okay, it is different if you want to use the system's proxy settings and certificate store. I don't know how that works, so I can't say much about this. However, I believe the bigger difference is between HTTP and HTTPS. Setting up the certificates and doing the connection handshake is a pretty complicated stuff. I don't know good the Haskell https libraries are at this; at least for Java, I can report that the default libraries do work but aren't exactly good at controlling the connection parameters or accurately reporting error situations. (That's why everybody uses Apache HttpClient, except the people who wrote the legacy app I'm tasked with maintaining and improving... well, if everything worked, there wouldn't interesting work, so there :-) )
And with http(s) you have to write an extra server side program (php) to save the uploaded file, return error messages... I just want a simple way to upload a generated file to my private website.
Use the SSH-based protocols then - either sftp/ssh (server side is the same, but without root access setup can be complicated) or rsync (almost universally available, and setting up an rsync server should be simple). Caveat: I don't know whether Haskell libraries for any of these exist. BTW dealing with HTTP POST is easier than setting up your own server. Been there, done that, on both sides.
Until now I upload a file by running a program within my Haskell program. That ftp-program sends a file with ftp to a server. The ftp-program is written in Delphi/Lazarus.
In that case, you'll be better off just falling back to rsync. Expect to spend an afternoon sifting through the options if you want to transfer entire directories. If your server doesn't know how to handle rsync and you can't isntall it: Use a different provider. FTP is generally the worst option, in oh so many ways. Really. Scout's honor. Regards, Jo
participants (3)
-
Brandon Allbery
-
Joachim Durchholz
-
Kees Bleijenberg