Wait... System.Posix.IO.ByteString reads data into a *String*?

I was very surprised to find "fdRead" returning a String in the ByteString
version of the library:
http://hackage.haskell.org/packages/archive/unix/2.5.1.0/doc/html/System-Pos...
Unlike, for example the fdRead in unix-bytestring:
http://hackage.haskell.org/packages/archive/unix-bytestring/0.3.5/doc/html/S...
I belatedly found the below thread when trying to figure out what went down
with unix-2.5.1.0, and it seems that the impetus for the ".ByteString"
modules was to change file path representation, not the representation of
file contents.
But I'm missing why this would be. If it is a distinct module namespace,
why would there be any backwards compatibility concerns? And if taking
this one step towards ByteString why not go all the way? (Especially as
file contents are much larger and therefore more performance critical than
file names.)
-Ryan
On Fri, Nov 11, 2011 at 11:23 AM, Simon Marlow
I propose to commit the attached patch to the unix package and release it with GHC 7.4.1. The commit log is reproduced below. Comments please!
The unix version number will of course be bumped appropriately.
Cheers, Simon
commit d5e43be90d3c6f8869dd2b0c65800c**9a6dd0ac70 Author: Simon Marlow
Date: Fri Nov 11 16:18:48 2011 +0000 Provide a raw ByteString version of FilePath and environment APIs
The new module System.Posix.ByteString provides exactly the same API as System.Posix, except that:
- There is a new type: RawFilePath = ByteString
- All functions mentioning FilePath in the System.Posix API use RawFilePath in the System.Posix.ByteString API
- RawFilePaths are not subject to Unicode locale encoding and decoding, unlike FilePaths. They are the exact bytes passed to and returned from the underlying POSIX API.
- Similarly for functions that deal in environment strings (System.Posix.Env): these use untranslated ByteStrings in System.Posix.Environment
- There is a new function
System.Posix.ByteString.**getArgs :: [ByteString]
returning the raw untranslated arguments as passed to exec() when the program was started.
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

On Mon, Feb 20, 2012 at 00:39, Ryan Newton
I belatedly found the below thread when trying to figure out what went down with unix-2.5.1.0, and it seems that the impetus for the ".ByteString" modules was to change file path representation, not the representation of file contents.
The point of that was that, at the time, GHC had changed FilePath in a way that was in fact rather broken. It has since been fixed, but still has no way to represent a *raw* FilePath but instead makes the non-POSIX (and in fact, only true on OS X and even then only guaranteed for paths used from Carbon/Cocoa apps) assumption that FilePath-s have known encodings. The ByteString version of the module is intended to allow apps that actually work reliably on Unix/POSIX systems to be written. The other use would perhaps be nice, but is already covered by the ByteString package, and while I understand that it is not considered appropriate to want to be able to write reliable filesystem-using applications on POSIX systems in most developer communities these days, some of us would still like the ability to do so. -- brandon s allbery allbery.b@gmail.com wandering unix systems administrator (available) (412) 475-9364 vm/sms

On 2/20/12 12:57 AM, Brandon Allbery wrote:
The ByteString version of the module is intended to allow apps that actually work reliably on Unix/POSIX systems to be written.
Then it should have been named System.Posix.IO.RawFilePath or similar, rather than clashing with unix-bytestring. -- Live well, ~wren

On 2/20/12 12:39 AM, Ryan Newton wrote:
I belatedly found the below thread when trying to figure out what went down with unix-2.5.1.0, and it seems that the impetus for the ".ByteString" modules was to change file path representation, not the representation of file contents.
It's worth mentioning that the unix-bytestring package grew out of my attempts at making System.Posix.IO efficient. Before releasing that package I sought inclusion into unix proper, however I was rebuffed because (other than the use of ByteStrings) I kept the same API--- but it turns out that noone liked the original API! Thus, I spun the package off in order to redevelop the API into something nicer; hoping that it would be included in the future. The issue about using ByteStrings for file names came about in a separate discussion because of some concerns about roundtripping file names on GHC 7.2. Given that the aforementioned unix-bytestring discussion happened on this list and in recent memory, I am *very* disappointed that the new version of unix has decided to steal the namespace that was specifically set aside for unix-bytestring. -- Live well, ~wren

Here's one more point for a unix-bytestring like behavior: Brandon above mentioned that the ByteString already provides IO routines, which is true. But the reason I went looking for System.Posix.IO.ByteString was that I was writing a named pipes backend for "distributed-process". https://github.com/haskell-distributed/distributed-process This was a pain in the ass because of the tricky blocking / erroring semantics of named pipes and the only way I ended up getting it working was by using the System.Posix.IO "raw system calls", not with any layer above that. But without unix-bytestring I've implemented a pretty silly low performance transport! Cheers, -Ryan It's worth mentioning that the unix-bytestring package grew out of my
attempts at making System.Posix.IO efficient. Before releasing that package I sought inclusion into unix proper, however I was rebuffed because (other than the use of ByteStrings) I kept the same API--- but it turns out that noone liked the original API! Thus, I spun the package off in order to redevelop the API into something nicer; hoping that it would be included in the future.
The issue about using ByteStrings for file names came about in a separate discussion because of some concerns about roundtripping file names on GHC 7.2. Given that the aforementioned unix-bytestring discussion happened on this list and in recent memory, I am *very* disappointed that the new version of unix has decided to steal the namespace that was specifically set aside for unix-bytestring. ~wren

On 21 February 2012 15:00, wren ng thornton
On 2/20/12 12:39 AM, Ryan Newton wrote:
I belatedly found the below thread when trying to figure out what went down with unix-2.5.1.0, and it seems that the impetus for the ".ByteString" modules was to change file path representation, not the representation of file contents.
It's worth mentioning that the unix-bytestring package grew out of my attempts at making System.Posix.IO efficient. Before releasing that package I sought inclusion into unix proper, however I was rebuffed because (other than the use of ByteStrings) I kept the same API--- but it turns out that noone liked the original API! Thus, I spun the package off in order to redevelop the API into something nicer; hoping that it would be included in the future.
The issue about using ByteStrings for file names came about in a separate discussion because of some concerns about roundtripping file names on GHC 7.2. Given that the aforementioned unix-bytestring discussion happened on this list and in recent memory, I am *very* disappointed that the new version of unix has decided to steal the namespace that was specifically set aside for unix-bytestring.
I am sorry about that - I hadn't realised you were using the same module names. All the more reason to work on getting your APIs moved into the unix package proper. Cheers, Simon
participants (4)
-
Brandon Allbery
-
Ryan Newton
-
Simon Marlow
-
wren ng thornton