On Thursday 15 March 2007 18:57, Bulat Ziganshin wrote:

Thursday, March 15, 2007, 8:10:18 PM, you wrote:

...

What is the general strategy for Haskell' regarding this topic: Is I/O out of the scope of Haskell' itself? If not, what are the plans/the strategy for transitioning to something more correct? To be honest: I don't see an easy way out without breaking lots of existing programs.

i see: just providing operations in libraries and allow one to import libraries he need. so, one can import FilePath 1.0 with String-based path representation or FilePath 2.0 with ByteString-based one

Alas, this is no solution at all. The point is: Currently there is no explicit conversion FilePath <-> String, although this is really needed. Users e.g. enter FilePaths somehow through a GUI, on the command line, in a configuration file etc., so you often start with a String. But this has to be converted somehow to a bunch of bytes for accessing the file, and this can't be handled without an API change (where does the encoding come from?). The same holds for reading e.g. a directory which should be presented to the user, only the other way round: You get a bunch of bytes and have to decode these for presentation purposes. Specifying a "default" encoding is not really the right way to go, because this leads to sloppy programs which only work under special circumstances and suddenly break on other platforms with different defaults. Cheers, S. P.S.: The command line arguments are not really strings in *nix, either, so the current API is wrong, too.

On Thursday 15 March 2007 19:18, Neil Mitchell wrote:

Surely both FilePath's and command line arguments are sufficiently string like that a reasonable interpretation of them is String? We don't want to provide an interface to Unix, we want to provide a proper abstraction over the underlying details.

It would be a shame if on my Windows box I had to jump through artificial hoops because Posix is a bit broken...

I'm lacking the time to list all those artificial hoops Windows has brought us, so I only urge everybody again: Let's not oversimplify things and handle things the Windows way. Although I would love to have a world where POSIX & friends used Unicode in tons of places, this is simply not the case. FilePath is meant to be an *abstraction*, hiding platform differences, and FilePath = String miserably fails to do this. Haskell got this wrong initially, but I'm hoping that Haskell' addresses this issue. As a compromise, we can leave the current FilePath as it is until things are settled, but I would really like to have a big, fat warning in the documentation then... Cheers, S.

Hello Neil, Thursday, March 15, 2007, 9:43:48 PM, you wrote:

Do you have a concrete proposal for what FilePath's should look like in Haskell'? "FilePath = String" is currently the least broken design, since no one has put forward a better one.

does http://haskell.org/haskellwiki/Library/IO not count? -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

Hello Neil, Thursday, March 15, 2007, 10:54:03 PM, you wrote:

...

...

Do you have a concrete proposal for what FilePath's should look like in Haskell'? "FilePath = String" is currently the least broken design, since no one has put forward a better one.

does http://haskell.org/haskellwiki/Library/IO not count?

It is a start, but not the whole answer. What is the type of readFile? Stringable a =>> a -> IO String ?

yes, if we say only about Filepath part and not about type for representing file contents. another variant - provide some FilePath class. but providing your module as separate library will allow to improve this while keeping backward compatibility while including it into base makes this scenario very problematic

Stringable a =>> What about getContents? Can all this be implemented without breaking existing code? Does this stop people from chopping and changing at strings? Is that desirable?

the beauty of this proposal is that we can issue 3rd, 4th and any other version based on real users feedback. including some module into base means stopping of its active development, you can see the situation with ByteString

There are also lots of other questions, does this code want to go into base? Do we want to replace the existing code in base?

Plenty of open questions, I suspect solving them all will take months at the very least...

i think that existing base should be conserved for keeping compatibility with already developed apps/libs and new functionality should be provided in form of cabal packages that can be easily upgraded and even several versions can co-exist in one installation then, functionality should be moved step-by-step from base to these new packages. new functionality should go immediately into packages. otherwise, for example, your shining new version of Cabal which uses System.FilePath module, will be not compatible with ghc 6.6, and the same will be true for all other apps/libs then these packages will evolve. backward compatibility will be provided by supplying old versions of libraries (via hackage or preinstalled)

Is breaking nhc/yhc for something as fundamental as open files acceptable?

i don't understood this question -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On Fri, Mar 16, 2007 at 09:44:26AM +1100, Duncan Coutts wrote:

Of course if a module is still maturing and in a state of flux then adding it to base wouldn't be a good idea as one could not promise API stability. System.FilePath doesn't seem to be in that situation however, in my opinion it's ok to go into base (and come out again if/when base gets split up).

I think that's roughly the argument you made for FPS, but looking at the repo history there are at least these API changes since the GHC 6.6 release: * export the right 'cycle' * Add a unit tests file, a test that append is lazy in the tail, and make it so. * "HEADS UP: Change CString api" * Make the lazy cons lazy, and add a cons' for when you want it to be strict * Make fromForeignPtr take the start so it is truly the inverse of toForeignPtr * Define headTail :: ByteString -> Maybe (Word8, ByteString) Sure, all of these can be worked around by importing internal modules and copying the corrected definitions (as I have had to do), but it would be much simpler if I could just depend on bytestring >= 1.1. Thanks Ian

Hello Neil, Friday, March 16, 2007, 6:22:05 AM, you wrote:

There is a key difference here, ByteString is actively being worked on as a research area. I'm 100% happy with FilePath, I don't intend to go anywhere near it - I have plenty of research to do and none of it involves putting /slashes\ the right way round!

yes, *you* are happy with it and therefore propose to add it to the base. *i* want to see it evolved sometime to support various bytestring kinds and therefore propose to keep it in separate library. otherwise, we will got either incompatibility between ghc versions or legacy system.filepath in base and new version in some 3rd-party library -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On Fri, 2007-03-16 at 02:55 +0000, Ian Lynagh wrote:

On Fri, Mar 16, 2007 at 09:44:26AM +1100, Duncan Coutts wrote:

...

Of course if a module is still maturing and in a state of flux then adding it to base wouldn't be a good idea as one could not promise API stability. System.FilePath doesn't seem to be in that situation however, in my opinion it's ok to go into base (and come out again if/when base gets split up).

I think that's roughly the argument you made for FPS, but looking at the repo history there are at least these API changes since the GHC 6.6 release:

* export the right 'cycle' * Add a unit tests file, a test that append is lazy in the tail, and make it so. * "HEADS UP: Change CString api" * Make the lazy cons lazy, and add a cons' for when you want it to be strict * Make fromForeignPtr take the start so it is truly the inverse of toForeignPtr * Define headTail :: ByteString -> Maybe (Word8, ByteString)

Sure, all of these can be worked around by importing internal modules and copying the corrected definitions (as I have had to do), but it would be much simpler if I could just depend on bytestring >= 1.1.

So some of these are just bug fixes and not api changes so will be in 6.6.1. But in general: so you could say that these bugs show we should have waited longer for the library to mature, on the other hand I rather suspect that we'd never have found these without the huge number of people using the lib that came from it being included as standard. That was not the only reason to include it of course. As I've mentioned before, a reason to keep it in base in future is that it gives us the opportunity to rewrite programs that don't even import ByteString into more efficient versions, for example: f . lines =<< readFile "foo" to f . map unpack . B.lines =<< B.readFile "foo" (and there may be further fusion opportunities if f is a fusible list consumer) Duncan

On Fri, 2007-03-16 at 12:54 +0300, Bulat Ziganshin wrote:

...

So some of these are just bug fixes and not api changes so will be in 6.6.1.

and some will be deferred until 6.8 when we will have one more incompatibility-break? thank you

The changes are really pretty minor and only in the very low level part of the api.

...

But in general: so you could say that these bugs show we should have waited longer for the library to mature, on the other hand I rather suspect that we'd never have found these without the huge number of people using the lib that came from it being included as standard.

there are no differences in this aspect between base and any other bundled library

Bulat, I'm not arguing against making base a bit more modular, but you know how hard a task that is. Just yesterday we were struggling with issues of circular dependencies in the modules in base as we're experimenting with a new list library implementation. Heh, just to add to the confusion we've got our experimental list lib available as a separate package for people to play with, though obviously for it to be really practical in the long run it has to replace the list implementation, which (oh noes!) means sticking it in base. (Since otherwise you can't desugar things like list comprehensions into the new implementation.)

oh my god! you plan to further evolve base library and its legacy i/o part. but how about using unicode filenames or newer async i/o api? it should be also implemented inside base? one library for anything? :)

It's quite independent. It could work with any standard IO system. I mention the H98 readFile etc because that's what we have right now. Note that one could take advantage of async IO without changing the Handle API.

i think that it will be much better if fps will be detached from base and you will provide i/o library on top of fps and lowIO [1] that makes all these cool things. this way you will got support of all the features lowIO provides for free

As I said, it's quite independent. You could use ByteStrings with any sane IO library since all it needs is to be able to read/write data buffers. Duncan

Duncan Coutts wrote:

But in general: so you could say that these bugs show we should have waited longer for the library to mature, on the other hand I rather suspect that we'd never have found these without the huge number of people using the lib that came from it being included as standard.

I certainly don't argue against bundling FPS (or any other package) with GHC (or any other compiler), and I don't think Bulat (or anybody else) is either. The problem is *packages*: we have a nice and flexible infrastructure for manipulating these, but with the exception of 'base'. When a library is in base, I can no longer depend on a particular version, upgrade to current development version, experiement with modifications, etc. With any other package, I can do so, and together with darcs and cabal, lots of code is very accessible, with a relatively smooth incline for getting gradually more involved. The 'base' package has a much wider gap between consumers and developers. To avoid confusion, could we use a separate term to describe required bundled libraries, "core libraries" perhaps? -k

On Fri, 2007-03-16 at 14:03 +0000, Ian Lynagh wrote:

On Fri, Mar 16, 2007 at 08:07:24PM +1100, Duncan Coutts wrote:

...

suspect that we'd never have found these without the huge number of people using the lib that came from it being included as standard.

Please, please, please, packages can come "as standard" (i.e. be a core package) without having all their modules be part of the base package.

Yes.

...

As I've mentioned before, a reason to keep it in base in future is that it gives us the opportunity to rewrite programs that don't even import ByteString into more efficient versions, for example:

f . lines =<< readFile "foo"

to

f . map unpack . B.lines =<< B.readFile "foo"

(is B ByteString.Lazy here?)

Yes, that's the only way to preserve the semantics.

If we want to do that then we'd either need to put ByteString into base or split io out into its own package,

Aye, that'd probably work. I don't think the above transformation requires the list implementation be in the same package, since you could put rules into the bytestring/io lib that match on things defined in the list lib, eg: lines . B.unpack = map B.lines . B.lines

yes (and incidentally, the latter I think would be fantastic; looking only at the cabal dependencies and (transitively, admittedly) extensions you would be able to see if a library was able to do IO).

And if the dependencies (transitively) do not use the FFI which you can check by looking in the .cabal file and every module for OPTIONS pragmas. Not trivial but at least it could be automated. Duncan

Chris Kuklewicz wrote:

Ian Lynagh wrote:

...

...

suspect that we'd never have found these without the huge number of people using the lib that came from it being included as standard. Please, please, please, packages can come "as standard" (i.e. be a core

On Fri, Mar 16, 2007 at 08:07:24PM +1100, Duncan Coutts wrote: package) without having all their modules be part of the base package.

A useful example of package coming "as standard" with GHC 6.6 is Text.Regex.

In GHC 6.4 all of Text.Regex and Text.Regex.Posix was part of the base package.

I wrote a replacement set of packages: regex-base providing Text.Regex.Base* regex-posix providing Text.Regex.Posix* regex-compat provided Text.Regex

The replacement Text.Regex was exactly the same API as the old Text.Regex, but used the new regex-posix package as the lower level.

The new lower level Text.Regex.Posix was compatible with old Text.Regex.Posix

wrote too fast, I meant The new lower level Text.Regex.Posix was incompatible with old Text.Regex.Posix ^^^^^^^^^^^^

On Thursday 15 March 2007 19:43, Neil Mitchell wrote:

What would the warning say?

"Haskell currently does not work when reading files or accessing command line arguments on Unix. Until such time as this is fixed please avoid opening files, or allowing the user to pass command line flags. The workaround is to either use Windows or Perl." :-)

At least this would be a starting point... ;-)

Do you have a concrete proposal for what FilePath's should look like in Haskell'? "FilePath = String" is currently the least broken design, since no one has put forward a better one.

Of course I can't come up with an elaborate proposal in 5 minutes, but I think the main points should be: * FilePath is abstract, perhaps even constructed from FilePathElements (there are pros and cons for introducing the latter, I'm not totally sure about all implications). * Explicit operations for constructing and accessing a FilePath are provided. * Operations like the ones in your proposal are provided. Before one can make a concrete proposal, a proposal for a general encoding framework is needed and should be standardized, because we need a "fit" here. Another option would be handling things exactly the other way round: FilePath could be a synonym for ByteString, so things could look like the following (assuming functions from our imaginary encoding framework) : ... entries1 <- getDirectoryContents (encode utf8 "blah") mapM_ (putStrLn . decode utf8) entries1 entries2 <- getDirectoryContents myByteString ... But in general, I have a bad feeling about type synonyms for such cases... Cheers, S.

On Thu, Mar 15, 2007 at 06:43:48PM +0000, Neil Mitchell wrote:

Do you have a concrete proposal for what FilePath's should look like in Haskell'? "FilePath = String" is currently the least broken design, since no one has put forward a better one.

I proposed something in http://www.haskell.org/pipermail/libraries/2005-July/004189.html although I never made code for it. I agree that we don't need to wait for this problem to be solved before doing something with your FilePath module, though. Thanks Ian

On Friday 16 March 2007 00:15, Wolfgang Thaller wrote:

Indeed, paths and command-line arguments are becoming very string- like on Unix systems, too. On Mac OS X, the locale for file names is pretty much hardcoded to UTF-8. Mac OS X's native file system stores file names in UTF-16, but the POSIX layer sees it as UTF-8. [...]

I had a look how other languages and toolkits handle this issue. For those which do not completely ignore it, the consensus seems to be: * On Mac OS X, the POSIX layer indeed seems to use UTF-8, but in a *decomposed* form. This could be a little bit surprising, so some normalization is probably needed. * On Windows, the current ANSI code page is assumed, which could vary from installation to installation and can be changed by the user AFAIK. * For *nices the story is a bit tricky, but often a combination of * nl_laninfo(CODESET) * setlocale(LC_TYPE, 0) * the environment variables LC_ALL, LC_TYPE and LANG * iconv is used to figure out the current local encoding and use that. Depending on the distribution, it can be UTF-8 (e.g. recent SuSE distros), but it doesn't have to be. So I propose a compromise, we don't really have to be better than most languages/toolkits out there: Let's keep FilePath = String, but improve the real culprit, i.e. CString and friends. Currently, peekCString{,Len}, newCString{,Len} and withCString{,Len} simply use their "CA" ASCII counterparts. If we put the above common logic into Foreign.C.String, we could already achieve a lot. In addition, we might consider adding some e.g. ByteString-based API entries to the POSIX package for the real low-level stuff, but I think this is not a topmost priority. Opinions? Cheers, S.

On Friday 16 March 2007 14:26, Bulat Ziganshin wrote:

Friday, March 16, 2007, 4:24:47 PM, you wrote:

...

* On Windows, the current ANSI code page is assumed, which could vary from installation to installation and can be changed by the user AFAIK.

it was true for Win9x. NT-based systems supports new api which uses utf16 (of course they support old api too). generally, developers say that their programs supports unicode filenames if their programs supports this new api. its support implemented in Win32Files module from http://www.haskell.org/bz/FreeArc-sources.tar.gz

Well, I know that there is a "wide" API now, I just wanted to point out what is commonly done when an 8bit <-> character conversion is needed. The problem is that Haskell's I/O libs current use e.g. _sopen instead of _wsopen, so I should probably rephrase my proposal then as follows: * Improve CStrings and friends to do some actual de-/encoding with the common default for the platform * Use a "wide" API when available internally Cheers, S.

On Friday 16 March 2007 14:56, Bulat Ziganshin wrote:

Friday, March 16, 2007, 4:42:00 PM, you wrote:

...

Well, I know that there is a "wide" API now, I just wanted to point out what is commonly done when an 8bit <-> character conversion is needed. The problem is that Haskell's I/O libs current use e.g. _sopen instead of _wsopen, so I should probably rephrase my proposal then as follows:

* Improve CStrings and friends to do some actual de-/encoding with the common default for the platform

* Use a "wide" API when available internally

my proposal is to implement second as part of new low i/o library because it's not only issue that we should fix. so, instead of patching here and there, new modular design should be started

I would strongly object against waiting so long, because agreeing on a new I/O API will probably take months, and I'm seriously not expecting any kind of consensus even within this year for something as fundamental. The nice parts about my proposal, as imperfect as it is from a purely theoretical viewpoint, are: * :We can do this *now*. * We can do this incrementally behind the scenes, no big bang is needed. * The Haskell APIs stay as they are at the moment, only the semantics are improved. Cheers, S.

On Friday 16 March 2007 15:32, Bulat Ziganshin wrote:

it is just the source of our current problems with base. of course, it is easier to add something to base instead of refactoring it. so we have that we have - more and more features added here and it makes refcatoring more and more problematic. now FPS team wants to implement something in Base via bytestrings that will make them unremovable from base. i think that it is old way, new way to make software is small building blocks which are maintained independently [...]

I'm slowly getting tired of your arguments: If you think that your way is the right way, implement whatever you like and come back with what you've got afterwards for discussion, this is OpenSource SW, after all. Having highly modular libraries with perfect APIs everybody likes would be great, but this is not where we are and is not where we will be in the near future. Stopping all further improvements just because there is a Grand Shiny Master Plan in somebody's brain doesn't get us anywhere. I'm a big fan of the 80:20 rule, and my hope is that my tiny proposal might get us a long way until the upcoming API wars (there *will* be some, I can guarantee you, looking back at past discussions) are settled. Go ahead, split base, keep GHC/Hugs/nhc98 alive while doing it, implement your I/O on the tons of platforms we currently support, but be prepared for funny dependency cycles, built-in compiler knowledge, nasty platform/performance hacks, etc. The reason the libraries currently look as they are is not that the people writing them have been idiots or have never heard about the fundamentals of good SW design, the reason is that the libs slowly evolved from something rather simple and are maintained in a evolutionary manner with very few "big bangs". We do not have 100 paid people working full-time on this, after all... Cheers, S.

Hello Neil, Friday, March 16, 2007, 6:32:55 PM, you wrote:

This is exactly what I've done with System.FilePath. I have been through several iterations making the entire concrete source code available at each point. I have changed everything until no one seems to object anymore to anything in the library - only to type FilePath = String, which isn't in my library.

i like the result (and i like FPS, too)! all that i want is to have ability to upgrade it in future -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On Fri, Mar 16, 2007 at 04:24:21PM +0000, Malcolm Wallace wrote:

"Neil Mitchell" wrote:

...

So can I suggest that the people objecting to inclusion in base:

1) Write the code that adds filepath as a core library, update build scripts for nhc/ghc/hugs, talk to everyone and gather agreement to include the filepath library always,

OK, I have added the filepath package to the default builds of nhc98, and agree to distribute it in all future versions of the compiler.

I'm happy to do the work for adding it to GHC, but I'll need to talk to SimonM when he returns about the details first: It would make sense to either do the switch to building with cabal in 6.8 first (thus avoiding needing to add it the Makefile way), or to do it the Makefile way and also put it into 6.6.1 (which obviously has the advantage that it's available RSN by default with GHC; I don't think there's an interface issue here as it's a separate package, but I'd like to get Simon's input first!). If it's a sticking point then I'm sure I can work out how to get hugs to include it too, assuming Ross doesn't object. One thing: It would make things a lot simpler if filepath was moved to (or kept in sync with) http://darcs.haskell.org/packages/filepath/ (I know we could change darcs-all to take an optional repo URL, but it's messier when we want to fork the libraries into a ghc-6.8/ subdirectory). Thanks Ian

On Wed, Mar 21, 2007 at 12:42:34AM +0000, Ian Lynagh wrote:

If it's a sticking point then I'm sure I can work out how to get hugs to include it too, assuming Ross doesn't object.

I can do that; it's no big deal.

One thing: It would make things a lot simpler if filepath was moved to (or kept in sync with) http://darcs.haskell.org/packages/filepath/

Yes indeed. I think it should be a requirement for core packages (those guaranteed to be shipped with all implementations) that their master development repositories are in http://darcs.haskell.org/packages.

On Wed, 2007-03-21 at 11:26 +0000, Neil Mitchell wrote:

Hi

...

Yes indeed. I think it should be a requirement for core packages (those guaranteed to be shipped with all implementations) that their master development repositories are in http://darcs.haskell.org/packages.

Agreed. I also suggest that any visible interface changes have to go though the libraries list in exactly the same as for base.

With the exception of Cabal presumably. Duncan

On Wed, 2007-03-21 at 11:38 +0000, Neil Mitchell wrote:

Hi Duncan,

...

With the exception of Cabal presumably.

How far away is Cabal from presenting a stable interface? Is this not a goal in the long run?

It is a goal.

I realise Cabal is changing too fast at the moment, so perhaps apply this rule to Cabal at some future date.

Right. Duncan

On Wednesday 21 March 2007 18:32, Malcolm Wallace wrote:

[ Tangential note: nhc98's version of darcs-all already permits optional repo URLs to appear in the default-packages database, if you want to cross-steal that code back again. ]

Even more tangential note: When Hugs is in darcs, too, I'll merge all darcs-all scripts (for Hugs/GHC/nhc98), this will ease maintenance. Cheers, S.

Hi

...

I'm happy to do the work for adding it to GHC, but I'll need to talk to SimonM when he returns about the details first: It would make sense to either do the switch to building with cabal in 6.8 first (thus avoiding needing to add it the Makefile way), or to do it the Makefile way and also put it into 6.6.1 (which obviously has the advantage that it's available RSN by default with GHC; I don't think there's an interface issue here as it's a separate package, but I'd like to get Simon's input first!).

This hasn't come up before, but I don't see any reason why we shouldn't add to the set of core-packages we ship in a minor patchlevel release.

http://darcs.haskell.org/packages/filepath/ Once you've checked it meets whatever you need, I'll add a darcs tag. If anyone has any comments, please speak up quickly. I'm now NOT looking for remarks like "how about filepath's based on bytestrings?", but if anyone was to take a quick look over the structure, the use of CPP, anything else etc, I'd be grateful. Thanks Neil

Hello all, On Fri, 16 Mar 2007 22:42:00 +0900, Sven Panne wrote:

...

it was true for Win9x. NT-based systems supports new api which uses utf16 (of course they support old api too).( snip )

Well, I know that there is a "wide" API now, I just wanted to point out what is commonly done when an 8bit <-> character conversion is needed. The problem is that Haskell's I/O libs current use e.g. _sopen instead of _wsopen, so I should probably rephrase my proposal then as follows:

* Use a "wide" API when available internally

I trid to change by just changing from ascii API to wide API tonight. But ... there is one problem. _wopendir defined in mingwex library, and mingwex requires dll. http://www.mingw.org/MinGWiki/index.php/mingwex C:\home>C:\home\ghc-6.7.20070314\ghc-6.7.20070314\bin\ghci.exe ___ ___ _ / _ \ /\ /\/ __(_) / /_\// /_/ / / | | GHC Interactive, version 6.7.20070314, for Haskell 98 . / /_\\/ __ / /___| | http://www.haskell.org/ghc/ \____/\/ /_/\____/|_| Type :? for help. : C:/home/GHC-67~1.200/GHC-67~1.200/HSbase.o: unknown symbol `__wopendir' Loading package base ... linking ... : unable to load package `base' C:\home>C:\home\ghc-6.7.20070314\ghc-6.7.20070314\bin\ghci.exe ___ ___ _ / _ \ /\ /\/ __(_) / /_\// /_/ / / | | GHC Interactive, version 6.7.20070314, for Haskell 98 . / /_\\/ __ / /___| | http://www.haskell.org/ghc/ \____/\/ /_/\____/|_| Type :? for help. Loading package base ... : can't load .so/.DLL for: mingwex (addDLL: unknown error) I can't find mingwex from anywhere. So I think we must change this part. Best Regards, -- shelarcy <shelarcy capella.freemail.ne.jp> http://page.freett.com/shelarcy/

Hello shelarcy, Friday, March 16, 2007, 7:19:34 PM, you wrote:

...

* Use a "wide" API when available internally

I trid to change by just changing from ascii API to wide API tonight. But ... there is one problem. _wopendir defined in mingwex library, and mingwex requires dll.

i've used _wfindfirsti64/_wfindnexti64 functions. of course, their semantics isn't the same as semantics of _wopendir. look at Win32Files module from http://www.haskell.org/bz/FreeArc-sources.tar.gz - it implements all the function i use to work with files in my program your patch is great, but it is incompatible with Unix and Win9x systems may be, you will be interested to work on low-level IO library as proposed in http://haskell.org/haskellwiki/Library/IO ? -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

Hello shelarcy, Saturday, March 17, 2007, 6:01:20 AM, you wrote:

...

your patch is great, but it is incompatible with Unix and Win9x systems

Yes, it's just partial modified files. I don't test to build on Unix yet. But if we want to support Win98, we can use opencow (or MSLU) like Win32 package HEAD.

i've heard about this MSLU, but don't known that it is supported in Win32 package. so, we need to define only: open = #if WINDOWS wopen #else open #endif

...

may be, you will be interested to work on low-level IO library as proposed in http://haskell.org/haskellwiki/Library/IO ?

Yes, I'm interested in it.

in this case, i propose the following plan - write and test this library and only then write patch for the base. i think that debugging of independent library should be easier than working directly with base source modules for this library can be extracted from Streams (File, MappedFile) and base (System.Directory.*, System.Posix.Internals) if you are student, you can apply for SoC ticket dedicated to this work: http://hackage.haskell.org/trac/summer-of-code/ http://hackage.haskell.org/trac/summer-of-code/ticket/1113 http://hackage.haskell.org/trac/summer-of-code/wiki/StudApply -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On Fri, Mar 16, 2007 at 02:24:47PM +0100, Sven Panne wrote:

So I propose a compromise, we don't really have to be better than most languages/toolkits out there: Let's keep FilePath = String, but improve the real culprit, i.e. CString and friends. Currently, peekCString{,Len}, newCString{,Len} and withCString{,Len} simply use their "CA" ASCII counterparts. If we put the above common logic into Foreign.C.String, we could already achieve a lot.

I'm not clear on exactly what you're proposing: would getDirectoryContents "." >>= mapM_ removeFile remove all the files in ".", regardless of what my locale is, my OS is, etc? i.e. would this solve my problem in http://www.haskell.org/pipermail/libraries/2005-July/004189.html ? Thanks Ian

Hello Sven, Friday, March 16, 2007, 5:42:10 PM, you wrote:

On this platform, peekCString would decode from UTF-8 to Unicode (probably

For Windows platforms we should use the "wide" API internally, and the CString functions would be equivalent to their CWString counterparts.

of course, we can't change CString functions - they are used in many other places :) implementation just should use other functions. again, you may look on my Win32Files module. ideal Files module should at compile time select between unix/mac/win versions and on windows at runtime select between wide/legacy versions. also, i hope that it will support UTF8- and UTF16-encoded bytestrings as filenames because this allows to save time/space and absolutely required for programs that work with many thousands of files -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On Fri, Mar 16, 2007 at 03:42:10PM +0100, Sven Panne wrote:

On Friday 16 March 2007 15:10, Ian Lynagh wrote:

...

I'm not clear on exactly what you're proposing: would

getDirectoryContents "." >>= mapM_ removeFile

remove all the files in ".", regardless of what my locale is, my OS is, etc? i.e. would this solve my problem in

http://www.haskell.org/pipermail/libraries/2005-July/004189.html

OK, let's make this a bit more concrete and let's take Mac OS X as an example.

I was really hoping for an answer along the lines of "yes" or "no" :-)

On this platform, peekCString would decode from UTF-8 to Unicode (probably normalizing the Unicode string on the way),

Why normalise it? A lot of the time we'll just be converting back to a filename later, so this sounds like wasted work.

so no information is lost.

Right, the normalising won't lose information as the filesystem requires fully denormalised(or whatever the right word is) filenames AIUI.

OTOH, withCString would encode the Unicode string to UTF-8 (decomposing on the way? Wolfgang?). Both functions are used by getDirectoryContents internally, so this will be "Unicode-enabled" then, too. The main point here is that UTF-8 => Unicode => UTF-8 is lossless,

OK, so I think my answer is "Yes" for MacOS X, which makes sense as both the filenames and strings are unicode.

and the same holds for my proposed change for *nices, too, as long as the local encoding is invertible in this sense. I am not sure if there are encodings in use out there which do not have this property, but even if they are: All e.g. Qt-based programs would share the same problems.

I got lost somewhere along the way here. Linux filenames are (to a first approximation) arbitrary sequences of word8s, unrelated to the "local encoding". So I think the answer to my question is no, getDirectoryContents "." >>= mapM_ removeFile might not remove everything in "." (if, say, my French friend created them and we are using Linux/ext2)? Thanks Ian

On Thu, Mar 15, 2007 at 06:10:18PM +0100, Sven Panne wrote:

What is the general strategy for Haskell' regarding this topic: Is I/O out of the scope of Haskell' itself?

When I last asked Isaac it (well, the fate of the standard libraries in general rather than IO in particular) hadn't been decided, and I haven't heard anything since. I recently opened a haskell-prime ticket for this issue: http://hackage.haskell.org/trac/haskell-prime/ticket/118 It would be good to resolve this ASAP as a number of other issues are blocked on it. Thanks Ian

On 27/03/07, Simon Marlow wrote:

The main thing that Haskell' should do, IMO, is to get rid of everything IOish from the Prelude and drop the IO module. Then it becomes possible to disentangle the IO library from the base package (well, more possible at least), and we can think about experimenting with alternative IO APIs.

Personally, my hope for solving the libraries problem (libraries need to be updated more often than a compiler release rolls around), is cabal-install. IMO GHC should ship with a _very_ minimal set of libraries, perhaps none at all, then cabal-install should be used to install, and keep up-to-date, the associated libraries. Then changes like adding System.FilePath will take place within a week or two, not within a year or two. As I see it, this approach still has two major disadvantages: 1) It makes it impractical to code Haskell without a network connection, although this is become less of a problem all the time (who currently does code without one?) 2) Breaking backward-compatibility would have to be treated _very_ seriously, as otherwise we'd end up with literally hundreds of different versions of the libraries, none of which compatible with one another, and with applications written with only specific versions in mind. Still, food for though. Don't let's detract from the System.FilePath discussion, though. -- -David House, dmhouse@gmail.com

Hello Andres, Thursday, March 15, 2007, 10:31:56 PM, you wrote:

...

a) Add to base.

b) Add as a library which we demand all Haskell compilers ship, and tweak Cabal to automatically use this library at all times. I think option 2a would be a huge improvement over the situation we have right now, so I'd be strongly in favor of adding FilePath to base for the time being.

i think that it will be degradation because new base version will be not available for users before 6.8 arrives and then it will be hard to change this interface without breaking compatibility with existing code. otoh, providing it as separate library solves these problems. if developer is too lazy to cabalize his code, its not the cause to add his module to the base. let's wait for someone who will be more eager -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

Hello ArtemGr, Friday, March 16, 2007, 12:22:22 PM, you wrote:

I don't think so. If you use a complex library which uses the old FilePath API, you usually stuck with the old FilePath API. For example, HAIFA (http://www.dcs.shef.ac.uk/~simonf/HAIFA.html) requires SYB3, which i couldn't find on the web, so, it being a separate package from the base didn't solve the problem.

of course, to solve this problem, one should also upload his libraries to hackage :) this may be not possible for syb3 because it's closely coupled with ghc - i don't know -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

Hello ArtemGr, Friday, March 16, 2007, 1:14:24 PM, you wrote:

Another example is hs-plugins (http://www.cse.unsw.edu.au/~dons/hs-plugins/). It still doesn't work with GHC6.6, as far as i know, and as such is simply unusable, despite all its greatness.

The same goes to your compression library (http://www.haskell.org/haskellwiki/Library/Compression), which i couldn't build with recent GHC under CYGWIN.

are you tried to aks their authors? :) i never really supported Compression library because there are no user requests. really, i evolve it as a part of my app (http://www.haskell.org/bz/FreeArc-sources.tar.gz) and it definitely compiled on my box with 6.4 and 6.6. i use mingw, though -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com