On 22/09/2009 16:43, Duncan Coutts wrote:

On Mon, 2009-09-21 at 12:00 +0100, Simon Marlow wrote:

...

The main change is that

splitFileName "foo" = (".", "foo")

and

takeDirectory "foo" = "."

and

"." x = x

So to re-iterate the point made in the ticket, the principle here is that the representation of the current directory is "." and not "". This lets us safely use results of System.FilePath functions as arguments to system and directory functions without having to check for "".

However the principle is not without exception, we also want to elide the explicit representation of the current directory when it is not needed.

If these are the principles then can we look at whether we apply them elsewhere in System.FilePath consistently?

What about:

joinPath [".", "foo"]

should it be "./foo" or "foo" ?

If "./foo" then it's not the same as, if the latter then we loose the property

Valid x => joinPath (splitPath x) == x

it would be some kind of limited normalisation.

So I'm happy with the first two changes, I'm less convinced about changing to elide "." on the left. Perhaps people who worry about leading "./" when the FilePath is displayed to the user should just use normalise. That's what they do now and we seem to get along ok.

So this subject was discussed between Ian and myself in the original thread, see e.g. http://www.haskell.org/pipermail/libraries/2007-December/008776.html The conclusion was that in the filesystem semantics "./foo" is equal to "foo", but the string passed to rawSystem (and execp()) is not a FilePath, it is something like Either FilePath String. However, there are some oddities with the current proposal. e.g. splitFileName "./foo" == ("./", "foo") "./" "foo" == "./foo" The current proposal just about hangs together because the tiny bit of normalisation that does exactly undoes the creation of "." in splitFileName, and there's no other way that splitFileName can generate ".". That's a terribly fragile property. If we "fixed" to do more normalisation, then we would no longer have the property that uncurry () (splitFileName x) == x (*1) I don't actually care about the details here as long as we have a story that is reasonably consistent. My main concern is that isValid x => isValid (takeDirectory x) the lack of which is the main problem with the current formulation, as you (Duncan) mentioned above. Perhaps we should drop the magic normalisation that does, and apply the normalise function to both sides of (*1). There would probably be a bunch more properties that would have to change too, I'm guessing that normalise would proliferate. Incedentally, I dislike the way that trailing slashes are treated in the current filepath implementation. A trailing slash is significant in POSIX: $ ls -l foo lrwxrwxrwx 1 simonmar GHC 15 2009-09-23 14:04 foo -> /does/not/exist $ ls foo foo@ $ ls foo/ ls: cannot access foo/: No such file or directory The path with the trailing slash dereferences a symbolic link, and may fail if the link points nowhere. I think trailing slashes should be dropped by splitFileName. Cheers, Simon

On 23/09/2009 14:36, Bulat Ziganshin wrote:

Hello Simon,

Wednesday, September 23, 2009, 5:17:58 PM, you wrote:

...

...

...

The main change is that

let me ask another question - are it's expected to raise FilePath library major version number?

Yes, this is an incompatible change and would therefore warrant a version number bump. OTOH, I validated GHC with this change and it worked without modification. Cheers, Simon

On 24/09/2009 18:20, Duncan Coutts wrote:

...

However, there are some oddities with the current proposal. e.g.

splitFileName "./foo" == ("./", "foo") "./" "foo" == "./foo"

The current proposal just about hangs together because the tiny bit of normalisation that does exactly undoes the creation of "." in splitFileName, and there's no other way that splitFileName can generate ".". That's a terribly fragile property.

So if we take the first bit of your proposal and not the second we have:

splitFileName "foo" == ("./", "foo") "./" "foo" == "./foo"

and thus we do not have:

uncurry () (splitFileName x) == x

because we end up with "foo" an "./foo"

However I think that's fine. The splitFileName function is asking for the directory part of a relative/incomplete filepath and expecting it to be a real directory. Thus we are interpreting the original filepath as a complete filepath that is relative to ".". So given that by applying splitFileName we are taking that interpretation it's ok to get back "./foo", because we in that context we really were interpreting "foo" as "./foo".

I've amended the patch as suggested above, it turned out to be not too hard. A few places were using splitFileName internally, and that broke some properties, e.g. -- > Valid x => replaceFileName x (takeFileName x) == x replaceFileName :: FilePath -> String -> FilePath replaceFileName x y = dropFileName x y the property doesn't hold any more because dropFileName "foo" == "./". So I worked around cases like this with an internal version of splitFileName with the old semantics. This isn't a big problem - it just means we get to keep some of these nice simple equality properties, which are arguably wrong anyway, and fewer "./" prefixes will show up to surprise users. Neil's comprehensive test suite still passes with the new patch. Ticket, with new patch attached: http://hackage.haskell.org/trac/ghc/ticket/2034 The discussion deadline has long passed, so I propose we have another 2 weeks (18 November). Cheers, Simon

Excerpts from Duncan Coutts's message of Wed Nov 04 12:11:49 +0100 2009:

On Wed, 2009-11-04 at 10:00 +0000, Simon Marlow wrote:

...

I've amended the patch as suggested above, it turned out to be not too hard.

[..]

...

Neil's comprehensive test suite still passes with the new patch.

Ticket, with new patch attached:

http://hackage.haskell.org/trac/ghc/ticket/2034

The discussion deadline has long passed, so I propose we have another 2 weeks (18 November).

I support this new proposal.

So do I. -- Nicolas Pouillard http://nicolaspouillard.fr

I agree with the idea behind this proposal, but am pretty sure it will break quite a few filepath things that I maintain. So +1, but also +1 for a really strong release announcement and version bump. The problem with filepaths is that they lack much structure, and are very system dependent - meaning every change will break something. A lovely abstract type would be wonderful, but no one has yet released such a library on hackage... Thanks, Neil 2009/11/4 Duncan Coutts :

On Wed, 2009-11-04 at 10:00 +0000, Simon Marlow wrote:

...

I've amended the patch as suggested above, it turned out to be not too hard.

[..]

...

Neil's comprehensive test suite still passes with the new patch.

Ticket, with new patch attached:

   http://hackage.haskell.org/trac/ghc/ticket/2034

The discussion deadline has long passed, so I propose we have another 2 weeks (18 November).

I support this new proposal.

To summarise for people who have not been paying full attention:

Currently:        takeDirectory "Main.hs"  =  ""

Proposed:        takeDirectory "Main.hs"  =  "./"

The purpose being that one can always take the directory and get a valid path to use with a file system function, eg createDirectory.

Unchanged behaviour:

       "./" "Main.hs"  =  "./Main.hs"

It was initially proposed that eat "./" on the left hand side but the current proposal leaves the behaviour of unchanged.

Duncan

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

A lovely abstract type would be wonderful, but no one has yet released such a library on hackage...

I'm certainly not going to claim it's 'lovely' ... but there is now on hackage a library which offers an abstract type: http://hackage.haskell.org/package/pathtype It's very early days, hasn't been used in anger, doesn't support a lot of Windows things yet, but the basics do work. I'd be grateful for any comments (and of course patches!) Oh, and.... *System.Path> takeDirectory ("Main.hs"::RelFile) . --Ben On 5 Nov 2009, at 20:36, Neil Mitchell wrote:

I agree with the idea behind this proposal, but am pretty sure it will break quite a few filepath things that I maintain. So +1, but also +1 for a really strong release announcement and version bump.

The problem with filepaths is that they lack much structure, and are very system dependent - meaning every change will break something. A lovely abstract type would be wonderful, but no one has yet released such a library on hackage...

Thanks, Neil

2009/11/4 Duncan Coutts :

...

On Wed, 2009-11-04 at 10:00 +0000, Simon Marlow wrote:

...

I've amended the patch as suggested above, it turned out to be not too hard.

[..]

...

Neil's comprehensive test suite still passes with the new patch.

Ticket, with new patch attached:

http://hackage.haskell.org/trac/ghc/ticket/2034

The discussion deadline has long passed, so I propose we have another 2 weeks (18 November).

I support this new proposal.

To summarise for people who have not been paying full attention:

Currently: takeDirectory "Main.hs" = ""

Proposed: takeDirectory "Main.hs" = "./"

The purpose being that one can always take the directory and get a valid path to use with a file system function, eg createDirectory.

Unchanged behaviour:

"./" "Main.hs" = "./Main.hs"

It was initially proposed that eat "./" on the left hand side but the current proposal leaves the behaviour of unchanged.

Duncan

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

On 14 Nov 2009, at 14:02, Duncan Coutts wrote:

So the type distinctions you are making are:

* whether a path refers to a file or to a directory * whether a path is relative or absolute

Yep, that's the idea.

By relative and absolute it looks like you mean whether it starts with the root "/" vs not. So "foo/bar.txt" and "./foo/bar.txt" are both considered as relative.

Yes.

So in the similar design I was working on, instead of distinguishing relative and absolute, I distinguish incomplete and complete. I initially started with the relative/absolute distinction and moved onto this one. It'd be interesting to see which is most useful in practise.

What I mean by complete is a path referring to an actual file/dir that you could pass to a system call. This means they are anchored to some point the system knows about, such a "." the current directory or "/" or "C:\".

Then an incomplete path is one that is not anchored. Incomplete paths become complete by them to a existing complete one (including the current dir).

So the difference between this complete/incomplete notion and relative/absolute is for paths that are relative to the current directory.

You would say "Main.hs" :: RelFile

and

readFile :: AbsRelClass ar => FilePath ar -> IO String

and so readFile "Main.hs" is ok, and reads the file "./Main.hs" in the current directory.

I would probably say, "Main.hs" :: IncompleteFilePath

(though I think I used somewhat shorter type names than that!)

and

readFile :: CompleteFilePath -> IO String

and thus readFile "Main.hs" is not well typed, instead it would be:

readFile (currentDir "Main.hs")

because that gives us a complete path, and if we want that to be rooted at the processes' current directory, then we do so explicitly.

Ah, ok, I see. I can see the attraction to that.

My intuition with file paths in Cabal, is that this distinction would catch more bugs. Ideally cabal's building code would be independent of the current directory, but the fact that relative paths get automagically completed to being relative to the current directory means that it's very easy to break this property.

Interesting. One possible approach with pathtype as it stands would be a wrapper module which hides "readFile :: AbsRelClass ar => FilePath ar -> IO String" and exposes it only at the restricted type: "readAbsFile :: AbsFile -> IO String; readAbsFile = readFile". This blocks accidental reading of relative files and clients are forced to use something like 'makeAbsoluteFromCwd'. I don't think this is exactly the same as what you're suggesting (it doesn't permit the "completion" process to be separated from the reading), but I think it would be a way to catch some of the same bugs?

There are various other distinctions one could try to make, the question becomes which type distinctions are useful and when does it just become too much.

I think this is a very important point. It's definitely a balancing act. I have ondered whether even capturing Abs/Rel was overkill, but my current feeling (not yet based on much real-world experience) is that it is worth it. Cheers, --Ben

On Wed, Nov 4, 2009 at 8:00 AM, Simon Marlow wrote:

the property doesn't hold any more because dropFileName "foo" == "./". So I worked around cases like this with an internal version of splitFileName with the old semantics.  This isn't a big problem - it just means we get to keep some of these nice simple equality properties, which are arguably wrong anyway, and fewer "./" prefixes will show up to surprise users.

If FilePath was a newtype, could its Eq instance not distinguish between "foo" and "./foo"? Note that we must distinguish "./foo/../" of "./" because this may not be valid ("foo" does not exist) or may be something different ("foo" is a symlink). This Eq instance would just eat all superfluous "./". Cheers, -- Felipe.

Duncan Coutts wrote:

Some people have mentioned before that some systems (though not the native low level POSIX interfaces) do actually distinguish "./blah/blah" from "blah/blah". The distinction is whether a file is relative to the current directory or to some other unspecified root (eg a search path). Unix shells make this distinction for example when running "a.out" vs "./a.out". The latter is independent of the current $PATH, it really refers to the one in the current directory, not any such file on the $PATH.

Just for reference, this comes up with shell settings other than just $PATH as well. For instance, cd in Bash looks through $CDPATH in the same way command selection looks through $PATH (i.e. the default value is "." which is sensible, but some folks like ".:~" to make their home dir everywhere, and you can do crazier things). And various other programs have their own $*PATH variables.

On a related note, when I have experimented with designing a typed version of FilePath, one of the things I wanted to distinguish in the types is exactly this thing about "relative to something specific" (eg relative to "/" or ".") vs "relative to something not yet specified". Another terminology might be "complete" and "incomplete". The OS file manipulation functions would only work on complete filepaths.

I like this distinction. Regardless of the settlement of "" vs "./", it is helpful to distinguish "paths being built" (and therefore potentially invalid) from "real paths" anchored to the file system. -- Live well, ~wren