On Thu, Dec 29, 2011 at 2:45 PM, Antoine Latter wrote:

[...]

When GHC opens files for reading, it asks windows to disallow write access to the file. I'm guessing that Framemaker has the file open for writing, so GHC can't get that permission.

In fact, this is required behavior according to the Haskell Report:

Implementations should enforce as far as possible, at least locally to the Haskell process, multiple-reader single-writer locking on files. That is, there may either be many handles on the same file which manage input, or just one handle on the file which manages output.

I guess on Windows, "as far as possible" means locking it across the whole system. (See http://www.haskell.org/onlinereport/haskell2010/haskellch41.html#x49-3280004... for the gory details) -- Chris

On Thu, Dec 29, 2011 at 3:45 AM, Antoine Latter wrote:

On Wed, Dec 28, 2011 at 3:52 PM, Michael Snoyman wrote:

...

Hi all,

I just received a bug report from a client that, when an input file is open in FrameMaker, my program gives a "permission denied error". This bug is reproducible with a simple Haskell program:

import System.IO

main = do    putStrLn "here1"    h <- openFile "filename.txt" ReadMode    putStrLn "here2"

I tried writing a simple C program using fopen, and it ran just fine. Does anyone have experience with this issue, and know of a workaround?

When GHC opens files for reading, it asks windows to disallow write access to the file. I'm guessing that Framemaker has the file open for writing, so GHC can't get that permission.

I imagine that the Haskell runtime does this to make lazy-io less crazy.

Here's the call GHC makes: https://github.com/ghc/packages-base/blob/0e1a02b96cfd03b8488e3ff4ce232466d6...

To open a file for reading in your C demo in a similar way you could do something like:

fd = _sopen("file_name", _O_RDONLY | _O_NOINHERIT,_SH_DENYWR, 0);

Here "_SH_DENYWR" is telling windows to deny others from writing to this file.

Here's the msdn link for _sopen and _wsopen: http://msdn.microsoft.com/en-us/library/w7sa2b22%28VS.80%29.aspx

I haven't tested any of that, but that should help you in reproducing how GHC opens files for read on windows.

You should be able to use System.Win32.File.createFile with a share mode of (fILE_SHARE_READ .|. fILE_SHARE_WRITE) and then wrangling a Haskell Handle out of that, but I haven't tried it.

Or you could modify the call to _wsopen and FFI call that - it takes fewer parameters and might be less confusing.

Antoine

Thanks for the advice Antoine, it was spot on. I modified my uri-conduit package with the following commit: https://github.com/snoyberg/xml/commit/a4763739093c525d8f509b11e72a2d17894af... Since conduits re-implement buffering themselves, I don't think there's any advantage to wrapping up the FD in a Handle again, except perhaps for better integration with the async calls of the multi-threaded runtime. But since I'm not using that for my Windows code, and I don't think the multi-threaded runtime supports Windows particularly well in the first place, this seems like an acceptable trade-off. Does anyone see any issues with the code? Would it be useful for me to expose this code elsewhere, such as in conduit itself? Michael

On 29/12/2011 04:29 AM, Antoine Latter wrote:

On Wed, Dec 28, 2011 at 3:52 PM, Michael Snoyman wrote:

...

Hi all,

I just received a bug report from a client that, when an input file is open in FrameMaker, my program gives a "permission denied error". This bug is reproducible with a simple Haskell program:

This bug and its discussion is similar, but not identical: http://hackage.haskell.org/trac/ghc/ticket/4363

This one has been rumbling on for ages. As others have said, the Report demands that locking occur, which is probably a mistake. The daft thing is, apparently Linux doesn't really support locking, so on that platform these types of thing all work fine, and only on Windows, which does and always has supported propper locking, people get these errors. And yet, many people seem surprised to hear that Windows can actually turn off the locking; they seem completely unaware of the extensive and highly flexible locking facilities that Windows provides. Every time I hear "oh, I don't think Windows can handle that", I sigh with resignation. Whatever we change the default to, it would be useful if we could expose locking mechanisms in a reasonably portable way so that people who care about such things can alter the default if it is not what they want.

...

Every time I hear "oh, I don't think Windows can handle that", I sigh with resignation.

Sorry to say, but it seems you yourself are unaware of the "extensive and highly flexible" locking facilities on Linux :) The defaults on Linux are advisory locking, not mandatory, but claiming Linux doesn't support locking is plain wrong.

I would suggest that advisory locking isn't particularly useful. I gather that Linux does now support real locking though. (And file update notifications, and ACLs, and lots of other things that Windows has had for far longer.) Either way, I have no interest in starting a Windows vs Linux flamewar. I'm just saying it would be nice if Haskell could support more of what these two OSes have to offer.

...

I gather that Linux does now support real locking though. (And file update notifications, and ACLs, and lots of other things that Windows has had for far longer.)

Hrmm: "Mandatory File Locking For The Linux Operating System, 15 April 1996" :)

Have a reference for when it was actually implemented? I tried to look this up, and got this: http://0pointer.de/blog/projects/locking.html Sounds like it's not so "solved" after all. (Or perhaps the author just doesn't know enough about it, it's hard to say...) The Wikipedia page on file locking also points out a list of Unix-related locking problems. (But again, that might just be editor bias.)

...

I'm just saying it would be nice if Haskell could support more of what these two OSes have to offer.

Totally agreed!

Good, that was my main point. Now, who's volunteering to implement all this? :-}

On Thu, Dec 29, 2011 at 10:53 AM, Donn Cave wrote:

Quoth Andrew Coppin ,

...

On 29/12/2011 04:29 AM, Antoine Latter wrote: ...

...

This bug and its discussion is similar, but not identical: http://hackage.haskell.org/trac/ghc/ticket/4363

This one has been rumbling on for ages. As others have said, the Report demands that locking occur, which is probably a mistake.

The rationale that followed may have been a little sketchy, but apparently everyone agrees with with the point.  I see the ticket led to a discussion on the libraries list - http://www.haskell.org/pipermail/libraries/2011-October/016978.html

... wherein Ian Lynagh proposed to remove this feature and let the programmer enforce locking or not, as in other programming languages' base I/O libraries.  This met with enthusiastic universal support, so whatever the Report may say, it looks to me like the GHC libraries will eventually not do this.

Wouldn't this lead to 'getContents' and friends being much less safe than they already are? Or would we have to do something at the GHC runtime level of things to add locking? Antoine

On Thu, Dec 29, 2011 at 11:49 AM, Donn Cave wrote:

Quoth Antoine Latter , ...

...

...

http://www.haskell.org/pipermail/libraries/2011-October/016978.html

... wherein Ian Lynagh proposed to remove this feature and let the programmer enforce locking or not, as in other programming languages' base I/O libraries.  This met with enthusiastic universal support, so whatever the Report may say, it looks to me like the GHC libraries will eventually not do this.

Wouldn't this lead to 'getContents' and friends being much less safe than they already are? Or would we have to do something at the GHC runtime level of things to add locking?

Interesting question.  I tend to steer clear of that function, rather than try to keep track of all the things that can go wrong with it! but I would guess, the effect is the same on any I/O strategy that executes the same way: e.g. if you were to read pieces of the file iteratively you would be similarly exposed to the possibility of concurrent modifications.

That risk may be less obvious with getContents if you take the naive view that it returns the contents of the file in the same way that normal I/O operations return data, but then you will eventually be punished for that naivety anyway!

Are you aware of some possible special issue with getContents?

Well, the issue would more be with 'hGetContents'. Would this program then loop: append fromFilePath toFilePath = do str <- readFile fromFile writeFile toFile str if 'from' and 'to' where the same file? Currently the locking prevents this. Antoine

On Thu, Dec 29, 2011 at 12:28 PM, Donn Cave wrote:

Quoth Antoine Latter , ...

...

Would this program then loop:

append fromFilePath toFilePath = do   str <- readFile fromFile   writeFile toFile str

if 'from' and 'to' where the same file?

Currently the locking prevents this.

Do you mean, locking makes that work, or just makes it fail in a different way?  Anyway, I think that's an example that would have the same issues without hGetContents - that is, any way you set out to overwrite a file by modifying its contents, you have the same problem.

Locking makes this fail with an exception, I think. Not locking would make this mis-behave. And lets pretend I wrote the function that was in my head: append fromFilePath toFilePath = do str <- readFile fromFile appendFile toFile str Here, without locking, I would expect to fill my disk up, whereas previously my program would have crashed before writing out any data. Maybe these are pathological examples, but they are (I think!) the justification for having the locking in the first place. Antoine