Are there any plans to get rid of hGetContents and the semi-closed handle state for Haskell Prime? (I call hGetContents unsafe because it adds side effects to pattern matching, stricly speaking invalidating most of the transformations which are expected to be valid in a pure language.)
fw:
Are there any plans to get rid of hGetContents and the semi-closed handle state for Haskell Prime?
(I call hGetContents unsafe because it adds side effects to pattern matching, stricly speaking invalidating most of the transformations which are expected to be valid in a pure language.)
Isn't this a broader complaint about lazy IO in general? -- Don
* Don Stewart:
fw:
Are there any plans to get rid of hGetContents and the semi-closed handle state for Haskell Prime?
(I call hGetContents unsafe because it adds side effects to pattern matching, stricly speaking invalidating most of the transformations which are expected to be valid in a pure language.)
Isn't this a broader complaint about lazy IO in general?
Yes, sort of. But doesn't lazy input derive its justification from being present in the prelude?
Excerpts from Florian Weimer's message of Wed Sep 16 22:17:08 +0200 2009:
Are there any plans to get rid of hGetContents and the semi-closed handle state for Haskell Prime?
(I call hGetContents unsafe because it adds side effects to pattern matching, stricly speaking invalidating most of the transformations which are expected to be valid in a pure language.)
Would you consider something like [1] as an acceptable replacement? [1]: http://hackage.haskell.org/package/safe-lazy-io -- Nicolas Pouillard http://nicolaspouillard.fr
On 17/09/2009 13:58, Nicolas Pouillard wrote:
Excerpts from Florian Weimer's message of Wed Sep 16 22:17:08 +0200 2009:
Are there any plans to get rid of hGetContents and the semi-closed handle state for Haskell Prime?
(I call hGetContents unsafe because it adds side effects to pattern matching, stricly speaking invalidating most of the transformations which are expected to be valid in a pure language.)
Would you consider something like [1] as an acceptable replacement?
I rater like this as a workaround for the most common practical problems with lazy I/O, those of resource control. It doesn't address the deeper concern that lazy I/O requires a particular evaluation order and is therefore a bit warty as a language feature - implementing lazy I/O properly in GHC's parallel mutator was somewhat tricky. I'm not of the opinion that we should throw out lazy I/O, but it's still a problematic area in Haskell. Cheers, Simon
Excerpts from Simon Marlow's message of Mon Sep 21 11:52:41 +0200 2009:
On 17/09/2009 13:58, Nicolas Pouillard wrote:
Excerpts from Florian Weimer's message of Wed Sep 16 22:17:08 +0200 2009:
Are there any plans to get rid of hGetContents and the semi-closed handle state for Haskell Prime?
(I call hGetContents unsafe because it adds side effects to pattern matching, stricly speaking invalidating most of the transformations which are expected to be valid in a pure language.)
Would you consider something like [1] as an acceptable replacement?
I rater like this as a workaround for the most common practical problems with lazy I/O, those of resource control.
It doesn't address the deeper concern that lazy I/O requires a particular evaluation order and is therefore a bit warty as a language feature
When using safe-lazy-io we no longer rely (or a lot less) on the evaluation order (assuming you mean the order of side-effects). Since the way of combining the different inputs is statically chosen by user.
- implementing lazy I/O properly in GHC's parallel mutator was somewhat tricky. I'm not of the opinion that we should throw out lazy I/O, but it's still a problematic area in Haskell.
Maybe the 'unsafeGetContents' feature required by a safe-lazy-io would be less problematic, in particular it does not have to ignore exceptions. Best regards, -- Nicolas Pouillard http://nicolaspouillard.fr
* Nicolas Pouillard:
Excerpts from Florian Weimer's message of Wed Sep 16 22:17:08 +0200 2009:
Are there any plans to get rid of hGetContents and the semi-closed handle state for Haskell Prime?
(I call hGetContents unsafe because it adds side effects to pattern matching, stricly speaking invalidating most of the transformations which are expected to be valid in a pure language.)
Would you consider something like [1] as an acceptable replacement?
It only addresses two known issues with lazy I/O, doesn't it? It still injects input operations into pure code not in the IO monad.
On 03/10/2009 19:59, Florian Weimer wrote:
* Nicolas Pouillard:
Excerpts from Florian Weimer's message of Wed Sep 16 22:17:08 +0200 2009:
Are there any plans to get rid of hGetContents and the semi-closed handle state for Haskell Prime?
(I call hGetContents unsafe because it adds side effects to pattern matching, stricly speaking invalidating most of the transformations which are expected to be valid in a pure language.)
Would you consider something like [1] as an acceptable replacement?
It only addresses two known issues with lazy I/O, doesn't it? It still injects input operations into pure code not in the IO monad.
While what you say is true, and I've complained about the same thing myself in the past, it turns out to be quite difficult to demonstrate the unsafety. Try it! Here's the rules. - write a program that gives different results when compiled with different optimisation flags only. (one exception: you're not allowed to take advantage of -fno-state-hack). - Using exceptions is not allowed (they're non-determinstic). - A difference caused by resources (e.g. stack overflow) doesn't count. - The only "unsafe" operation you're allowed to use is hGetContents. - You're allowed to use any other I/O operations, including from libraries, as long as they're not unsafe, and as long as the I/O itself is deterministic. The reason it's hard is that to demonstrate a difference you have to get the lazy I/O to commute with some other I/O, and GHC will never do that. If you find a way to do it, then we'll probably consider it a bug in GHC. You can get lazy I/O to commute with other lazy I/O, and perhaps with some cunning arrangement of pipes (or something) that might be a way to solve the puzzle. Good luck! Cheers, Simon
Excerpts from Simon Marlow's message of Tue Oct 06 14:59:06 +0200 2009:
On 03/10/2009 19:59, Florian Weimer wrote:
* Nicolas Pouillard:
Excerpts from Florian Weimer's message of Wed Sep 16 22:17:08 +0200 2009:
Are there any plans to get rid of hGetContents and the semi-closed handle state for Haskell Prime?
(I call hGetContents unsafe because it adds side effects to pattern matching, stricly speaking invalidating most of the transformations which are expected to be valid in a pure language.)
Would you consider something like [1] as an acceptable replacement?
It only addresses two known issues with lazy I/O, doesn't it? It still injects input operations into pure code not in the IO monad.
While what you say is true, and I've complained about the same thing myself in the past, it turns out to be quite difficult to demonstrate the unsafety.
Try it! Here's the rules.
- write a program that gives different results when compiled with different optimisation flags only. (one exception: you're not allowed to take advantage of -fno-state-hack).
- Using exceptions is not allowed (they're non-determinstic).
- A difference caused by resources (e.g. stack overflow) doesn't count.
- The only "unsafe" operation you're allowed to use is hGetContents.
- You're allowed to use any other I/O operations, including from libraries, as long as they're not unsafe, and as long as the I/O itself is deterministic.
The reason it's hard is that to demonstrate a difference you have to get the lazy I/O to commute with some other I/O, and GHC will never do that. If you find a way to do it, then we'll probably consider it a bug in GHC.
You can get lazy I/O to commute with other lazy I/O, and perhaps with some cunning arrangement of pipes (or something) that might be a way to solve the puzzle. Good luck!
Oleg's example is quite close, don't you think? URL: http://www.haskell.org/pipermail/haskell/2009-March/021064.html Cheers, -- Nicolas Pouillard http://nicolaspouillard.fr
On 06/10/2009 14:18, Nicolas Pouillard wrote:
Excerpts from Simon Marlow's message of Tue Oct 06 14:59:06 +0200 2009:
On 03/10/2009 19:59, Florian Weimer wrote:
* Nicolas Pouillard:
Excerpts from Florian Weimer's message of Wed Sep 16 22:17:08 +0200 2009:
Are there any plans to get rid of hGetContents and the semi-closed handle state for Haskell Prime?
(I call hGetContents unsafe because it adds side effects to pattern matching, stricly speaking invalidating most of the transformations which are expected to be valid in a pure language.)
Would you consider something like [1] as an acceptable replacement?
It only addresses two known issues with lazy I/O, doesn't it? It still injects input operations into pure code not in the IO monad.
While what you say is true, and I've complained about the same thing myself in the past, it turns out to be quite difficult to demonstrate the unsafety.
Try it! Here's the rules.
- write a program that gives different results when compiled with different optimisation flags only. (one exception: you're not allowed to take advantage of -fno-state-hack).
- Using exceptions is not allowed (they're non-determinstic).
- A difference caused by resources (e.g. stack overflow) doesn't count.
- The only "unsafe" operation you're allowed to use is hGetContents.
- You're allowed to use any other I/O operations, including from libraries, as long as they're not unsafe, and as long as the I/O itself is deterministic.
The reason it's hard is that to demonstrate a difference you have to get the lazy I/O to commute with some other I/O, and GHC will never do that. If you find a way to do it, then we'll probably consider it a bug in GHC.
You can get lazy I/O to commute with other lazy I/O, and perhaps with some cunning arrangement of pipes (or something) that might be a way to solve the puzzle. Good luck!
Oleg's example is quite close, don't you think?
URL: http://www.haskell.org/pipermail/haskell/2009-March/021064.html
Ah yes, if you have two lazy input streams both referring to the same underlying stream, that is enough to demonstrate a problem. As for whether Oleg's example is within the rules, it depends whether you consider fdToHandle as "unsafe": Haskell's IO library is carefully designed to not run into this problem on its own. It's normally not possible to get two Handles with the same FD, however GHC.IO.Handle.hDuplicate also lets you do this. Cheers, Simon
* Simon Marlow:
Oleg's example is quite close, don't you think?
URL: http://www.haskell.org/pipermail/haskell/2009-March/021064.html
Ah yes, if you have two lazy input streams both referring to the same underlying stream, that is enough to demonstrate a problem. As for whether Oleg's example is within the rules, it depends whether you consider fdToHandle as "unsafe":
Is relying on seq to show the difference allowed, according to your rules on an insecurity proof? What about handles from System.Process? Do they count as well?
On 11/10/2009 09:26, Florian Weimer wrote:
* Simon Marlow:
Oleg's example is quite close, don't you think?
URL: http://www.haskell.org/pipermail/haskell/2009-March/021064.html
Ah yes, if you have two lazy input streams both referring to the same underlying stream, that is enough to demonstrate a problem. As for whether Oleg's example is within the rules, it depends whether you consider fdToHandle as "unsafe":
Is relying on seq to show the difference allowed, according to your rules on an insecurity proof?
Absolutely.
What about handles from System.Process? Do they count as well?
Sure - we hopefully don't consider System.Process to be unsafe. Cheers, Simon
* Simon Marlow:
What about handles from System.Process? Do they count as well?
Sure - we hopefully don't consider System.Process to be unsafe.
Here's a demonstration that lazy input has an observable effect. It needs the Perl helper script included below. Of course, this example is constructed, but there are similar issues to consider when network IO is involved. For instance, not reading the lazy structure to its end causes the server to keep the connection open longer than necessary. ---------------------------------------------------------------------- -- Based on Oleg Kiselyov's example in: -- http://www.haskell.org/pipermail/haskell/2009-March/021064.html module Main where import System.IO (hGetContents) import System.Process (runInteractiveProcess) f1, f2:: String -> String -> String f1 e1 e2 = e1 `seq` e2 `seq` e1 f2 e1 e2 = e2 `seq` e1 `seq` e1 f = head . tail . lines spawn :: () -> IO String spawn () = do (inp,out,err,pid) <- runInteractiveProcess "perl" ["magic.pl"] Nothing Nothing hGetContents out main = do s1 <- spawn () s2 <- spawn () print $ f1 (f s1) (f s2) -- print $ f2 (f s1) (f s2) ---------------------------------------------------------------------- #!/usr/bin/perl # Magic program to demonstrate that lazy I/O leads to observable # differences in behavior. use strict; use warnings; use Fcntl ':flock'; open my $self, '<', $0 or die "opening $0: $!\n"; # use this file as lock flock($self, LOCK_SH) or die "flock(LOCK_SH): $!\n"; print "x" x 100_000 . "\n"; # blocks if reader blocks print flock($self, LOCK_EX | LOCK_NB) ? "locked\n" : "failed\n"; # only succeeds if the other process has exited
On Tue, 2009-10-06 at 15:18 +0200, Nicolas Pouillard wrote:
The reason it's hard is that to demonstrate a difference you have to get the lazy I/O to commute with some other I/O, and GHC will never do that. If you find a way to do it, then we'll probably consider it a bug in GHC.
You can get lazy I/O to commute with other lazy I/O, and perhaps with some cunning arrangement of pipes (or something) that might be a way to solve the puzzle. Good luck!
Oleg's example is quite close, don't you think?
URL: http://www.haskell.org/pipermail/haskell/2009-March/021064.html
I didn't think that showed very much. He showed two different runs of two different IO programs where he got different results after having bypassed the safety switch on hGetContents. It shows that lazy IO is non-deterministic, but then we knew that. It didn't show anything was impure. As a software engineering thing, it's recommended to use lazy IO in the cases where the non-determinism has a low impact, ie where the order of the actions with respect to other actions doesn't really matter. When it does matter then your programs will probably be more comprehensible if you do the actions more explicitly. For example we have the shoot-yourself-in-the-foot restriction that you can only use hGetContents on a handle a single time (this is the safety mechanism that Oleg turned off) and after that you cannot write to the same handle. That's not because it'd be semantically unsound if those restrictions were not there, but it would let you write some jolly confusing non-deterministic programs. Duncan
On 16/09/2009 21:17, Florian Weimer wrote:
Are there any plans to get rid of hGetContents and the semi-closed handle state for Haskell Prime?
(I call hGetContents unsafe because it adds side effects to pattern matching, stricly speaking invalidating most of the transformations which are expected to be valid in a pure language.)
There is no current proposal for this, no. Feel free to start one; information about the process for Haskell Prime proposals is here http://hackage.haskell.org/trac/haskell-prime/wiki/Process Cheers, Simon
Excerpts from Simon Marlow's message of Mon Sep 21 11:41:38 +0200 2009:
On 16/09/2009 21:17, Florian Weimer wrote:
Are there any plans to get rid of hGetContents and the semi-closed handle state for Haskell Prime?
(I call hGetContents unsafe because it adds side effects to pattern matching, stricly speaking invalidating most of the transformations which are expected to be valid in a pure language.)
There is no current proposal for this, no. Feel free to start one; information about the process for Haskell Prime proposals is here
An alternate proposition (instead of removing it) would to to move it to System.IO.Unsafe. -- Nicolas Pouillard http://nicolaspouillard.fr
participants (5)
-
Don Stewart -
Duncan Coutts -
Florian Weimer -
Nicolas Pouillard -
Simon Marlow