It seems I need to read Oleg's paper. I might have over simplified the problem by using ReaderT in my example. In my original problem this role is played by the Pipes library (and instead of using 'ask', I wanted to 'yield' control to a downstream pipe). Thanks for the feedback! :-) Dimitri On 10/26/15 7:55 AM, Oleg wrote:

...

Not really. You are passing 'ref' into myReadFile by hand here. If you are willing to pass parameters by hand, there is no need in IORef at all; you could just as well pass header directly. I confess in embellishing the problem and tacitly upgrading Reader to

Roman Cheplyaka wrote: the State. The original problem seemed too simple. In penance, I show the solution to the original problem, using exactly the signature of the original poster, and using exactly his code for myReadFile, (but with the lifted type, which is what he wanted, it seems).

...

data ColumnHeaders = FirstLine | None getFileContents :: ReaderT ColumnHeaders IO String getFileContents = do header <- ask -- can ask it here lift $ withCSV "data.csv" (\handle -> runReaderT (myReadFile handle) header) where

...

myReadFile :: Handle -> ReaderT ColumnHeaders IO String myReadFile handle = do header <- ask -- Now, we can ask it, alright case header of None -> return "" FirstLine -> lift $ hGetLine handle -- skip first line text <- lift $ hGetContents handle return text The idea is simple: since withCSV wants its callback to be IO rather than ReaderT IO, it means the withCSV is not capable of altering the environment of the callback. Therefore, the myReadFile is executed in the same environment, regardless of the Handle. Once we understand this, the solution is trivial.

So, to answer the question of the original poster: it seems we can do what he wants withCSV he got. There is no need to ask withCSV author for anything.

...

I agree that *given a ReaderT* (or implicit params, or your implicit configurations), you can emulate StateT/WriterT using IORefs (but only one way of stacking w.r.t. exceptions). Actually, it is easy to get other ways of `stacking with respect to exceptions.' Once we get persistent state (which is what IORef provide), we can always discard that state by writing an appropriate exception handler. I guess I can make another stab at monad transformers and say that thinking in terms of stacks is not always productive.

Here's a final pipes example then. I don't think there's a way to fix the problem as Oleg proposed because pipes are monad transformers by design. The Pipe monad transformer augments the base monad with two operations: - await: gets a result from an upstream pipe - yield: sends a result to a downstream pipe I have a producer (which is a pipe that can only 'yield') that produces the lines of the .CSV file as Strings and returns () when done: getFileContentsLifted :: Producer String IO () getFileContentsLifted = withCSVLifted "data.csv" myReadFile where myReadFile :: Handle -> Producer String IO () myReadFile handle = do eof <- lift $ hIsEOF handle unless eof $ do str <- lift $ hGetLine handle yield str myReadFile handle I then have a simple pipeline that reads each line and prints it twice: lineDoubler :: Pipe String String IO () lineDoubler = forever $ do s <- await yield s yield s main = do runEffect $ getFileContentsLifted >-> lineDoubler >-> stdoutLn The problem as before is that this code does not work with the original version of withCSV: withCSV :: FilePath -> (Handle -> IO r) -> IO r withCSV path action = do putStrLn "opening file" h <- openFile path ReadMode r <- action h hClose h putStrLn "file closed" return r only with the lifted (i.e. generalized) one. withCSVLifted :: MonadIO mIO => FilePath -> (Handle -> mIO r) -> mIO r withCSVLifted path action = do liftIO $ putStrLn "opening file" h <- liftIO $ openFile path ReadMode r <- action h liftIO $ hClose h liftIO $ putStrLn "file closed" return r And I have the same question: Should I always "generalize" my monadic actions that take callbacks as parameters? I hope this version is still clear. Thanks for everyone for their input. I thought this was an easier problem than it now appears to be. Dimitri PS. Full code is here https://gist.github.com/dimitri-xyz/f1f5bd4c0f7f2bf85379 On 10/26/15 10:47 AM, Kim-Ee Yeoh wrote:

On Mon, Oct 26, 2015 at 11:36 PM, Dimitri DeFigueiredo mailto:defigueiredo@ucdavis.edu> wrote:

I might have over simplified the problem by using ReaderT in my example. In my original problem this role is played by the Pipes library (and instead of using 'ask', I wanted to 'yield' control to a downstream pipe).

Is there a way you could introduce just enough complexity to allow Oleg another stab?

Also, there's always the fallback of showing your Pipes-based code although that library doesn't enjoy universal familiarity.

-- Kim-Ee

Hi Chris, You are right. The implementation is totally dodgy! In fact, Pipes already has fromHandle which does this properly. I'm just trying to come up with an example of an IO action that takes another IO action as a parameter and what to do about using that with a monad transformer such as pipes. My focus is what to do when you need to use an action such as withCSV with a action that is *not* in the IO monad. Dimitri On 10/27/15 5:52 PM, Chris Wong wrote:

Hi Dimitri,

The implementation of `withCSVLifted` looks dodgy to me. If a downstream consumer terminates early, then the file will never get closed.

For pipes, the standard solution to resource management is the pipes-safe[1] package. It handles early termination and IO exceptions automatically. The example in the docs should fit your use case pretty well.

[1] https://hackage.haskell.org/package/pipes-safe-2.2.3/docs/Pipes-Safe.html

On Wed, Oct 28, 2015 at 12:25 PM, Dimitri DeFigueiredo wrote:

...

Here's a final pipes example then. I don't think there's a way to fix the problem as Oleg proposed because pipes are monad transformers by design.

The Pipe monad transformer augments the base monad with two operations: - await: gets a result from an upstream pipe - yield: sends a result to a downstream pipe

I have a producer (which is a pipe that can only 'yield') that produces the lines of the .CSV file as Strings and returns () when done:

getFileContentsLifted :: Producer String IO () getFileContentsLifted = withCSVLifted "data.csv" myReadFile where myReadFile :: Handle -> Producer String IO () myReadFile handle = do eof <- lift $ hIsEOF handle unless eof $ do str <- lift $ hGetLine handle yield str myReadFile handle

I then have a simple pipeline that reads each line and prints it twice:

lineDoubler :: Pipe String String IO () lineDoubler = forever $ do s <- await yield s yield s

main = do runEffect $ getFileContentsLifted >-> lineDoubler >-> stdoutLn

The problem as before is that this code does not work with the original version of withCSV:

withCSV :: FilePath -> (Handle -> IO r) -> IO r withCSV path action = do putStrLn "opening file" h <- openFile path ReadMode r <- action h hClose h putStrLn "file closed" return r

only with the lifted (i.e. generalized) one.

withCSVLifted :: MonadIO mIO => FilePath -> (Handle -> mIO r) -> mIO r withCSVLifted path action = do liftIO $ putStrLn "opening file" h <- liftIO $ openFile path ReadMode r <- action h liftIO $ hClose h liftIO $ putStrLn "file closed" return r

And I have the same question: Should I always "generalize" my monadic actions that take callbacks as parameters?

I hope this version is still clear. Thanks for everyone for their input. I thought this was an easier problem than it now appears to be.

Dimitri

PS. Full code is here https://gist.github.com/dimitri-xyz/f1f5bd4c0f7f2bf85379

On 10/26/15 10:47 AM, Kim-Ee Yeoh wrote:

On Mon, Oct 26, 2015 at 11:36 PM, Dimitri DeFigueiredo wrote:

...

I might have over simplified the problem by using ReaderT in my example. In my original problem this role is played by the Pipes library (and instead of using 'ask', I wanted to 'yield' control to a downstream pipe).

Is there a way you could introduce just enough complexity to allow Oleg another stab?

Also, there's always the fallback of showing your Pipes-based code although that library doesn't enjoy universal familiarity.

-- Kim-Ee

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe

Just to finish beating this horse: let me explain why the function with the following signature is actually a bad idea.

withCSVLifted :: MonadIO mIO => FilePath -> (Handle -> mIO r) -> mIO r withCSVLifted path action = do liftIO $putStrLn "opening file" h <- liftIO $ openFile path ReadMode r <- action h liftIO $ hClose h liftIO $ putStrLn "file closed" return r

It accepts the callback that can do any MonadIO action, really any, including an action that throws an exception. If the callback 'action' throws an exception, who would close the CSV file? Further, the mIO action could do a non-deterministic choice. One may think of a such a choice as `forking' a lightweight `thread': m1 `mplus` m2 could be understood as forking a thread to execute m2, whereas the parent will execute m1. If the parent dies (if the m1 action or its consequences turned out unsuccessful), the child is awoken. The parent thread will eventually return through withCSVLifted and the CSV file will be closed. Suppose, the parent dies shortly after that. The child wakes up and tries to read from the Handle, which by that time will already be closed. (UNIX fork(2) does not have such problem because the OS duplicates not only the address space of the process but also all open file descriptors.) There is another problem with the withCSVLifted signature: the type of the return value is unrestricted. Therefore, r may be instantiated to Handle (or, more subtly, to the type of a closure containing a Handle), hence leaking the Handle out of the scope of withCSVLifted. The goal of withCSVLifted is to encapsulate a resource (file handle). Such encapsulation is far more difficult than it appears. I will recommend the old paper http://okmij.org/ftp/Haskell/regions.html#light-weight that illustrates these problems with simple approaches. It was a pleasant surprise that extensible effects (Haskell 2015 paper) turn out to implement monadic regions simpler than before. Please see Sec 7 of that paper (Freer monads, more extensible effects). The section also talks about the restrictions we have to impose on effects that are allowed to cross the region's boundary, so to speak.

pipes are monad transformers by design. That is regrettable.

BTW, the Haskell 2015 paper describes the extensible-effect implementation of Reader and Writer effects. Those effects are more general than their name implies: Reader is actually an iteratee and Writer can write to a file. As to Monad Control, that was recently mentioned: there is a problem with them that is explained in Sec 6 of the Haskell 2015 paper.