Intercept stdin in Haskell

Hello all, I need to do something strange and terrible in Haskell: intercept `stdin`. In other words, I need to detect (in another thread, probably?) when my running program is trying to read from `stdin`, and then feed it some data. I know I can use `hDupTo` and other similar things to replace the stdin handle with my own handle, and I know I could probably use `createPipe` or similar from the `unix` package in order to write things to these handles, but I have no idea how I might go about detecting that a handle is being read from. Any ideas? I've racked my brain and cannot come up with a way to do this. I am using the GHC API elsewhere if that leads to any sort of crazy hackery that might save the day. -- Andrew

What's the motivation / use case? You could alternatively do the redirection from launching the program from shell if you can't midify the Haskell code yourself. On Sunday, January 5, 2014, Andrew Gibiansky wrote:
Hello all,
I need to do something strange and terrible in Haskell: intercept `stdin`. In other words, I need to detect (in another thread, probably?) when my running program is trying to read from `stdin`, and then feed it some data.
I know I can use `hDupTo` and other similar things to replace the stdin handle with my own handle, and I know I could probably use `createPipe` or similar from the `unix` package in order to write things to these handles, but I have no idea how I might go about detecting that a handle is being read from.
Any ideas? I've racked my brain and cannot come up with a way to do this. I am using the GHC API elsewhere if that leads to any sort of crazy hackery that might save the day.
-- Andrew

I cannot modify the Haskell code. I am working on IHaskell - it is effectively a Haskell interpreter. It reads some code from the user, which may contain something like `getLine`. However, the frontend to the interpreter is not a shell but is a GUI in the web browser (IPython!). In order to do input, the interpreter sends a message to the frontend via the network saying "give me input", the frontend reads some input, and then sends things back. In order to do this communication, I need to know when the getLine is called so that I know I need to send the message to the frontend. Ideas? The thing is, getLine needs to not actually read from any shell - it will read from a pipe I create, and I simply need to know when to put stuff into that pipe. -- Andrew

Hrm. Have you considered using the ghc API? Ghci itself is written using it, and I imagine you could adapt that code for your purposes perhaps? On Sunday, January 5, 2014, Andrew Gibiansky wrote:
I cannot modify the Haskell code.
I am working on IHaskell - it is effectively a Haskell interpreter. It reads some code from the user, which may contain something like `getLine`. However, the frontend to the interpreter is not a shell but is a GUI in the web browser (IPython!). In order to do input, the interpreter sends a message to the frontend via the network saying "give me input", the frontend reads some input, and then sends things back.
In order to do this communication, I need to know when the getLine is called so that I know I need to send the message to the frontend.
Ideas? The thing is, getLine needs to not actually read from any shell - it will read from a pipe I create, and I simply need to know when to put stuff into that pipe.
-- Andrew

I am using the GHC API extensively. The entire application is built on the
GHC API, with the occasional bit of code practically stolen from GHCi :)
However, I don't know of a way to intercept stdin even if I am using the
GHC API. Given a block of code, there is no sane static analysis than can
be done to detect whether it reads from stdin, as far as I know.
Is there any way to check whether a process is blocked when reading from a
Handle or an Fd? Maybe there are some low-level hacks with unsafeCoerce or
something that would let me pass a non-handle as a Handle, and do it that
way?
The only other option is just disabling all uses of stdin, or perhaps
reimplementing myself the Prelude functions that use stdin. (But this would
still break libraries that have already compiled in the Prelude versions.)
On Sun, Jan 5, 2014 at 2:52 PM, Carter Schonwald wrote: Hrm. Have you considered using the ghc API? Ghci itself is written using
it, and I imagine you could adapt that code for your purposes perhaps? On Sunday, January 5, 2014, Andrew Gibiansky wrote: I cannot modify the Haskell code. I am working on IHaskell - it is effectively a Haskell interpreter. It
reads some code from the user, which may contain something like `getLine`.
However, the frontend to the interpreter is not a shell but is a GUI in the
web browser (IPython!). In order to do input, the interpreter sends a
message to the frontend via the network saying "give me input", the
frontend reads some input, and then sends things back. In order to do this communication, I need to know when the getLine is
called so that I know I need to send the message to the frontend. Ideas? The thing is, getLine needs to not actually read from any shell -
it will read from a pipe I create, and I simply need to know when to put
stuff into that pipe. -- Andrew

On Sun, Jan 5, 2014 at 12:03 PM, Andrew Gibiansky < andrew.gibiansky@gmail.com> wrote:
I am using the GHC API extensively. The entire application is built on the GHC API, with the occasional bit of code practically stolen from GHCi :)
However, I don't know of a way to intercept stdin even if I am using the GHC API. Given a block of code, there is no sane static analysis than can be done to detect whether it reads from stdin, as far as I know.
Could you rebind stdin to the ipython input channel every time a cell from the ipython interface is evaluated? (and then reset it, if necessary, after the cell finishes and in related exception handlers, of course). --Rogan
Is there any way to check whether a process is blocked when reading from a Handle or an Fd? Maybe there are some low-level hacks with unsafeCoerce or something that would let me pass a non-handle as a Handle, and do it that way?
The only other option is just disabling all uses of stdin, or perhaps reimplementing myself the Prelude functions that use stdin. (But this would still break libraries that have already compiled in the Prelude versions.)
On Sun, Jan 5, 2014 at 2:52 PM, Carter Schonwald < carter.schonwald@gmail.com> wrote:
Hrm. Have you considered using the ghc API? Ghci itself is written using it, and I imagine you could adapt that code for your purposes perhaps?
On Sunday, January 5, 2014, Andrew Gibiansky wrote:
I cannot modify the Haskell code.
I am working on IHaskell - it is effectively a Haskell interpreter. It reads some code from the user, which may contain something like `getLine`. However, the frontend to the interpreter is not a shell but is a GUI in the web browser (IPython!). In order to do input, the interpreter sends a message to the frontend via the network saying "give me input", the frontend reads some input, and then sends things back.
In order to do this communication, I need to know when the getLine is called so that I know I need to send the message to the frontend.
Ideas? The thing is, getLine needs to not actually read from any shell - it will read from a pipe I create, and I simply need to know when to put stuff into that pipe.
-- Andrew
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

That's a good idea, but there's no IPython input channel. Instead, the backend (my interpreter) has to *request* input from the frontend, which then supplies this input. The input is entered via a web browser, so there's not even a handle or file descriptor to speak of.

On Sun, Jan 5, 2014 at 12:19 PM, Andrew Gibiansky < andrew.gibiansky@gmail.com> wrote:
That's a good idea, but there's no IPython input channel. Instead, the backend (my interpreter) has to *request* input from the frontend, which then supplies this input. The input is entered via a web browser, so there's not even a handle or file descriptor to speak of.
Ugh, so you really do need to trigger an event when the code in the cell requests input. How does python handle this? Here's a horrible idea, but in the spirit of brainstorming... you could run the cell code in a separate app (fork off your interpreter), using ptrace (for linux; I'm not sure how to do this on other OSes) to detect system calls that involve stdin, and then supply input based on that. Aside from being tricky to do for all platforms, that's also going to have a horrible user experience for any code that loops over std in rapidly to get what would otherwise appear as a single input to the user, but I'm grasping at straws... --Rogan

Yeah, I am pretty sure I have no choice but trigger an event when the code
requests input. I think Python is just much more flexible - you can
probably just substitute something that isn't a file for sys.stdin, and
when you call getline() on it it just does what it needs to. Flexible
dynamic languages and all that.
That's an interesting way to go about it. Will look into it, maybe. So far
I've been browsing the source of GHC.IO.Handle in the hopes of finding a
way to do this - maybe use the underlying IORefs in the Handle_ constructor
for the Handle data type in order to unsafeCoerce something... I don't even
know. I'm beginning to think this is just more or less impossible, though.
On Sun, Jan 5, 2014 at 3:42 PM, Rogan Creswick
On Sun, Jan 5, 2014 at 12:19 PM, Andrew Gibiansky < andrew.gibiansky@gmail.com> wrote:
That's a good idea, but there's no IPython input channel. Instead, the backend (my interpreter) has to *request* input from the frontend, which then supplies this input. The input is entered via a web browser, so there's not even a handle or file descriptor to speak of.
Ugh, so you really do need to trigger an event when the code in the cell requests input. How does python handle this?
Here's a horrible idea, but in the spirit of brainstorming... you could run the cell code in a separate app (fork off your interpreter), using ptrace (for linux; I'm not sure how to do this on other OSes) to detect system calls that involve stdin, and then supply input based on that.
Aside from being tricky to do for all platforms, that's also going to have a horrible user experience for any code that loops over std in rapidly to get what would otherwise appear as a single input to the user, but I'm grasping at straws...
--Rogan

I bet a quarter you can't do it. You'd need access to the process state - whether it's blocking for I/O and whether one of the units in the input set is 0 ("stdin".) Even if you could get that? you'd have to poll for it, which would be hideous. That's the UNIX I/O model. I've always found it a little annoying, because I could do this with the VMS `mailbox' device, analogous to UNIX pipes - in various ways a more sophisticated interprocess communication system than UNIX's. Donn

I think we found a way! (With a *ton* of help from @aavogt - might actually
be more correct to say he found the way :) )
You can use `hDupTo` to change what a Handle points to. You can use
`mkFileHandle` in GHC.IO.Internal to create a new file handle. You can
implement your own IODevice and BufferedIO datatype to give to
`mkFileHandle` instead of using `Fd`. Then, when your "device" is being
read from, you just implement `newBuffer` and `readBuffer` to do whatever
you need them to.
Results pending.
-- Andrew
On Sun, Jan 5, 2014 at 4:14 PM, Donn Cave
I bet a quarter you can't do it. You'd need access to the process state - whether it's blocking for I/O and whether one of the units in the input set is 0 ("stdin".) Even if you could get that? you'd have to poll for it, which would be hideous.
That's the UNIX I/O model. I've always found it a little annoying, because I could do this with the VMS `mailbox' device, analogous to UNIX pipes - in various ways a more sophisticated interprocess communication system than UNIX's.
Donn _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Looks like the excitement was a bit premature. The types work, and in
Haskell that often means the program works... but looks like hDupTo relies
on the `dup2` of the IODevice class, and attempts to cast one IODevice to
another IODevice. Since I'm trying to replace stdin (with IODevice type Fd)
with my own IODevice, the cast fails and raises an exception. Practically
ClassCastException.... yeesh.
On Sun, Jan 5, 2014 at 5:19 PM, Andrew Gibiansky wrote: I think we found a way! (With a *ton* of help from @aavogt - might
actually be more correct to say he found the way :) ) You can use `hDupTo` to change what a Handle points to. You can use
`mkFileHandle` in GHC.IO.Internal to create a new file handle. You can
implement your own IODevice and BufferedIO datatype to give to
`mkFileHandle` instead of using `Fd`. Then, when your "device" is being
read from, you just implement `newBuffer` and `readBuffer` to do whatever
you need them to. Results pending. -- Andrew On Sun, Jan 5, 2014 at 4:14 PM, Donn Cave I bet a quarter you can't do it. You'd need access to the process state -
whether it's blocking for I/O and whether one of the units in the input
set
is 0 ("stdin".) Even if you could get that? you'd have to poll for it,
which
would be hideous. That's the UNIX I/O model. I've always found it a little annoying,
because
I could do this with the VMS `mailbox' device, analogous to UNIX pipes -
in various ways a more sophisticated interprocess communication system
than
UNIX's. Donn
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Here's a stupid idea:
A Handle contains an MVar Handle__, and when a thread calls hGetLine stdin,
it will take that MVar, attempt to read from the buffered device, and then
block until there's data available to be read from the device. You could
check if the MVar is empty, and if so, assume that something is trying to
read from stdin and write your input into the device.
Horrible, unsound hack, I'm sure, but it's all I've got...
On Sun, Jan 5, 2014 at 3:14 PM, Andrew Gibiansky wrote: Looks like the excitement was a bit premature. The types work, and in
Haskell that often means the program works... but looks like hDupTo relies
on the `dup2` of the IODevice class, and attempts to cast one IODevice to
another IODevice. Since I'm trying to replace stdin (with IODevice type Fd)
with my own IODevice, the cast fails and raises an exception. Practically
ClassCastException.... yeesh. On Sun, Jan 5, 2014 at 5:19 PM, Andrew Gibiansky <
andrew.gibiansky@gmail.com> wrote: I think we found a way! (With a *ton* of help from @aavogt - might
actually be more correct to say he found the way :) ) You can use `hDupTo` to change what a Handle points to. You can use
`mkFileHandle` in GHC.IO.Internal to create a new file handle. You can
implement your own IODevice and BufferedIO datatype to give to
`mkFileHandle` instead of using `Fd`. Then, when your "device" is being
read from, you just implement `newBuffer` and `readBuffer` to do whatever
you need them to. Results pending. -- Andrew On Sun, Jan 5, 2014 at 4:14 PM, Donn Cave I bet a quarter you can't do it. You'd need access to the process state
-
whether it's blocking for I/O and whether one of the units in the input
set
is 0 ("stdin".) Even if you could get that? you'd have to poll for it,
which
would be hideous. That's the UNIX I/O model. I've always found it a little annoying,
because
I could do this with the VMS `mailbox' device, analogous to UNIX pipes -
in various ways a more sophisticated interprocess communication system
than
UNIX's. Donn
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe _______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

You are a saviour!
We'd actually already tried going down this path with takeMVars and
putMVars, but somehow it didn't quite work. Let's see if this works in
practice in IHaskell!
Working program below:
import Control.Concurrent
import Control.Monad
import GHC.IO.Handle
import GHC.IO.Handle.Types
import System.IO
import System.Posix.IO
main = do
-- Create a pipe using System.Posix and turn it into handles.
(readEnd, writeEnd) <- createPipe
newStdin <- fdToHandle readEnd
stdinInput <- fdToHandle writeEnd
-- Store old stdin and swap in new stdin.
oldStdin <- hDuplicate stdin
hDuplicateTo newStdin stdin
-- In a separate thread, wait for the read.
forkIO $ forever $ do
let FileHandle _ mvar = stdin
threadDelay $ 200 * 1000
empty <- isEmptyMVar mvar
when empty $ do
putStrLn "Empty!"
hPutStrLn stdinInput "foo"
hFlush stdinInput
putStrLn "Waiting."
threadDelay $ 3 * 1000 * 1000
putStrLn "Reading."
getChar >>= print
On Sun, Jan 5, 2014 at 7:11 PM, John Lato
Here's a stupid idea:
A Handle contains an MVar Handle__, and when a thread calls hGetLine stdin, it will take that MVar, attempt to read from the buffered device, and then block until there's data available to be read from the device. You could check if the MVar is empty, and if so, assume that something is trying to read from stdin and write your input into the device.
Horrible, unsound hack, I'm sure, but it's all I've got...
On Sun, Jan 5, 2014 at 3:14 PM, Andrew Gibiansky < andrew.gibiansky@gmail.com> wrote:
Looks like the excitement was a bit premature. The types work, and in Haskell that often means the program works... but looks like hDupTo relies on the `dup2` of the IODevice class, and attempts to cast one IODevice to another IODevice. Since I'm trying to replace stdin (with IODevice type Fd) with my own IODevice, the cast fails and raises an exception. Practically ClassCastException.... yeesh.
On Sun, Jan 5, 2014 at 5:19 PM, Andrew Gibiansky < andrew.gibiansky@gmail.com> wrote:
I think we found a way! (With a *ton* of help from @aavogt - might actually be more correct to say he found the way :) )
You can use `hDupTo` to change what a Handle points to. You can use `mkFileHandle` in GHC.IO.Internal to create a new file handle. You can implement your own IODevice and BufferedIO datatype to give to `mkFileHandle` instead of using `Fd`. Then, when your "device" is being read from, you just implement `newBuffer` and `readBuffer` to do whatever you need them to.
Results pending.
-- Andrew
On Sun, Jan 5, 2014 at 4:14 PM, Donn Cave
wrote: I bet a quarter you can't do it. You'd need access to the process state - whether it's blocking for I/O and whether one of the units in the input set is 0 ("stdin".) Even if you could get that? you'd have to poll for it, which would be hideous.
That's the UNIX I/O model. I've always found it a little annoying, because I could do this with the VMS `mailbox' device, analogous to UNIX pipes - in various ways a more sophisticated interprocess communication system than UNIX's.
Donn _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Hello,
That last version was still off. I think we are going with John's
idea, which is slightly more concretely in: http://lpaste.net/98017
I expect there are still issues concerning how much input to request,
so that getLine, getChar and getContents all behave as they do in the
console.
--
Adam
On Sun, Jan 5, 2014 at 7:45 PM, Andrew Gibiansky
You are a saviour!
We'd actually already tried going down this path with takeMVars and putMVars, but somehow it didn't quite work. Let's see if this works in practice in IHaskell!
Working program below:
import Control.Concurrent import Control.Monad import GHC.IO.Handle import GHC.IO.Handle.Types import System.IO import System.Posix.IO
main = do -- Create a pipe using System.Posix and turn it into handles. (readEnd, writeEnd) <- createPipe newStdin <- fdToHandle readEnd stdinInput <- fdToHandle writeEnd
-- Store old stdin and swap in new stdin. oldStdin <- hDuplicate stdin hDuplicateTo newStdin stdin
-- In a separate thread, wait for the read. forkIO $ forever $ do let FileHandle _ mvar = stdin threadDelay $ 200 * 1000 empty <- isEmptyMVar mvar when empty $ do putStrLn "Empty!" hPutStrLn stdinInput "foo" hFlush stdinInput
putStrLn "Waiting." threadDelay $ 3 * 1000 * 1000 putStrLn "Reading." getChar >>= print
On Sun, Jan 5, 2014 at 7:11 PM, John Lato
wrote: Here's a stupid idea:
A Handle contains an MVar Handle__, and when a thread calls hGetLine stdin, it will take that MVar, attempt to read from the buffered device, and then block until there's data available to be read from the device. You could check if the MVar is empty, and if so, assume that something is trying to read from stdin and write your input into the device.
Horrible, unsound hack, I'm sure, but it's all I've got...
On Sun, Jan 5, 2014 at 3:14 PM, Andrew Gibiansky
wrote: Looks like the excitement was a bit premature. The types work, and in Haskell that often means the program works... but looks like hDupTo relies on the `dup2` of the IODevice class, and attempts to cast one IODevice to another IODevice. Since I'm trying to replace stdin (with IODevice type Fd) with my own IODevice, the cast fails and raises an exception. Practically ClassCastException.... yeesh.
On Sun, Jan 5, 2014 at 5:19 PM, Andrew Gibiansky
wrote: I think we found a way! (With a *ton* of help from @aavogt - might actually be more correct to say he found the way :) )
You can use `hDupTo` to change what a Handle points to. You can use `mkFileHandle` in GHC.IO.Internal to create a new file handle. You can implement your own IODevice and BufferedIO datatype to give to `mkFileHandle` instead of using `Fd`. Then, when your "device" is being read from, you just implement `newBuffer` and `readBuffer` to do whatever you need them to.
Results pending.
-- Andrew
On Sun, Jan 5, 2014 at 4:14 PM, Donn Cave
wrote: I bet a quarter you can't do it. You'd need access to the process state - whether it's blocking for I/O and whether one of the units in the input set is 0 ("stdin".) Even if you could get that? you'd have to poll for it, which would be hideous.
That's the UNIX I/O model. I've always found it a little annoying, because I could do this with the VMS `mailbox' device, analogous to UNIX pipes - in various ways a more sophisticated interprocess communication system than UNIX's.
Donn _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
participants (6)
-
adam vogt
-
Andrew Gibiansky
-
Carter Schonwald
-
Donn Cave
-
John Lato
-
Rogan Creswick