Wrapping external process communication in a nice API?

Hi. For my chessIO package, I wrote a module to communicate with UCI (Universal Chess Interface) speaking chess engines. This sort of thing is pretty new to me. I settled to use attoparsec for parsing and TChan to feed incoming data to the library users. This is the first thing that I found which looked like it could work... I would be thrilled if someone with more experience in this area would give my approach a review. Let me know if something is glaringly unidiomatic, I am sure you will find things. https://github.com/mlang/chessIO/blob/master/src/Game/Chess/UCI.hs This is for me to learn, and to avoid putting weird stuff on hackage... I actually prefer private mail, but a GitHub issue or even a PR are also highly welcome. Thanks. -- CYa, ⡍⠁⠗⠊⠕

I wrote https://github.com/phadej/cabal-extras/blob/master/cabal-docspec/src/System/... learning from what doctest, hlint and few other tools do. They all (and you) seem to combine the process handling with the specific process, coupling things which don't need to be. That repository is under GPL, but if you find that module useful I can relicense it. - Oleg On 29.5.2021 17.59, Mario Lang wrote:
Hi.
For my chessIO package, I wrote a module to communicate with UCI (Universal Chess Interface) speaking chess engines.
This sort of thing is pretty new to me. I settled to use attoparsec for parsing and TChan to feed incoming data to the library users. This is the first thing that I found which looked like it could work...
I would be thrilled if someone with more experience in this area would give my approach a review. Let me know if something is glaringly unidiomatic, I am sure you will find things.
https://github.com/mlang/chessIO/blob/master/src/Game/Chess/UCI.hs
This is for me to learn, and to avoid putting weird stuff on hackage... I actually prefer private mail, but a GitHub issue or even a PR are also highly welcome.
Thanks.

I suggest https://github.com/haskell/lsp https://github.com/haskell/lsp should be considered the "most idiomatic" way to bridge Haskell domain code with external programs. It speaks JSON-RPC based LSP (https://microsoft.github.io/language-server-protocol https://microsoft.github.io/language-server-protocol) with VSCode and other IDEs.
On 2021-05-29, at 22:59, Mario Lang
wrote: Hi.
For my chessIO package, I wrote a module to communicate with UCI (Universal Chess Interface) speaking chess engines.
This sort of thing is pretty new to me. I settled to use attoparsec for parsing and TChan to feed incoming data to the library users. This is the first thing that I found which looked like it could work...
I would be thrilled if someone with more experience in this area would give my approach a review. Let me know if something is glaringly unidiomatic, I am sure you will find things.
https://github.com/mlang/chessIO/blob/master/src/Game/Chess/UCI.hs
This is for me to learn, and to avoid putting weird stuff on hackage... I actually prefer private mail, but a GitHub issue or even a PR are also highly welcome.
Thanks.
-- CYa, ⡍⠁⠗⠊⠕ _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

LSP is not applicable. Please read about UCI, https://en.wikipedia.org/wiki/Universal_Chess_Interface. I.e. change of protocol is not possible. If you want to draw an analogy, then Mario's chessIO package is the editor here, and various chess engines are "plugins", therefore I don't think that lsp package is much help as it implements different role. - Oleg On 31.5.2021 11.22, YueCompl via Haskell-Cafe wrote:
I suggest https://github.com/haskell/lsp should be considered the "most idiomatic" way to bridge Haskell domain code with external programs. It speaks JSON-RPC based LSP (https://microsoft.github.io/language-server-protocol) with VSCode and other IDEs.
On 2021-05-29, at 22:59, Mario Lang
mailto:mlang@blind.guru> wrote: Hi.
For my chessIO package, I wrote a module to communicate with UCI (Universal Chess Interface) speaking chess engines.
This sort of thing is pretty new to me. I settled to use attoparsec for parsing and TChan to feed incoming data to the library users. This is the first thing that I found which looked like it could work...
I would be thrilled if someone with more experience in this area would give my approach a review. Let me know if something is glaringly unidiomatic, I am sure you will find things.
https://github.com/mlang/chessIO/blob/master/src/Game/Chess/UCI.hs
This is for me to learn, and to avoid putting weird stuff on hackage... I actually prefer private mail, but a GitHub issue or even a PR are also highly welcome.
Thanks.
-- CYa, ⡍⠁⠗⠊⠕ _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Ah yes, their roles are flipped.
On 2021-05-31, at 16:40, Oleg Grenrus
wrote: LSP is not applicable. Please read about UCI, https://en.wikipedia.org/wiki/Universal_Chess_Interface https://en.wikipedia.org/wiki/Universal_Chess_Interface. I.e. change of protocol is not possible.
If you want to draw an analogy, then Mario's chessIO package is the editor here, and various chess engines are "plugins", therefore I don't think that lsp package is much help as it implements different role.
- Oleg
On 31.5.2021 11.22, YueCompl via Haskell-Cafe wrote:
I suggest https://github.com/haskell/lsp https://github.com/haskell/lsp should be considered the "most idiomatic" way to bridge Haskell domain code with external programs. It speaks JSON-RPC based LSP (https://microsoft.github.io/language-server-protocol https://microsoft.github.io/language-server-protocol) with VSCode and other IDEs.
On 2021-05-29, at 22:59, Mario Lang
mailto:mlang@blind.guru> wrote: Hi.
For my chessIO package, I wrote a module to communicate with UCI (Universal Chess Interface) speaking chess engines.
This sort of thing is pretty new to me. I settled to use attoparsec for parsing and TChan to feed incoming data to the library users. This is the first thing that I found which looked like it could work...
I would be thrilled if someone with more experience in this area would give my approach a review. Let me know if something is glaringly unidiomatic, I am sure you will find things.
https://github.com/mlang/chessIO/blob/master/src/Game/Chess/UCI.hs https://github.com/mlang/chessIO/blob/master/src/Game/Chess/UCI.hs
This is for me to learn, and to avoid putting weird stuff on hackage... I actually prefer private mail, but a GitHub issue or even a PR are also highly welcome.
Thanks.
-- CYa, ⡍⠁⠗⠊⠕ _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

On Sat, May 29, 2021 at 16:59 Mario Lang wrote:
Hi.
For my chessIO package, I wrote a module to communicate with UCI (Universal Chess Interface) speaking chess engines.
This sort of thing is pretty new to me. I settled to use attoparsec for parsing and TChan to feed incoming data to the library users. This is the first thing that I found which looked like it could work...
I would be thrilled if someone with more experience in this area would give my approach a review. Let me know if something is glaringly unidiomatic, I am sure you will find things.
https://github.com/mlang/chessIO/blob/master/src/Game/Chess/UCI.hs
It seems like it would be safer to use withCreateProcess instead of createProcess to ensure the process is not left unattended.
This is for me to learn, and to avoid putting weird stuff on hackage... I actually prefer private mail, but a GitHub issue or even a PR are also highly welcome.
I had the same need for a matrix bot where the events come from a ssh process stream. There is `streaming-commons` and `conduit-extra`, but I am not familiar enough with conduit to make it work. In the end I used `turtle` to callback with each line output. For what it worths, here is my current implementation: https://github.com/softwarefactory-project/gerritbot-matrix/blob/71644a21a9a... -Tristan

Tristan Cacqueray
On Sat, May 29, 2021 at 16:59 Mario Lang wrote:
Hi.
For my chessIO package, I wrote a module to communicate with UCI (Universal Chess Interface) speaking chess engines.
This sort of thing is pretty new to me. I settled to use attoparsec for parsing and TChan to feed incoming data to the library users. This is the first thing that I found which looked like it could work...
I would be thrilled if someone with more experience in this area would give my approach a review. Let me know if something is glaringly unidiomatic, I am sure you will find things.
https://github.com/mlang/chessIO/blob/master/src/Game/Chess/UCI.hs
It seems like it would be safer to use withCreateProcess instead of createProcess to ensure the process is not left unattended.
Maybe I am unimaginative, but it seems to me I can not use withCreateProcess as I need to read/write the Handles outside of the initialisation phase. It seems to me withCreateProcess would kill my external process once my initialisation function is done. Or do I misunderstand bracket somehow? AIUI, withCreateProcess always calls cleanupProcess once the action is done, right?
This is for me to learn, and to avoid putting weird stuff on hackage... I actually prefer private mail, but a GitHub issue or even a PR are also highly welcome.
I had the same need for a matrix bot where the events come from a ssh process stream. There is `streaming-commons` and `conduit-extra`, but I am not familiar enough with conduit to make it work.
In the end I used `turtle` to callback with each line output. For what it worths, here is my current implementation:
https://github.com/softwarefactory-project/gerritbot-matrix/blob/71644a21a9a...
Thanks. The output of my subprocess is also line-based, so I use ByteString.hGetLine and a call into attoparsec to make sense of the output I receive. Works so far. -- CYa, ⡍⠁⠗⠊⠕

On Mon, May 31, 2021 at 17:31 Mario Lang wrote:
Tristan Cacqueray
writes: On Sat, May 29, 2021 at 16:59 Mario Lang wrote:
Hi.
For my chessIO package, I wrote a module to communicate with UCI (Universal Chess Interface) speaking chess engines.
This sort of thing is pretty new to me. I settled to use attoparsec for parsing and TChan to feed incoming data to the library users. This is the first thing that I found which looked like it could work...
I would be thrilled if someone with more experience in this area would give my approach a review. Let me know if something is glaringly unidiomatic, I am sure you will find things.
https://github.com/mlang/chessIO/blob/master/src/Game/Chess/UCI.hs
It seems like it would be safer to use withCreateProcess instead of createProcess to ensure the process is not left unattended.
Maybe I am unimaginative, but it seems to me I can not use withCreateProcess as I need to read/write the Handles outside of the initialisation phase. It seems to me withCreateProcess would kill my external process once my initialisation function is done. Or do I misunderstand bracket somehow? AIUI, withCreateProcess always calls cleanupProcess once the action is done, right?
I guess you would have to replace `start'` with something like: ```haskell withEngine :: Time unit -> ... -> (Engine -> IO ()) -> IO () ``` And let your user provide the `Engine -> IO ()` callback. Alternatively, perhaps the `managed` library can be used to get something closer to your existing `start'`, e.g.: http://hackage.haskell.org/package/managed-1.0.8/docs/Control-Monad-Managed....
This is for me to learn, and to avoid putting weird stuff on hackage... I actually prefer private mail, but a GitHub issue or even a PR are also highly welcome.
I had the same need for a matrix bot where the events come from a ssh process stream. There is `streaming-commons` and `conduit-extra`, but I am not familiar enough with conduit to make it work.
In the end I used `turtle` to callback with each line output. For what it worths, here is my current implementation:
https://github.com/softwarefactory-project/gerritbot-matrix/blob/71644a21a9a...
Thanks. The output of my subprocess is also line-based, so I use ByteString.hGetLine and a call into attoparsec to make sense of the output I receive. Works so far.
You're welcome. Perhaps such line-based process wrapper could be a useful addition to the hackage registry? Note that I also tried to convert the Turtle.inproc Shell to a `pipes` or `streaming`, but without success. -Tristan

Tristan Cacqueray
On Mon, May 31, 2021 at 17:31 Mario Lang wrote:
Tristan Cacqueray
writes: It seems like it would be safer to use withCreateProcess instead of createProcess to ensure the process is not left unattended.
Maybe I am unimaginative, but it seems to me I can not use withCreateProcess as I need to read/write the Handles outside of the initialisation phase. It seems to me withCreateProcess would kill my external process once my initialisation function is done. Or do I misunderstand bracket somehow? AIUI, withCreateProcess always calls cleanupProcess once the action is done, right?
I guess you would have to replace `start'` with something like:
```haskell withEngine :: Time unit -> ... -> (Engine -> IO ()) -> IO () ```
And let your user provide the `Engine -> IO ()` callback.
Ahh, of course, push the problem up the caller chain :-) And I guess if that caller needs a sort of handle, it would do it with a channel to communicate with its action? I know I was asking for idiomatic code, and I am thankful for the pointer. Exception safety is clearly a good goal. But, I wonder, is limiting the API in such a way worth the trouble? It looks like this change would trigger a complete rewrite of the current module and all its clients. I'd like to be confident this is the right thing to do before doing that.
Alternatively, perhaps the `managed` library can be used to get something closer to your existing `start'`, e.g.: http://hackage.haskell.org/package/managed-1.0.8/docs/Control-Monad-Managed....
Ahh, thanks again. I'll have to digest all of this. -- CYa, ⡍⠁⠗⠊⠕

On Mon, May 31, 2021 at 19:03 Mario Lang wrote:
Tristan Cacqueray
writes: On Mon, May 31, 2021 at 17:31 Mario Lang wrote:
Tristan Cacqueray
writes: It seems like it would be safer to use withCreateProcess instead of createProcess to ensure the process is not left unattended.
Maybe I am unimaginative, but it seems to me I can not use withCreateProcess as I need to read/write the Handles outside of the initialisation phase. It seems to me withCreateProcess would kill my external process once my initialisation function is done. Or do I misunderstand bracket somehow? AIUI, withCreateProcess always calls cleanupProcess once the action is done, right?
I guess you would have to replace `start'` with something like:
```haskell withEngine :: Time unit -> ... -> (Engine -> IO ()) -> IO () ```
And let your user provide the `Engine -> IO ()` callback.
Ahh, of course, push the problem up the caller chain :-) And I guess if that caller needs a sort of handle, it would do it with a channel to communicate with its action? I know I was asking for idiomatic code, and I am thankful for the pointer. Exception safety is clearly a good goal. But, I wonder, is limiting the API in such a way worth the trouble? It looks like this change would trigger a complete rewrite of the current module and all its clients. I'd like to be confident this is the right thing to do before doing that.
Beware I'm in the same boat and I can't tell if this is the right thing to do :-) I like the with... idiom because it is straighforward and it should not limit the API, users just need to provide a callback instead of binding to get the resource. Though it is more tedious to use from a REPL, where you need to prefix each input with `withEngine ... $ \engine -> ...`
Alternatively, perhaps the `managed` library can be used to get something closer to your existing `start'`, e.g.: http://hackage.haskell.org/package/managed-1.0.8/docs/Control-Monad-Managed....
Ahh, thanks again. I'll have to digest all of this.
Cheers, -Tristan
participants (4)
-
Mario Lang
-
Oleg Grenrus
-
Tristan Cacqueray
-
YueCompl