How to design an network client with user program.

Hello. I have next problem: I have a network client that connects to server, listens for messages and generate responces. So the control flow can be represended as: server -- input -> {generate output} -> output Output can be generated using default implementation or can overriden by user. The main difficulty appeares when I need to add a user program on the top of this logic, i.e. from user-side I want to have dsl:smth like withClient $ do x <- send message waitFor x timeout 5000000 forever $ sendRandomMessage i.e. an ability to send messages, waiting for some event (message to come), waiting for timeout. The question is how to define such logic without a big overhead. I see a solution using conduit, it's possible to create 3 processes: listener, user, sender. +----> user ----+ | | -input -> listener +----------------->+---- sender -> and use TQueue or TChan to send messages between them, however there can be another possible solutions, that uses less resources, or another design. -- Alexander

It doesn't seem like you're trying to perform multiple actions simultaneously. For example, you don't need to be able to read from the server and send data back at the same time. Instead, you'll have a single thread of execution. Am I right? If so, it seems like the simplest thing would be for you to allow users to write something like: Conduit MsgFromServer m MsgToServer Assuming you had conduits to convert an incoming byte stream to a stream of MsgFromServer and the equivalent for sending, you'd end up with something like: appSource appData $$ toMsgFromServer =$ clientSuppliedConduit =$ fromMsgToServer =$ appSink appData Michael On Tue, Apr 9, 2013 at 1:09 PM, Alexander V Vershilov < alexander.vershilov@gmail.com> wrote:
Hello.
I have next problem: I have a network client that connects to server, listens for messages and generate responces. So the control flow can be represended as:
server -- input -> {generate output} -> output
Output can be generated using default implementation or can overriden by user.
The main difficulty appeares when I need to add a user program on the top of this logic, i.e. from user-side I want to have dsl:smth like
withClient $ do x <- send message waitFor x timeout 5000000 forever $ sendRandomMessage
i.e. an ability to send messages, waiting for some event (message to come), waiting for timeout.
The question is how to define such logic without a big overhead. I see a solution using conduit, it's possible to create 3 processes: listener, user, sender.
+----> user ----+ | | -input -> listener +----------------->+---- sender ->
and use TQueue or TChan to send messages between them, however there can be another possible solutions, that uses less resources, or another design.
-- Alexander
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On 10 April 2013 14:56, Michael Snoyman
It doesn't seem like you're trying to perform multiple actions simultaneously. For example, you don't need to be able to read from the server and send data back at the same time. Instead, you'll have a single thread of execution. Am I right?
Not exaclty, user code is not only SeverMessage driven but can generate messages and works on it's own (time-events, or some external events). For example user code may generate random messages even there is no message from server, (i.e. wait for some timeout and then feed sender with message), or do some long running events, (e.g. wait for 5 minutes), in both of those cases one threaded pipeline is broken. If so, it seems like the simplest thing would be for you to allow users to
write something like:
Conduit MsgFromServer m MsgToServer
Assuming you had conduits to convert an incoming byte stream to a stream of MsgFromServer and the equivalent for sending, you'd end up with something like:
appSource appData $$ toMsgFromServer =$ clientSuppliedConduit =$ fromMsgToServer =$ appSink appData
Michael
On Tue, Apr 9, 2013 at 1:09 PM, Alexander V Vershilov < alexander.vershilov@gmail.com> wrote:
Hello.
I have next problem: I have a network client that connects to server, listens for messages and generate responces. So the control flow can be represended as:
server -- input -> {generate output} -> output
Output can be generated using default implementation or can overriden by user.
The main difficulty appeares when I need to add a user program on the top of this logic, i.e. from user-side I want to have dsl:smth like
withClient $ do x <- send message waitFor x timeout 5000000 forever $ sendRandomMessage
i.e. an ability to send messages, waiting for some event (message to come), waiting for timeout.
The question is how to define such logic without a big overhead. I see a solution using conduit, it's possible to create 3 processes: listener, user, sender.
+----> user ----+ | | -input -> listener +----------------->+---- sender ->
and use TQueue or TChan to send messages between them, however there can be another possible solutions, that uses less resources, or another design.
-- Alexander
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
-- Alexander

On Wed, Apr 10, 2013 at 2:08 PM, Alexander V Vershilov < alexander.vershilov@gmail.com> wrote:
On 10 April 2013 14:56, Michael Snoyman
wrote: It doesn't seem like you're trying to perform multiple actions simultaneously. For example, you don't need to be able to read from the server and send data back at the same time. Instead, you'll have a single thread of execution. Am I right?
Not exaclty, user code is not only SeverMessage driven but can generate messages and works on it's own (time-events, or some external events). For example user code may generate random messages even there is no message from server, (i.e. wait for some timeout and then feed sender with message), or do some long running events, (e.g. wait for 5 minutes), in both of those cases one threaded pipeline is broken.
Then some kind of TQueue or TChan approach is going to be necessary.

I've been struggling with a similar situation: a client and server that communicate with binary-encoded messages, sending "heartbeats" (dummy messages) every 30 seconds, and timing out the connection if no response is received in 3 minutes. The client sends data to the server, while also listening for configuration changes from the server. The connection needs to interact with multiple threads on both sides. See http://ofps.oreilly.com/titles/9781449335946/sec_conc-server.html (Parallel and Concurrent Programming in Haskell, Chapter 9) for a thorough example of using STM to write a simple networked application. You may want to read some of the previous chapters if you have trouble understanding this one. If your program becomes an overwhelming tangle of threads, tackle it like any other complexity: break your program into modules with simple interfaces. In particular, build your connection module in layers. For example, you could have a module that serializes messages: data Config = Config { host :: HostName, port :: PortNumber } data Connection = Connection { connBase :: Handle , connRecvState :: IORef ByteString -- ^ Leftover bytes from last 'recv' } connect :: Config -> IO Connection close :: Connection -> IO () send :: Connection -> Request -> IO () recv :: Connection -> IO (Maybe Response) On top of that, timed messages: data Connection = Connection { connBase :: Base.Connection , connSendLock :: MVar SendState } data SendState = SendOpen | SendError SomeException Here, both 'send' and a timer thread take the send lock. If either fails, it places 'SendError' in the MVar to prevent subsequent accesses. MVar locking is pretty cheap: about 100 nanoseconds per withMVar, versus several microseconds per network I/O operation. Don't be afraid to stack MVar locks if it makes your code easier to maintain. @Michael Snoyman: The Connection example above is one case where resumable conduits might be useful. For Connection to use conduits, 'recv' and 'send' would have to feed conduits incrementally. Otherwise, it'd need a different interface (e.g. return Source and Sink). I wrote a conduit-resumable package [1], but did not release it because of a semantic issue regarding leftovers: if a conduit has leftovers, should they go back to the source, or stay with the conduit? conduit-resumable does the former, but we'd want the latter here. [1]: https://github.com/joeyadams/hs-conduit-resumable
participants (3)
-
Alexander V Vershilov
-
Joey Adams
-
Michael Snoyman