all threads are blocked by recvFrom

older
Unescaping with HaXmL (or anything...

Vitaliy Akimov

14 Mar 2008 14 Mar '08

9:48 a.m.

Hello, I have a problem with building multithreaded UDP server. If main thread is waiting for new request in recvFrom all other threads are blocked too. I've checked every variant with forkIO,forkOS,-threaded etc, nothing's helped. After reading GHC docs I've understood this is happened becouse foreign function call from recvFrom (network library) is marked to be unsefe, so it's execution blocks every other thread. How can I resolve it? Thank you. Vitaliy.

Show replies by date

Vitaliy Akimov

14 Mar 14 Mar

10:43 a.m.

Rebuilding of the network package with changed safety helped but I don't think this is the solution. BTW accept is declared as safe. What is the reason of declaring recvFrom as unsafe? I think this breaks highly required feature. Apparently it's impossible to make concurrent server for non connection based protocols. 2008/3/14, Vitaliy Akimov :

...

Hello, I have a problem with building multithreaded UDP server. If main thread is waiting for new request in recvFrom all other threads are blocked too. I've checked every variant with forkIO,forkOS,-threaded etc, nothing's helped. After reading GHC docs I've understood this is happened becouse foreign function call from recvFrom (network library) is marked to be unsefe, so it's execution blocks every other thread. How can I resolve it?

Thank you.

Vitaliy.

Adam Langley

10:57 a.m.

On Fri, Mar 14, 2008 at 7:43 AM, Vitaliy Akimov wrote:

...

Rebuilding of the network package with changed safety helped but I don't think this is the solution. BTW accept is declared as safe. What is the reason of declaring recvFrom as unsafe? I think this breaks highly required feature. Apparently it's impossible to make concurrent server for non connection based protocols.

I assume that you're binding the libc function directly here: In that case, you need to have the RTS manage sleeping your thread for you. You should make the socket non-blocking and handle the EAGAIN and EWOULDBLOCK cases by calling threadWaitRead[1] to block the current Haskell thread only, until the fd is readable. Note that threadWaitRead is GHC only. If you download the source to network-bytestring[2], you can see a very similar pattern in the code for send and recv in there. Alternatively, you can use the functions in Network.Socket, which should work fine. [1] http://haskell.org/ghc/docs/latest/html/libraries/base/Control-Concurrent.ht... [2] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/network-bytestrin... -- Adam Langley agl@imperialviolet.org http://www.imperialviolet.org

Vitaliy Akimov

11:51 a.m.

...

I assume that you're binding the libc function directly here:

I'm using Network.Socket. Sory if it's not clear from my previous posts.

...

In that case, you need to have the RTS manage sleeping your thread for you. You should make the socket non-blocking and handle the EAGAIN and EWOULDBLOCK cases by calling threadWaitRead[1] to block the current Haskell thread only, until the fd is readable. Note that threadWaitRead is GHC only. If you download the source to network-bytestring[2], you can see a very similar pattern in the code for send and recv in there.

Thanks, but I haven't managed to find a way of setting a socket into non blocking mode without using FFI directly (and I haven't found solution in network-bytestring too). How can I make this? The only way I've found is making handle by socketToHandle then reading by hGetBufNonBlocking. But this way seems not suited for non-connection based sockets.

Adam Langley

1:13 p.m.

On Fri, Mar 14, 2008 at 8:51 AM, Vitaliy Akimov wrote:

...

...
I assume that you're binding the libc function directly here:

I'm using Network.Socket. Sory if it's not clear from my previous posts.

Then everything should Just Work(tm). You might need to paste in code in order to figure out why this wouldn't be so. See [1] for an example which works for me. It starts a thread which prints "working..." once a second and, in another thread, listens for UDP packets on port 4112. I can use `nc -u 127.0.0.1 4112` to get this: "working..." "working..." ("testing\n",8,127.0.0.1:36179) "working..." "working..." ("testing two\n",12,127.0.0.1:36179) "working..." [1] http://hpaste.org/6362 -- Adam Langley agl@imperialviolet.org http://www.imperialviolet.org

Adam Langley

1:18 p.m.

On Fri, Mar 14, 2008 at 10:13 AM, Adam Langley wrote:

...

See [1] for an example which works for me.

(If you're on Windows, you probably need to wrap main in withSocketsDo) AGL -- Adam Langley agl@imperialviolet.org http://www.imperialviolet.org

Vitaliy Akimov

17 Mar 17 Mar

5:29 a.m.

Hi Adam, sorry for late answer. Here is my example [1], but yours doesn't work on my PC too. And it's strange it works on yours. According to documentation for Control.Concurrent module [2] every other thread should be blocked.

...

With the -threaded option, only foreign calls with the unsafe attribute will block all other threads. And after changing in the network package FFI declaration for c_recvfrom from unsafe to safe both examples start working. There are two solutions I see now. The first is to copy-paste definitions from the network package to mine with changing "unsafe" to "safe" for FFI declaration. The second is to use unblocking sockets. This by the way will help to get rid of hack-like solution of stopping server by closing listening socket, but it will get more effort.

[1] http://hpaste.org/6426 [2] http://haskell.org/ghc/docs/latest/html/libraries/base/Control-Concurrent.ht... 2008/3/14, Adam Langley :

...

On Fri, Mar 14, 2008 at 8:51 AM, Vitaliy Akimov

wrote:

...
...
I assume that you're binding the libc function directly here:

I'm using Network.Socket. Sory if it's not clear from my previous posts.

Then everything should Just Work(tm). You might need to paste in code in order to figure out why this wouldn't be so.

See [1] for an example which works for me. It starts a thread which prints "working..." once a second and, in another thread, listens for UDP packets on port 4112. I can use `nc -u 127.0.0.1 4112` to get this: "working..." "working..." ("testing\n",8,127.0.0.1:36179) "working..." "working..." ("testing two\n",12,127.0.0.1:36179) "working..."

[1] http://hpaste.org/6362

-- Adam Langley agl@imperialviolet.org http://www.imperialviolet.org

Adam Langley

1:06 p.m.

On Mon, Mar 17, 2008 at 2:29 AM, Vitaliy Akimov wrote:

...

Hi Adam, sorry for late answer. Here is my example [1], but yours doesn't work on my PC too. And it's strange it works on yours.

Are you running Windows? Because you're hpaste example works, nonblocking for me too.

...

And after changing in the network package FFI declaration for c_recvfrom from unsafe to safe both examples start working.

The important point here is that the recvFrom calls in Network.Socket[1] don't block. They are non-blocking calls so, although other threads may be suspended while in the FFI call, the FFI call returns -EAGAIN if it would block and blocking is passed to the RTS to handle. While sleeping in the RTS (with threadWaitRead etc), other Haskell threads can read. See the code in [1] (esp throwErrnoIfMinus1Retry_repeatOnBlock) So, the only reason that other threads would be blocked is if the socket wasn't marked as non-blocking. Here's a snippet of strace output from compiling your hpaste example: socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 5 fcntl64(5, F_GETFL) = 0x2 (flags O_RDWR) fcntl64(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 You can see that the socket is set to non-blocking immediately. [1] http://haskell.org/ghc/docs/latest/html/libraries/network/src/Network-Socket... -- Adam Langley agl@imperialviolet.org http://www.imperialviolet.org

Vitaliy Akimov

18 Mar 18 Mar

5:32 a.m.

2008/3/17, Adam Langley :

...

On Mon, Mar 17, 2008 at 2:29 AM, Vitaliy Akimov

The important point here is that the recvFrom calls in Network.Socket[1] don't block.

Yes, this is the answer. Network.Socket.socket calls System.Posix.Internals.setNonBlockingFD to set socket non-blocking. I'm on Windows, and this function is simply "return ()" for this platform. So that's why it doesn't work on Windows, I think I should find some way to make a socket unblocking after its creation. Thank you. Vitaliy.

Vitaliy Akimov

6:41 a.m.

...

So that's why it doesn't work on Windows, I think I should find some way to make a socket unblocking after its creation.

Unfortunally this way seems to be wrong. Error codes for winsockets and BSD-sockets are different.

Adam Langley

11:57 a.m.

On Tue, Mar 18, 2008 at 3:41 AM, Vitaliy Akimov wrote:

...

Unfortunally this way seems to be wrong. Error codes for winsockets and BSD-sockets are different.

Hmm, "Networking broken on Windows" would seem to be a pretty big issue. One of the Windows peeps like to speak up here? AGL -- Adam Langley agl@imperialviolet.org http://www.imperialviolet.org

Vitaliy Akimov

12:51 p.m.

It's not an issue of error codes. Though error code of WSAEWOULDBLOCK and EAGAIN is different this is not the problem. You've posted a link to preprocessed source code. And there is no implementation of throwErrnoIfMinus1Retry_repeatOnBlock for Windows. This function for Windows treat WSAEWOULDBLOCK as error, and it doesn't have loop for its correct processing. It's not a surprise since the socket isn't supposed to be blocked on windows. Here is source code with setting socket to non-blocking mode [1]. It terminates with error:

...

UdpEcho.exe: recvFrom: failed (Resource temporarily unavailable (WSAEWOULDBLOCK))

[1] http://hpaste.org/6476 Vitaliy. 2008/3/18, Adam Langley :

...

On Tue, Mar 18, 2008 at 3:41 AM, Vitaliy Akimov

wrote:

...
Unfortunally this way seems to be wrong. Error codes for winsockets and BSD-sockets are different.

Hmm, "Networking broken on Windows" would seem to be a pretty big issue. One of the Windows peeps like to speak up here?

AGL

-- Adam Langley agl@imperialviolet.org http://www.imperialviolet.org

Simon Marlow

1 Apr 1 Apr

1:15 p.m.

Vitaliy Akimov wrote:

...

Hello, I have a problem with building multithreaded UDP server. If main thread is waiting for new request in recvFrom all other threads are blocked too. I've checked every variant with forkIO,forkOS,-threaded etc, nothing's helped. After reading GHC docs I've understood this is happened becouse foreign function call from recvFrom (network library) is marked to be unsefe, so it's execution blocks every other thread. How can I resolve it?

Sorry for the late reply. This will be fixed in GHC 6.8.3: http://hackage.haskell.org/trac/ghc/ticket/1129 Cheers, Simon

6261

Age (days ago)

6279

Last active (days ago)

List overview

Download

12 comments

3 participants

participants (3)

Adam Langley
Simon Marlow
Vitaliy Akimov