Locking of threads in one OS thread

Hello Cafe, I'm using FFI to interact with a library which calls, when fail, leave the reason in some kind of "errno"-like variable which is retrived via another call. AFAIU, this is not thread-safe in Haskell even if thread-local storage is used inside the library, because Haskell uses its own thread management and the Haskell thread in the same OS thread might be switched between the actual call and the retrival of errno value. This should be somehow handled already in Haskell (errno is widely used with syscalls in Linux, for example), but the source of Foreign.C.Error suggests that this is not handled in any way at all. For example, throwErrnoIf is implemented as such: throwErrno loc = do errno <- getErrno ioError (errnoToIOError loc errno Nothing Nothing) throwErrnoIf pred loc f = do res <- f if pred res then throwErrno loc else return res So, the question is: how is it ensured that this block is "atomic" in sense that at most one Haskell thread computes this whole function at every moment in every OS thread? Thanks for any explanations! (And sorry if my English is poor) Nikolay.

several ideas
a) you can use forkOS to run the ffi call and the the subsequent errno
value checking in a pinned OS thread so you can guarantee you can access
the thread local storage
b) you could have a CallManager that keeps a MVar or queue or something,
along with a handle on the forkOS thread id, and "intiialize" the forkOS
thread with a teeny worker that read from an input MVar and writes to a
result MVar (thus guarantee only one call a time)
c) something that uses these ideas together.
theres a lot of strategies you can use. but i hope this gives you a sketch
(i can spell it out more later this week if need be)
On Tue, Aug 19, 2014 at 5:36 PM, Nikolay Amiantov
Hello Cafe,
I'm using FFI to interact with a library which calls, when fail, leave the reason in some kind of "errno"-like variable which is retrived via another call. AFAIU, this is not thread-safe in Haskell even if thread-local storage is used inside the library, because Haskell uses its own thread management and the Haskell thread in the same OS thread might be switched between the actual call and the retrival of errno value. This should be somehow handled already in Haskell (errno is widely used with syscalls in Linux, for example), but the source of Foreign.C.Error suggests that this is not handled in any way at all. For example, throwErrnoIf is implemented as such:
throwErrno loc = do errno <- getErrno ioError (errnoToIOError loc errno Nothing Nothing)
throwErrnoIf pred loc f = do res <- f if pred res then throwErrno loc else return res
So, the question is: how is it ensured that this block is "atomic" in sense that at most one Haskell thread computes this whole function at every moment in every OS thread?
Thanks for any explanations! (And sorry if my English is poor)
Nikolay. _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Thanks for the answer! I've thought about (c) already, and (a) and (b)
are interesting, too -- maybe (a) is my answer.
I'm very interested in how Haskell handles this already -- for
example, in "network" or "unix" package sources I haven't found any
handling of this (see [1], [2] for example) and it looks to me that
these packages are prone to this race condition -- am I right?
If this is not a known error, then my thoughts is that to add a way to
fix this without losing much speed we could add new prims -- something
like "maskPreemption#", "unmaskPreemption#" and "getPreemptionState#"
(using "mask" implementation as an example) and new "maskPreemption"
function in Control.Concurrent, for example. Prims can use
thread-local flag without any locking that runtime checks before
thread preemption -- this should be nearly unnoticeable operation in
terms of performance.
This is all just an idea of how I would implement this, and I don't
have knowledge of RTS internals, so for someone competent this may be
utter nonsense -- I'm sorry for this.
[1]: https://hackage.haskell.org/package/network-2.6.0.1/docs/src/Network-Socket-...
[2]: http://hackage.haskell.org/package/unix-2.7.0.1/docs/src/System-Posix-Error....
С уважением,
Николай Амиантов.
On Wed, Aug 20, 2014 at 5:02 AM, Carter Schonwald
several ideas a) you can use forkOS to run the ffi call and the the subsequent errno value checking in a pinned OS thread so you can guarantee you can access the thread local storage b) you could have a CallManager that keeps a MVar or queue or something, along with a handle on the forkOS thread id, and "intiialize" the forkOS thread with a teeny worker that read from an input MVar and writes to a result MVar (thus guarantee only one call a time) c) something that uses these ideas together.
theres a lot of strategies you can use. but i hope this gives you a sketch (i can spell it out more later this week if need be)
On Tue, Aug 19, 2014 at 5:36 PM, Nikolay Amiantov
wrote: Hello Cafe,
I'm using FFI to interact with a library which calls, when fail, leave the reason in some kind of "errno"-like variable which is retrived via another call. AFAIU, this is not thread-safe in Haskell even if thread-local storage is used inside the library, because Haskell uses its own thread management and the Haskell thread in the same OS thread might be switched between the actual call and the retrival of errno value. This should be somehow handled already in Haskell (errno is widely used with syscalls in Linux, for example), but the source of Foreign.C.Error suggests that this is not handled in any way at all. For example, throwErrnoIf is implemented as such:
throwErrno loc = do errno <- getErrno ioError (errnoToIOError loc errno Nothing Nothing)
throwErrnoIf pred loc f = do res <- f if pred res then throwErrno loc else return res
So, the question is: how is it ensured that this block is "atomic" in sense that at most one Haskell thread computes this whole function at every moment in every OS thread?
Thanks for any explanations! (And sorry if my English is poor)
Nikolay. _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On Tue, Aug 19, 2014 at 11:36 PM, Nikolay Amiantov
Hello Cafe,
I'm using FFI to interact with a library which calls, when fail, leave the reason in some kind of "errno"-like variable which is retrived via another call. AFAIU, this is not thread-safe in Haskell even if thread-local storage is used inside the library, because Haskell uses its own thread management and the Haskell thread in the same OS thread might be switched between the actual call and the retrival of errno value.
Good question!
This should be somehow handled already in Haskell (errno is widely used with syscalls in Linux, for example), but the source of Foreign.C.Error suggests that this is not handled in any way at all. For example, throwErrnoIf is implemented as such:
throwErrno loc = do errno <- getErrno ioError (errnoToIOError loc errno Nothing Nothing)
throwErrnoIf pred loc f = do res <- f if pred res then throwErrno loc else return res
So, the question is: how is it ensured that this block is "atomic" in sense that at most one Haskell thread computes this whole function at every moment in every OS thread?
I do not know the RTS very well, but I think this might be unsafe. If the thread is *bound*, then *getErrno* is guaranteed to be executed on the same thread, but otherwise no such guarantee is given. However, reading http://blog.ezyang.com/2013/01/the-ghc-scheduler/ it might be unlikely. If GC happens, "Threads are put in *front* (pushOnRunQueue) if: ... In the threaded runtime, if a thread was interrupted because another Capability needed to do a stop-the-world GC (see commit 6d18141d8);" However, the same post indicates that you can force this behavior using signals. "Threads are put in *back* (appendToRunQueue) in the case of pre-emption, or if it’s new; particularly, if: ...A thread was pre-empted via the context switch flag (e.g. incoming message from another thread, the timer fired, the thread cooperatively yielded, etc; see also [8] on how this interacts with heap overflows);" Reading errno directly after the FFI call can eliminate heap overflows, but the async exception and timer issues still seem possible. I would also like to see a good explanation of this. Alexander

On Wed, Aug 20, 2014 at 4:36 PM, Alexander Kjeldaas < alexander.kjeldaas@gmail.com> wrote:
Reading errno directly after the FFI call can eliminate heap overflows, but the async exception and timer issues still seem possible.
I have played around a bit and I can't understand what's going on with this anymore. Check out this example: https://gist.github.com/abbradar/76dafcee1807c9c5ac4d. Compile it with "ghc test_c.c test.hs". I used "mask_" here to check if it can fix the problem. There will be multiple discrepancies seen between written and read values because of threads preemption if my_errno is used. However, if you use "errno" (change #define for that) instead, everything seems good. Anyway, it looks like getErrno and friends rely on this magical behaviour of errno -- if some other library which uses global error state like "my_errno" in example is used (I remember SDL doing that, and I have also done it myself), there should be problems without some way to temporary disable threads preemption, which I haven't found. Hence -- should I maybe post a bug report about this?

AFAIK "errno" is handled explicitly in the RTS:
https://ghc.haskell.org/trac/ghc/browser/ghc/rts/Schedule.c#L484
-Sylvain
2014-08-26 16:22 GMT+02:00 Nikolay Amiantov
On Wed, Aug 20, 2014 at 4:36 PM, Alexander Kjeldaas < alexander.kjeldaas@gmail.com> wrote:
Reading errno directly after the FFI call can eliminate heap overflows, but the async exception and timer issues still seem possible.
I have played around a bit and I can't understand what's going on with this anymore. Check out this example: https://gist.github.com/abbradar/76dafcee1807c9c5ac4d. Compile it with "ghc test_c.c test.hs". I used "mask_" here to check if it can fix the problem.
There will be multiple discrepancies seen between written and read values because of threads preemption if my_errno is used. However, if you use "errno" (change #define for that) instead, everything seems good.
Anyway, it looks like getErrno and friends rely on this magical behaviour of errno -- if some other library which uses global error state like "my_errno" in example is used (I remember SDL doing that, and I have also done it myself), there should be problems without some way to temporary disable threads preemption, which I haven't found. Hence -- should I maybe post a bug report about this?
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
participants (4)
-
Alexander Kjeldaas
-
Carter Schonwald
-
Nikolay Amiantov
-
Sylvain Henry