#7718: ios patch no 8: adjustor pools ------------------------------+--------------------------------------------- Reporter: StephenBlackheath | Owner: Type: feature request | Status: patch Priority: normal | Component: Compiler Version: 7.7 | Keywords: Os: Other | Architecture: arm Failure: None/Unknown | Blockedby: Blocking: | Related: ------------------------------+--------------------------------------------- Changes (by StephenBlackheath): * status: new => patch -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7718#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7718: ios patch no 8: adjustor pools ----------------------------------+----------------------------------------- Reporter: StephenBlackheath | Owner: Type: feature request | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.7 Keywords: | Os: Other Architecture: arm | Failure: None/Unknown Difficulty: Unknown | Testcase: Blockedby: | Blocking: 7724 Related: | ----------------------------------+----------------------------------------- Changes (by igloo): * difficulty: => Unknown Comment: If we have to do this, can't we share a single pool between all the wrappers? Then the number of them can be specified as a linker flag. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7718#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7718: ios patch no 8: adjustor pools --------------------------------+------------------------------------------- Reporter: StephenBlackheath | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.7 Resolution: | Keywords: Os: Other | Architecture: arm Failure: None/Unknown | Difficulty: Unknown Testcase: | Blockedby: Blocking: 7724 | Related: --------------------------------+------------------------------------------- Comment(by simonmar): Are you sure there's no way to do this on iOS? I googled "libffi ios" and there's a port on github that claims to have been merged into upstream libffi: https://github.com/landonf/libffi-ios I'd be surprised if there were no way to do it at all, because we only need the same functionality as dynamically-loaded libraries. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7718#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7718: ios patch no 8: adjustor pools --------------------------------+------------------------------------------- Reporter: StephenBlackheath | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.7 Resolution: | Keywords: Os: Other | Architecture: arm Failure: None/Unknown | Difficulty: Unknown Testcase: | Blockedby: Blocking: 7724 | Related: --------------------------------+------------------------------------------- Comment(by StephenBlackheath): UPDATE: Those changes were done in Apr 12, 2011 so they're likely to have been integrated, but I haven't checked fully. What I did do was just switch LIBFFI on for iOS and see what it would do. It generated a function pointer pointing to garbage. So this is what I've got to do: 1. Find out why LIBFFI doesn't work. 2. If it really can't be fixed, then finish developing the global adjustor pool. This is the only outstanding work for the iOS cross compiler now. Watch this space. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7718#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7718: ios patch no 8: adjustor pools --------------------------------+------------------------------------------- Reporter: StephenBlackheath | Owner: Type: feature request | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.7 Resolution: | Keywords: Os: Other | Architecture: arm Failure: None/Unknown | Difficulty: Unknown Testcase: | Blockedby: Blocking: 7724 | Related: --------------------------------+------------------------------------------- Comment(by StephenBlackheath): I forgot to mention that I re-tested this patch on Linux to make sure my refactoring didn't break it: {{{ {-# LANGUAGE ForeignFunctionInterface #-} import Foreign import Foreign.C foreign import ccall "wrapper" wrap :: (CInt -> IO CInt) -> IO (FunPtr (CInt -> IO CInt)) foreign import ccall safe "callme" callme :: (FunPtr (CInt -> IO CInt)) -> IO () main = do putStrLn "hi" f <- wrap $ \i -> do putStrLn $ "haskell got "++show i return $ i + 1 callme f freeHaskellFunPtr f putStrLn "bye" }}} {{{ #include int callme(int (*f)(int)) { printf("%d -> %d\n", 5, f(5)); printf("%d -> %d\n", 10, f(10)); } }}} {{{ hi haskell got 5 5 -> 6 haskell got 10 10 -> 11 bye }}} -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7718#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7718: ios patch no 8: adjustor pools --------------------------------+------------------------------------------- Reporter: StephenBlackheath | Owner: Type: feature request | Status: patch Priority: normal | Milestone: 7.8.1 Component: Compiler | Version: 7.7 Resolution: | Keywords: Os: Other | Architecture: arm Failure: None/Unknown | Difficulty: Unknown Testcase: | Blockedby: Blocking: 7724 | Related: --------------------------------+------------------------------------------- Description changed by igloo: Old description:

"Adjustor" is the term used for a C function pointer that allows C code to call back to Haskell. Normally these are generated at runtime.

However, the iOS kernel doesn't allow self-modifying code. So, on iOS we use a pool of precompiled adjustors of a fixed size, and this patch is the implementation for that.

It consists of three parts:

1. A POOLSIZE pragma, that is used like this:

foreign import ccall safe "wrapper" {-# POOLSIZE 100 #-} mkDelegate :: IO () -> IO (FunPtr (IO ()))

This patch makes this pragma work on all platforms, but it'll have no effect on platforms other than iOS.

I am not sure what the procedure is for additions of pragmas. Do pragmas require {-# LANGUAGE xx #-} ? Anyway, please review whether the approach taken here is acceptable.

2. The Haskell code in the compiler to generate the stubs for the pooled adjustors.

3. The runtime system's implementation of pooled adjustors in C.

New description: "Adjustor" is the term used for a C function pointer that allows C code to call back to Haskell. Normally these are generated at runtime. However, the iOS kernel doesn't allow self-modifying code. So, on iOS we use a pool of precompiled adjustors of a fixed size, and this patch is the implementation for that. It consists of three parts: 1. A POOLSIZE pragma, that is used like this: {{{ foreign import ccall safe "wrapper" {-# POOLSIZE 100 #-} mkDelegate :: IO () -> IO (FunPtr (IO ())) }}} This patch makes this pragma work on all platforms, but it'll have no effect on platforms other than iOS. I am not sure what the procedure is for additions of pragmas. Do pragmas require {-# LANGUAGE xx #-} ? Anyway, please review whether the approach taken here is acceptable. 2. The Haskell code in the compiler to generate the stubs for the pooled adjustors. 3. The runtime system's implementation of pooled adjustors in C. -- -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7718#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7718: ios patch no 8: adjustor pools --------------------------------+------------------------------------------- Reporter: StephenBlackheath | Owner: Type: feature request | Status: patch Priority: normal | Milestone: 7.8.1 Component: Compiler | Version: 7.7 Resolution: | Keywords: Os: Other | Architecture: arm Failure: None/Unknown | Difficulty: Unknown Testcase: | Blockedby: Blocking: 7724 | Related: --------------------------------+------------------------------------------- Comment(by ian@…): commit 310735e7adce0145c653386c21686b4a1b96aea9 {{{ Author: Ian Lynagh Date: Sun Apr 21 16:03:40 2013 +0100 Use ffi_prep_closure_loc rather than ffi_prep_closure The latter is deprecated. Part of #7718. rts/Adjustor.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) }}} -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7718#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7718: ios patch no 8: adjustor pools --------------------------------+------------------------------------------- Reporter: StephenBlackheath | Owner: Type: feature request | Status: patch Priority: normal | Milestone: 7.8.1 Component: Compiler | Version: 7.7 Resolution: | Keywords: Os: Other | Architecture: arm Failure: None/Unknown | Difficulty: Unknown Testcase: | Blockedby: Blocking: 7724 | Related: --------------------------------+------------------------------------------- Comment(by igloo): Here's the Linux code: {{{ void *allocateExec (W_ bytes, void **exec_ret) { void **ret, **exec; ACQUIRE_SM_LOCK; ret = ffi_closure_alloc (sizeof(void *) + (size_t)bytes, (void**)&exec); RELEASE_SM_LOCK; if (ret == NULL) return ret; *ret = ret; // save the address of the writable mapping, for freeExec(). *exec_ret = exec + 1; return (ret + 1); } }}} Note that it puts the writable address at the start of the memory allocated, so `freeExec` can find it: {{{ void freeExec (void *addr) { void *writable; writable = *((void**)addr - 1); }}} Doesn't this do what you use the hash table for? -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7718#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7718: ios patch no 8: adjustor pools --------------------------------+------------------------------------------- Reporter: StephenBlackheath | Owner: Type: feature request | Status: patch Priority: normal | Milestone: 7.8.1 Component: Compiler | Version: 7.7 Resolution: | Keywords: Os: Other | Architecture: arm Failure: None/Unknown | Difficulty: Unknown Testcase: | Blockedby: Blocking: 7724 | Related: --------------------------------+------------------------------------------- Comment(by simonmar): Stephen - we do have Linux platforms that return different addresses for exec and writable: SE Linux on Fedora, for example. The two addresses refer to the same memory, which is why you can write to one and read from the other. I haven't looked at your patch in detail, but I am also wondering why we need an extra hash table. When I wrote this code it was intended to support the write and exec addresses being different. We had a bug in this that was discovered recently, see #7629, and with the fix it is working fine. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7718#comment:18 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7718: ios patch no 8: adjustor pools --------------------------------+------------------------------------------- Reporter: StephenBlackheath | Owner: Type: feature request | Status: patch Priority: normal | Milestone: 7.8.1 Component: Compiler | Version: 7.7 Resolution: | Keywords: Os: Other | Architecture: arm Failure: None/Unknown | Difficulty: Unknown Testcase: | Blockedby: Blocking: 7724 | Related: --------------------------------+------------------------------------------- Comment(by StephenBlackheath): Sorry - that was slightly wrong: It's actually ffi_closure_free() on line 529 that uses trampoline_table and trampoline_table_entry fields stored by ffi_closure_alloc() (line 756), not the closure code. It seems that even though they assume they're using the same structure, ffi_closure_alloc/free() and ffi_prep_closure_loc() could possibly be indepdendent if you make several assumptions about the internal functioning of libffi and allocate two adjacent ffi_closure structures (though the existing Linux implementation would not work). The performance overhead of the hash table seems preferable to the risk of breakage caused by future changes to libffi. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7718#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7718: ios patch no 8: adjustor pools --------------------------------+------------------------------------------- Reporter: StephenBlackheath | Owner: Type: feature request | Status: patch Priority: normal | Milestone: 7.8.1 Component: Compiler | Version: 7.7 Resolution: | Keywords: Os: Other | Architecture: arm Failure: None/Unknown | Difficulty: Unknown Testcase: | Blockedby: Blocking: 7724 | Related: --------------------------------+------------------------------------------- Comment(by StephenBlackheath): It's specifically the iOS version of libffi (not ARM) that has the assumption that ffi_closure_alloc() / ffi_prep_closure_loc() are dealing with a ffi_closure*, and currently the RTS assumes we can do whatever we like with this memory (these two assumptions being incompatible). As you say, if the RTS managed both addresses and was able to pass the write address back to freeExec, we wouldn't need the hash table I put into my ios-patch-8e. The executable pointer is the only one that gets passed to C code when you use a FunPtr through the FFI, though. Even though it would be unusual, it seems that it is possible to pass a FunPtr that originated with a 'foreign import ccall "wrapper"' into C code and back out, and then freeHaskellFunPtr it. If this is so, I can't see how the hash table can be completely avoided. I think it's a reasonable compromise to go with my ios-patch-8e and pay a small performance penalty on freeHaskellFunPtr on iOS only rather than complicating the RTS for the benefit of a single architecture. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7718#comment:22 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7718: ios patch no 8: adjustor pools --------------------------------+------------------------------------------- Reporter: StephenBlackheath | Owner: Type: feature request | Status: closed Priority: normal | Milestone: 7.8.1 Component: Compiler | Version: 7.7 Resolution: fixed | Keywords: Os: Other | Architecture: arm Failure: None/Unknown | Difficulty: Unknown Testcase: | Blockedby: Blocking: 7724 | Related: --------------------------------+------------------------------------------- Changes (by igloo): * status: patch => closed * resolution: => fixed Comment: OK, thanks Stephen. I've applied a slightly simplified version of the patch. I haven't been able to test on iOS, so please let me know if it doesn't work there. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7718#comment:24 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler