[GHC] #14900: Calling isByteArrayPinned# on compact ByteArray

7 Mar 2018

      #14900: Calling isByteArrayPinned# on compact ByteArray
-------------------------------------+-------------------------------------
           Reporter:  andrewthad     |             Owner:  (none)
               Type:  bug            |            Status:  new
           Priority:  normal         |         Milestone:
          Component:  Compiler       |           Version:  8.2.2
           Keywords:                 |  Operating System:  Unknown/Multiple
       Architecture:                 |   Type of failure:  None/Unknown
  Unknown/Multiple                   |
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 The documentation explaining the relationship between pinnedness and
 compact regions is incomplete. From `Data.Compact`:
...
Pinned ByteArray# objects cannot be compacted. This is for a good
 reason: the memory is pinned so that it can be referenced by address (the
 address might be stored in a C data structure, for example), so we can't
 make a copy of it to store in the Compact.
...
Data in a compact doesn't ever move, so compacting data is also a way to
This is half-way true since it only considers one a the ways in which the
 GHC runtime tracks the pinnedness of an object. From experimenting with
 compact regions, it seems like there are two different notions of
 pinnedness:

 1. Did the user explicitly ask for the `ByteArray` to be pinned?
 2. Is the `ByteArray` pinned?

 If (1) is true, then (2) must always be true, but if (1) is false, then
 could be true or false. `ByteArray`s over 3KB are pinned, and `ByteArray`s
 under 3KB are not (or somewhere around that number).

 With that background information in place, here's the scenario I've
 encountered:

 {{{
 {-# LANGUAGE MagicHash #-}

 import Data.Primitive
 import Data.Compact
 import GHC.Int
 import GHC.Prim

 main :: IO ()
 main = do
   ByteArray arr1# <- fmap getCompact $ newByteArray 65000 >>=
 unsafeFreezeByteArray >>= compact
   ByteArray arr2# <- newByteArray 65000 >>= unsafeFreezeByteArray
   print (I# (isByteArrayPinned# arr1#))
   print (I# (isByteArrayPinned# arr2#))
   putStrLn "Finished"
 }}}

 When compiled and run, this gives:

 {{{
 0
 1
 Finished
 }}}

 We can see that the 65KiB `ByteArray` that was not compacted let's the
 user know that it is pinned. The compacted `ByteArray` claims to not be
 pinned, but this is not true. The docs in `Data.Compact` claim:

 pin arbitrary data structures in memory.

 I propose that the behavior of `compact` be modified to accurately convey
 the pinnedness of the `ByteArray`s that copies. This would mean that even
 small, previously unpinned, `ByteArray`s would also be designated as
 pinned. It's a small change, but it makes the information the runtime
 gives us more accurate. This is occasionally handy when dealing with the
 FFI.

-- 
Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14900
GHC http://www.haskell.org/ghc/
The Glasgow Haskell Compiler