Allocating a new ByteString

I'm calling a function by FFI that returns two strings by writing into output buffers that you provide. The code I currently have looks like this (minus some error handling):
import qualified Data.ByteString as S import qualified Data.ByteString.Unsafe as SU
import Foreign.C (CChar, CInt(..), CSize(..)) import Foreign.ForeignPtr (mallocForeignPtrBytes, withForeignPtr) import Foreign.Ptr (Ptr) import System.IO.Unsafe (unsafePerformIO)
seed_keypair :: S.ByteString -> (S.ByteString, S.ByteString) seed_keypair seed | S.length seed /= signSeed = error "seed has incorrect length" | otherwise = unsafePerformIO $ do pk <- mallocForeignPtrBytes signPK sk <- mallocForeignPtrBytes signSK SU.unsafeUseAsCString seed $ \pseed -> withForeignPtr pk $ \ppk -> withForeignPtr sk $ \psk -> do 0 <- c_sign_seed_keypair ppk psk pseed bpk <- S.packCStringLen (ppk, signPK) bsk <- S.packCStringLen (psk, signSK) return (bpk, bsk)
foreign import ccall "crypto_sign_seed_keypair" c_sign_seed_keypair :: Ptr CChar -> Ptr CChar -> Ptr CChar -> IO CInt
However, this needlessly makes a copy of the output buffers to create the final result. What I really want to do is just write the string directly into a buffer allocated and used by a new ByteString; is there some way to accomplish this? (Any other comments about what I'm doing would also be appreciated, this happens to be my first attempt at using FFI!) -- mithrandi, i Ainil en-Balandor, a faer Ambar

Looks like you need 'unsafePackCStringFinalizer' (or one of it's variants) from Data.ByteString.Unsafe module. It lets you reuse the buffer and deallocate it in finalizer. Something similar is used e.g. in 'kyotocabinet' package. 23 Дек 2014 г. 1:48 пользователь "Tristan Seligmann" < mithrandi@mithrandi.net> написал:
I'm calling a function by FFI that returns two strings by writing into output buffers that you provide. The code I currently have looks like this (minus some error handling):
import qualified Data.ByteString as S import qualified Data.ByteString.Unsafe as SU
import Foreign.C (CChar, CInt(..), CSize(..)) import Foreign.ForeignPtr (mallocForeignPtrBytes, withForeignPtr) import Foreign.Ptr (Ptr) import System.IO.Unsafe (unsafePerformIO)
seed_keypair :: S.ByteString -> (S.ByteString, S.ByteString) seed_keypair seed | S.length seed /= signSeed = error "seed has incorrect length" | otherwise = unsafePerformIO $ do pk <- mallocForeignPtrBytes signPK sk <- mallocForeignPtrBytes signSK SU.unsafeUseAsCString seed $ \pseed -> withForeignPtr pk $ \ppk -> withForeignPtr sk $ \psk -> do 0 <- c_sign_seed_keypair ppk psk pseed bpk <- S.packCStringLen (ppk, signPK) bsk <- S.packCStringLen (psk, signSK) return (bpk, bsk)
foreign import ccall "crypto_sign_seed_keypair" c_sign_seed_keypair :: Ptr CChar -> Ptr CChar -> Ptr CChar -> IO CInt
However, this needlessly makes a copy of the output buffers to create the final result. What I really want to do is just write the string directly into a buffer allocated and used by a new ByteString; is there some way to accomplish this?
(Any other comments about what I'm doing would also be appreciated, this happens to be my first attempt at using FFI!) -- mithrandi, i Ainil en-Balandor, a faer Ambar _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On 23 December 2014 at 08:10, Yuras Shumovich
Looks like you need 'unsafePackCStringFinalizer' (or one of it's variants) from Data.ByteString.Unsafe module. It lets you reuse the buffer and deallocate it in finalizer. Something similar is used e.g. in 'kyotocabinet' package.
I tried using this, but it seems like the pointer created by mallocForeignPtrBytes is freed as soon as it is no longer referenced (upon returning from the function). Then, later, when the bytestring is finalized, the pointer is freed again causing an abort on double-free. How do I "transfer" ownership of the pointer? -- mithrandi, i Ainil en-Balandor, a faer Ambar

On Tue, 2014-12-23 at 08:19 +0200, Tristan Seligmann wrote:
On 23 December 2014 at 08:10, Yuras Shumovich
wrote: Looks like you need 'unsafePackCStringFinalizer' (or one of it's variants) from Data.ByteString.Unsafe module. It lets you reuse the buffer and deallocate it in finalizer. Something similar is used e.g. in 'kyotocabinet' package.
I tried using this, but it seems like the pointer created by mallocForeignPtrBytes is freed as soon as it is no longer referenced (upon returning from the function). Then, later, when the bytestring is finalized, the pointer is freed again causing an abort on double-free. How do I "transfer" ownership of the pointer?
'mallocForeignPtrBytes' attaches finalizer to the 'ForeignPtr', but you need to attach it to 'ByteString'. So you can't use 'mallocForeignPtrBytes', but plain old 'malloc' (from 'Foreign.Marshal.Alloc') should work in combination with 'unsafePackMallocCStringLen'. Unless I'm missing something. Thanks, Yuras

On 23 December 2014 at 08:47, Yuras Shumovich
On Tue, 2014-12-23 at 08:19 +0200, Tristan Seligmann wrote:
On 23 December 2014 at 08:10, Yuras Shumovich
wrote: Looks like you need 'unsafePackCStringFinalizer' (or one of it's variants) from Data.ByteString.Unsafe module. It lets you reuse the buffer and deallocate it in finalizer. Something similar is used e.g. in 'kyotocabinet' package.
I tried using this, but it seems like the pointer created by mallocForeignPtrBytes is freed as soon as it is no longer referenced (upon returning from the function). Then, later, when the bytestring is finalized, the pointer is freed again causing an abort on double-free. How do I "transfer" ownership of the pointer?
'mallocForeignPtrBytes' attaches finalizer to the 'ForeignPtr', but you need to attach it to 'ByteString'. So you can't use 'mallocForeignPtrBytes', but plain old 'malloc' (from 'Foreign.Marshal.Alloc') should work in combination with 'unsafePackMallocCStringLen'. Unless I'm missing something.
Aha, that seems to work, thanks! I think I do have one minor issue left; if an exception happens in the middle of that code, the memory will never be free()d, but that should be solvable with an appropriate exception handling construction, I just need to figure out exactly what. (Actual code is here[1] if anyone is interested) [1] https://github.com/mithrandi/ssh-key-generator/blob/master/Argh.hs#L11 -- mithrandi, i Ainil en-Balandor, a faer Ambar
participants (2)
-
Tristan Seligmann
-
Yuras Shumovich