April 2001 - FFI - Haskell.org

FFI Definition
by Manuel M. T. Chakravarty 07 May '01

07 May '01

Folks, As promised, I have started on a definition of the Haskell FFI. I started by formalising those aspects of foreign exports and imports that we have recently discussed here. You can access the current text at http://www.cse.unsw.edu.au/~chak/haskell/ffi.ps.gz In addition to general comments, I would particularly welcome your opinion on the FIXME issues. You may note that I have tilted the specification of the calling convention and the related string determining the external entity very much into the direction of the execution platform. In my opinion that makes the whole description much more consistent. In particular, we are now talking about JVM calls rather than Java calls. This fits much better with ccall, wincall, and dotnet. It also makes it much more natural to talk about calling conventions rather than programming languages implemented in terms of these calling conventions. I will add more material (including also a specification of the FFI library) as we go. Cheers, Manuel

5 12

RE: FFI typing issues [was: Typing f.e.d.]
by Simon Marlow 30 Apr '01

30 Apr '01

> The first one is the typing of `foreign import dynamic', which is > documented as > > Addr -> (prim_args -> IO prim_result) (actually the Addr can currently be 'Ptr a' or 'FunPtr a' too). > GHC currently implements more or less `Addr -> prim_type', > but that's not > the point here. I propose changing this to > > FunPtr prim_type1 -> prim_type2 -- where prim_type1 == prim_type2 > > again deprecating and keeping the old typing. Apart from the purely > technical `IO' (hmmmm, is this *really* necessary?), it makes things > almost symmetrical to f.e.d. Looks reasonable to me. > Another point is the typing of `foreign label', which is > currently simply > > Addr > > I propose to change this to *two* possible typings: > > Ptr a > FunPtr a I think it's currently Ptr a, FunPtr a, or Addr (see TysWiredIn.isFFILabelTy). > Should we make newtype completely transparent? I think so, yes. Cheers, Simon

1 0

Typing f.e.d.
by Sven Panne 30 Apr '01

30 Apr '01

I have a small change request regarding foreign export dynamic. Currently the FFI doc says: topdecl : ... .. | 'foreign' 'export' [callconv] 'dynamic' varid :: prim_type -> IO Addr GHC additionally allows: prim_type -> IO Ptr As usual the FFI "looks through" newtypes. But now that we have FunPtr, the following typing makes much more sense: 'foreign' 'export' [callconv] 'dynamic' varid :: prim_type -> IO (FunPtr prim_type) where both prim_types have to be the *same*. We should probably allow the old Addr-typing as well for some time to facilitate the transition, but not the Ptr-typing (bleeding edge people will know what to do :-). The corresponding changes to GHC look easy, so I'd like to commit this if there are no objections. Furthermore, the FFI docs still talk about Addr only, not Ptr/FunPtr. Now that Addr is deprecated, this should be changed, too. Cheers, Sven

5 7

RE: FFI progress
by Simon Marlow 30 Apr '01

30 Apr '01

> In this context, I actually find it the identifier `stdcall' > strange. Why is it standard? I'd argue that ccall is the > standard (in fact, it so far was the default, wasn't it?) It's known as 'stdcall' on Windows. In fact, VC++ has __stdcall keyword. All the Win32 API functions are declared __stdcall. So, I guess that's why we called it stdcall in Haskell. Cheers, Simon

2 1

RE: FFI progress
by Simon Peyton-Jones 17 Apr '01

17 Apr '01

Manuel | The FFI discussion seems to be completely stalled. Would | you, as our Tsar, like to summarise the state of play, and | re-invigorate it? There was a bit of discussion, which led, I think to a simplification of the library stuff. But we don't have a summary (even informal) of the current state of play, and that's beginning to be a problem here because we're about to implement the .NET FFI for GHC. Might I suggest http://research.microsoft.com/~simonpj/ffi.html as the basis for a state-of-play-summary. You can grab the source and modify it as you see fit. There's one particular issue we havn't discussed. For Java and .NET we want to call static methods, dynamic methods, and constructors. We already have foreign import static foreign import dynamic but 'new' is different again. The obvious place for it is in the language-specific string foreign import static "new foo(int x)" foo :: Int -> IO Foo It seems a bit odd to have static/dynamic *outside* but "new" inside the language-specific string. I suppose the justification is that 'new' is really a static method with a funny way to call it. Whereas the 'self' parameter on a dynamic call is treated specially. So that's ok, but we should agree that's what we want. Simon

3 2

QForeign-0.65 is out
by qrczak＠knm.org.pl 15 Apr '01

15 Apr '01

http://prdownloads.sourceforge.net/qforeign/qforeign-0.65.tar.gz qforeign-0.65 (Apr 15, 2001) ---------------------------- * Make libraries for GHCi too (HS*.o). * hsc2hs should work on Windows too (untested). * Works with nhc98-1.02 too. ghc-4.09 support dropped. * hsc2hs parser rewritten. #{stuff} added (like #stuff but self-delimited). Now correctly handles '--' in operators and backslash-newline pairs in C lexical world. * Use {-# LINE #-} pragmas for nhc98 too. * Zlib and Bzip2 express byte lists as [Word8] instead of String. * QCError.errnoToIOError uses strerror instead of its own error strings, and uses FFI.mkIOError under nhc98 if available. * Add destruct method to class Storable (thanks Wojciech Moczydlowski <khaliff(a)astercity.net> for the suggestion). * Add QCFile: conversion between Handle and FILE * (compiled when module Posix is available). * Support readline-2.2.1 as well as readline-4.x. * Support Unicode 3.1.0 in unidata. Characters up to 0x10FFFF. Better definitions of some character properties. More efficient GenUnicodeData. * README is more interesting. * License changed from GPL to LGPL. -- __("< Marcin Kowalczyk * qrczak(a)knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZASTÊPCZA QRCZAK

1 0

RE: FFI progress
by Simon Peyton-Jones 12 Apr '01

12 Apr '01

| In case you aren't aware of this, Windows DLL's by default, (actually | always, as far as I know) use the stdcall (Pascal) calling convention. | | I personally hope that you won't drop anything which facilitates using | Haskell under Windows. Dead right. I wasn't proposing removing the functionality, just changing how to get at it. More concretely, there are two alternatives 1. Treat C-via-stdcall and C-via-ccall as two different "languages". E.g. foreign import "C/stdcall" "gtk:foo" foo :: Int -> IO Int 2. Treat C as one language, but put the calling convention into the language specific string. E.g. foreign import "C" "stdcall/ gtk:foo" foo :: Int -> IO Int I prefer (1) but it's not a big deal. Simon

2 1

RE: FFI progress
by Simon Peyton-Jones 09 Apr '01

09 Apr '01

Is anyone actually specifying stdcall ccall calling conventions in a foreign import? I want to pull it out of GHC but I don't want to break anything. Simon

3 2

RE: FFI progress
by Simon Peyton-Jones 07 Apr '01

07 Apr '01

One other thing. What did we decide about the 'calling convention' field. I reckon it should vanish into the language-specific string, or alternatively treat C as two languages that differ in their calling convention. No way C calling conventions deserve special treatment. Simon

1 0

Re: FFI - some comments
by qrczak＠knm.org.pl 06 Apr '01

06 Apr '01

Thu, 5 Apr 2001 07:45:44 -0700 (PDT), Ronald Legere <rjljr2(a)yahoo.com> pisze: > KDirect: Looked at it abit. Pretty sparsely documented. I could not > figure out how it fits in with the latest tools. How does QForeign > relate to Foreign? What is an hsc file? Also it is pretty limitted > in the marshalling. No callbacks etc. Here is the story of QForeign, hsc and KDirect. One year ago the core 'foreign' declarations were already stable and implemented in ghc (for sure) and nhc (probably already at that time). The FFI documentation said that support for more complex marshalling is delegated to separate tools. Available tools, namely HDirect and Manuel's C2HS, were too heavy for my taste. But Sven's modules provided by ghc: FFI (later renamed to Foreign) and Marshal, were insufficient. I realized that a few more functions would make the basic FFI usable directly, without sophisticated preprocessors. I also wanted to experiment with typed 'Ptr a' instead of amorphic 'Addr'. So I merged ideas from ghc's module FFI and C2HS with my own FFI vision, and QForeign was born. QForeign evolved. The basic style of calling a C function while marshalling arguments and results changed a few times. It got examples. It suggested how to handle structs and enums. It became my Unicode laboratory. Since writing lots of small C functions for getting struct fields and constants was painful, I wrote glue-hsc: a simple preprocessor which translates a Haskell module with embedded C bits into a C program which outputs a real Haskell module. It allows embedding snippets of C code which are extracted into .c and .h files to be compiled separately. The source file extension .hsc means: .hs + bits of C included. QForeign was following changes in ghc and it also influenced ghc. ghc got typed pointers, alloca, withForeignObj, C array support, hsc2hs (renamed from glue-hsc), errno handling, C string support interface. As soon as some QForeign's feature got into ghc, QForeign switched to use it from ghc instead of providing it itself. If a feature mutated during its path to ghc, QForeign reflected the change. ghc's FFI support and my FFI vision were slowly converging. I had to write a networking program at the University, so of course I decided to use Haskell. That's why QForeign has pcap in examples. I asked my admin to install ghc in students' lab. The latest stable version was 4.08.1, so I #ifdefed parts of QForeign to support ghc-4.08* too by providing its own typed pointers and Storable again (no Unicode, no --add-package). Supporting several ghc versions at once happened to be easy (hsc2hs delegates #ifdefs to cpp). So I tried to port QForeign to nhc too and it worked, after some bugs that I reported were fixed. So I also thought about supporting hbc, which would require writing a preprocessor which translates 'foreign' declarations to appropriate hbc's magic, but hbc is dead, so I forgot it. Since Hugs recently got 'foreign' declarations, it might be possible to port QForeign to Hugs in future, but its foreign module management looks hard. QForeign and FFI modules provided by ghc-5.00 (to be released really soon) are now very close, after Manuel put some marshalling utilities into ghc and I backported them to QForeign as usual. The largest difference is that QForeign makes real use of Unicode (when the compiler has wide Chars) by converting most strings exchanged with the world between the specified encoding (default: locale dependent) and Haskell's Unicode, even though Unicode support modules are still experimental and transparent conversion on Handle IO is temporarily hacked on top of original Handle functions. In essence QForeign provides the interface of ghc-5.00's FFI on ghc >= 4.08 and nhc98. My friend Wojciech Moczydlowski <khaliff(a)astercity.net> asked me what is the currently recommended way of interfacing between Haskell and C. I showed him how do I do it. He said that Greencard and HDirect were easier: you didn't have to write marshalling code around each function, but all arguments were magically coverted basing on a single type declaration of the function. I replied that my solution is more general, especially if the Haskell's interface of a C module doesn't want to exactly mirror the original C interface, and that it's impossible to match everything automatically, and that it's not that painful to write function wrappers by hand. He said that a tool could handle simple cases automatically, leaving only hard types to the programmer. I said that I don't know how express a C interface to archieve this, and that C libraries are too complex to be wrapped in Haskell automatically, and thus I can't provide anything more sophisticated than hsc2hs. So he created KDirect. KDirect currently works on top of QForeign and hsc2hs. I assume that in future it will be able to use FFI modules provided by ghc >= 5.00, and that eventually all Haskell implementations will provide common FFI modules. -- __("< Marcin Kowalczyk * qrczak(a)knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZASTÊPCZA QRCZAK

2 1