- Haskell-Cafe - Haskell.org

RE: [Haskell-cafe] Suggestions for #defines with FFI
by Dimitry Golubovsky 28 Jul '05

28 Jul '05

John, Could you please give an example of such constant definitions? Is it possible to look at the include file? -- Dimitry Golubovsky Anywhere on the Web

4 5

hssfig: Unresolved symbols when importing a foreign library: what is the preferred way of handling?
by Dimitry Golubovsky 27 Jul '05

27 Jul '05

I would like to collect opinions from interested people on this matter. Background: Hsffig parses include files (.h) and generates FFI import declarations for every function/external variable it encounters a prototype declaration for. The result is a .hsc file which is converted into a Haskell source and compiled into an object file and linked into an executable along with other application source files using ghc as frontend (so far, hsffig is targeted to ghc users only). This object file will contain references to all foreign functions to be imported. Problem: In some cases, a include file may contain prototypes of functions not existing in the library it corresponds to. In my case, <unistd.h> (Linux, glibc2) contains prototypes for __ftruncate and pthread_atfork which come up unresolved when linking the executable without additional libraries specified. pthread_atfork is resolved when linking with -lpthread. Where is __ftruncate defined, I don't know, and I do not really want ot know unless I need to use this function. Possible solutions, sorted by expected amount of work necessary to implement (please pick one you would prefer, or suggest yours): - leave everything as is: the executable will not be linked until the developer finds all the libraries to resolve all the missing symbols. That's how it works (or doesn't work) now. - tell ld only to warn about unresolved symbols: the executable will be built anyway, but if the unresolved function is called from under the hood it most likely results in segfault hard to explain (which function caused it). Only special linker option is required. - for each foreign function imported, generate a stub which, when called, complains loudly and aborts the program. These stubs are placed in a separate object file and linked after all possible libraries, so library-defined symbols always take precedence. The disadvantage is: the developer would not know which symbols remained unresolved when the executable was linked, only will he know when the application crashes. I am trying to implement this now, using the hsc2hs' #def macro to define those stub functions. - analyze the linker output and make auto-correction in the generated .hsc file just by excluding FFI declarations for unresolved functions. Non-elegant, requires second pass and highly platform and tool dependent. - instead of generation of a single object file referencing all the functions, create one per funciton import and place them all in an archive (.a). Then, when linking, only actually used functions will be picked from the archive. I am not sure how would this interact with ghc --make: I got an impression that a .hi file is necessary for all modules (.hs/.hi/.o triple) imported unless they are gotten from a package. If I have in my program: import FOO_H and there is no FOO_H.hi nor FOO_H.hs, only libFOO_H.a: how ghc would live with this? The latter approach does not take much to implement in hsffig itself (I already have a splitter which splits into one object file per a structure/union). I am more concerned how to make ghc live with this. Creation of a package for each imported include file seems like overkill. PS I believe that hsffig will be an useful utility. I noticed several people downloaded nightly tarballs or checked out from the DARCS repo. If these persons have found this program useful, would they please suggest something on the matter I described. -- Dimitry Golubovsky Anywhere on the Web

1 0

RE: [Haskell-cafe] ForeignPtrs with liveness dependencies
by Simon Marlow 27 Jul '05

27 Jul '05

On 26 July 2005 16:49, Einar Karttunen wrote: > "Simon Marlow" <simonmar(a)microsoft.com> writes: >> No, unfortunately not. You have foo's finalizer which refers to bar >> via a touchForeignPtr. If both foo and bar are unreachable >> (references from finalizers don't count), then both foo and bar's >> finalizers will be started together, and may run in any order. > > I didn't realize the "references from finalizers don't count" rule. It does make sense. If references from finalizers *did* keep objects alive, then if you had two objects that referreed to each other by their finalizers, neither would ever be finalized. This paper has all the rationale, you might find it interesting: http://www.haskell.org/~simonmar/papers/weak.pdf Expressing ordering relationships between finalizers is known to be hard - for instance, if you have a chain of N dependent finalizers that are all unreachable, you don't want to take N major GCs to discover the fact. The canonical paper on this is Hans Boehm's "Destructors, finalizers, and synchronization": http://portal.acm.org/citation.cfm?id=604153 > What would happen if the finalizer of foo would resurrect bar after > bar's finalizer has been run? That can happen. Nothing goes wrong on the Haskell side, but if this is a problem for your C objects then you need to be aware of it. >> So touchForeignPtr does only one thing: it expresses the precise >> relationship "bar is alive if foo is alive". If both are not alive, >> then both finalizers can run, in any order. > > So reference counting the objects is the solution? Perhaps. It seems robust, anyway. >> I realise this is very subtle. By all means suggest improvements to >> the docs. > > Mentioning that references from finalizers don't count could help > someone not to repeat my mistakes. Ok, will do. Cheers, Simon

1 0

Newbie question about Read strings in a line of File
by Huong Nguyen 27 Jul '05

27 Jul '05

Hello everybody, Now I am writing a module to handles messages of a system. The messages are stored in a file. The content of each line of the file likes this: 1 001 001 Error message 1 1 001 002 Error message 2 1 002 001 Warning message 1 in which: The first word is the language Code (e.g: 1 is English, 2 is French), the second is type of messages (e.g: 001 is error message, 002 is warning message,etc), the third is the index number of messages, and the last is the message. I need to read 3 first words of each line and assign them to 3 variables. After that, I need to show the error message based on the tuple of those 3 first variable, e.g: errMsg("1","001","001") = "Error message 1" At first, I tried to read only one first line of the file as follows: import Prelude import List import Char import IO main = do hdl <- openFile "message.txt" ReadMode msgLine <-hGetLine hdl --stdMsg: cut space at the begining of the line. let stdMsg = dropWhile(not.isSpace) msgLine -Assign langCode with the first word of the line. let langCode = takeWhile(not.isSpace) stdMsg However, after that, I do not know how to read second element, third element of the line. And how to read continuous next line after first? Please help me. Thanks a lot.

2 1

RE: [Haskell-cafe] ForeignPtrs with liveness dependencies
by Simon Marlow 26 Jul '05

26 Jul '05

On 26 July 2005 14:15, Einar Karttunen wrote: > "Simon Marlow" <simonmar(a)microsoft.com> writes: >>> Now the association becomes >>> associate (Foo _ ref) bar = >>> atomicModifyIORef ref (\lst -> (touchForeignPtr bar : lst, ())) >> >> Isn't that equivalent to using addForeignPtrFinalizer? I don't think >> this fixes anything: the finalizer for bar can still run before the >> finalizer for foo. > > foo has a single finalizer which is defined like: > > fooFinalizer cfoo ref = do cdeleteFoo cfoo > vs <- readIORef ref > mapM_ (\c -> c) vs the last line is equivalent to "sequence_ vs", FWIW. > and foo is created like > > createFoo ptr = do ref <- newIORef [] > fp <- newForeigPtr ptr (fooFinalizer ptr ref) > return (Foo fp ref) > > As the finalizer of foo references the IORef which contains > the list of actions containing the "touchForeignPtr bar" > the finalizer of foo is run first. The finalizer to bar > should be able to run only when the touchForeignPtr has been > executed in the mapM_ which only happens after foo > has been cleaned up - if I understand things correctly. No, unfortunately not. You have foo's finalizer which refers to bar via a touchForeignPtr. If both foo and bar are unreachable (references from finalizers don't count), then both foo and bar's finalizers will be started together, and may run in any order. So touchForeignPtr does only one thing: it expresses the precise relationship "bar is alive if foo is alive". If both are not alive, then both finalizers can run, in any order. I realise this is very subtle. By all means suggest improvements to the docs. Cheers, Simon

2 1

Data-types in dynamically loaded hs-plugins
by Thomas Sutton 26 Jul '05

26 Jul '05

Hi all, I'm [trying to] develop a system that uses plug-ins to implement a "black-box" ADT, that is a data-type and a few typeclass instances for it. I want my system (a theorem prover) to be able to work with any of these plug-ins (which define a logic and operations on its formulae) but have been having trouble. I've had a go at using forall's, and at adding a type variable (constrained to the appropriate class) to the interface of the plug- in, but I haven't been able to get it to work and have run out of ideas. Is it possible to define data-types in a module loaded at run-time with hs-plugins and have values pass out of the plug-in (though only being used by functions within it)? Can someone point me in the right direction? Cheers, Thomas Sutton

1 0

RE: [Haskell-cafe] ForeignPtrs with liveness dependencies
by Simon Marlow 26 Jul '05

26 Jul '05

On 26 July 2005 11:26, Einar Karttunen wrote: > "Simon Marlow" <simonmar(a)microsoft.com> writes: >> You might be able to find more information on this in the mailing >> list archives. It's true that touchForeignPtr isn't enough to >> enforce an ordering on the running of finalizers, but it *can* be >> used to express a liveness relationship between one ForeignPtr and >> another (ForeignPtr A is alive if ForeignPtr B is alive). This >> should be enough if you're dealing with pointer relationships >> between memory objects, for example, where it doesn't matter which >> one gets freed first when they're both unreferenced. > > The order of the cleanup functions is significant in this case, > so that does not unfortunately help. > >> If you really do need ordering, maybe it would be possible to use >> reference counting in your case? > > I ended up using the following design, which seems to work fine: > data Foo = Foo (ForeignPtr Foo) (IORef [IO ()]) > Each ForeignPtr Foo has a single finalizer which first calls the > C-side cleanup function for Foo and then executes all the IO-actions > inside the IORef. > > Now the association becomes > associate (Foo _ ref) bar = > atomicModifyIORef ref (\lst -> (touchForeignPtr bar : lst, ())) Isn't that equivalent to using addForeignPtrFinalizer? I don't think this fixes anything: the finalizer for bar can still run before the finalizer for foo. Cheers, Simon

2 1

RE: [Haskell-cafe] FFI and callbacks
by Simon Marlow 26 Jul '05

26 Jul '05

On 26 July 2005 11:32, Sebastian Sylvan wrote: > On 7/26/05, Simon Marlow <simonmar(a)microsoft.com> wrote: >> On 25 July 2005 22:41, Sebastian Sylvan wrote: >>> Well how about not touching the "bound threads" design at all, but >>> modifying the lightweigh threads design to also include "forkIOHere" >>> (or something) which would produce a lightweight thread which is >>> always run in the "main" OS thread. >> >> This is what Duncan was angling for, I think. My point is that this >> requires the implementation to support lightweight threads, which is >> something we intentionally avoided in the bound threads design. > > No it doesn't. If they don't support lightweight threads, they won't > have the forkIOHere function available. > I.e. leave it out of the bound threads design and just implement it as > a feature in the GHC lightweight threads system. I think you misunderstand me. I'm merely pointing out that the fact that it would be a GHC-specific feature is a drawback. (not that any other compilers have bound threads either, but still). Cheers, Simon

1 0

RE: [Haskell-cafe] FFI and callbacks
by Simon Marlow 26 Jul '05

26 Jul '05

On 25 July 2005 22:41, Sebastian Sylvan wrote: > On 7/25/05, Simon Marlow <simonmar(a)microsoft.com> wrote: >> On 23 July 2005 03:38, Duncan Coutts wrote: >> >>> The problem then as John noted is that the main loop of these >>> toolkits block and so the other Haskell threads would not get a >>> chance to schedule. So the challenge is to give the Haskell threads >>> a chance to schedule. >> >> [ good description of the multi-threaded GUI problem deleted ] >> >> Thanks for describing the problem in detail, I understand it better >> now. I think it comes down to this conflict: >> >> - you want to take advantage of the fact that GHC has lightweight >> "green" threads in order to do multithreading within a single OS >> thread, but >> >> - our "bound threads" design does not require the implementation >> to support lightweight threads, and hence doesn't let the >> programmer take advantage of them. > > Well how about not touching the "bound threads" design at all, but > modifying the lightweigh threads design to also include "forkIOHere" > (or something) which would produce a lightweight thread which is > always run in the "main" OS thread. This is what Duncan was angling for, I think. My point is that this requires the implementation to support lightweight threads, which is something we intentionally avoided in the bound threads design. While I don't doubt that we could implement some way to fork another thread bound to the same OS thread as the parent, I'm worried that this would add yet more complexity to GHC's scheduler, so I think we should fully explore other possibilities first. Cheers, Simon

2 1

RE: [Haskell-cafe] ForeignPtrs with liveness dependencies
by Simon Marlow 26 Jul '05

26 Jul '05

On 25 July 2005 14:44, Einar Karttunen wrote: > What is the correct way to express liveness dependencies for > ForeignPtrs? I am wrapping a C library and need a way to keep > ForeignPtrs alive until the finalizer for an another ForeignPtr > has been executed. > > Basically I have two types, ForeignPtr A and ForeignPtr B and a > function associate :: ForeignPtr A -> ForeignPtr B -> IO (). I want > to keep all of the ForeignPtr Bs associated with a given ForeignPtr A > alive until its finalizer has been run. The relationship is M:N - > each ForeignPtr A may be associated with multiple ForeignPtr B and > each ForeignPtr B may be associated with multiple ForeignPtr A. > > GHC documentation tells that touchForeignPtr is not enough as it makes > no guarantees about when the finalizers are run. If it helps the > finalizers are C functions which neither block nor perform callbacks > into Haskell. You might be able to find more information on this in the mailing list archives. It's true that touchForeignPtr isn't enough to enforce an ordering on the running of finalizers, but it *can* be used to express a liveness relationship between one ForeignPtr and another (ForeignPtr A is alive if ForeignPtr B is alive). This should be enough if you're dealing with pointer relationships between memory objects, for example, where it doesn't matter which one gets freed first when they're both unreferenced. If you really do need ordering, maybe it would be possible to use reference counting in your case? Cheers, Simon

2 1