FFI, C/C++ and undefined references
Note: I'm relatively new to Haskell, and my knowledge of C and C++ is basically pretty minimal -- I can read, modify and compile C/C++ programs (usually). I'm trying to interface with some C++ code by writing a little bit of C code that uses that C++ code, and I'm getting "undefined reference" errors when I try to 'ghc --make' a client application to test it. Actually, I'm modifying Nitin Madnani's (freely available) Python SRILM toolkit wrapper code. (SRILM, by the bye, is a C++-based toolkit for training and using statistical n-gram language models. I was surprised that no-one has tried to do this yet -- or at least not that they have shared with the rest of us.) Anyhow, I've verified that my modification of Madnani's C code works by compiling it and running it through a SWIG interface in Madnani's Python code, so I'm pretty confident the C client of SRILM is solid. The culprit is either my Haskell FFI code or the client of that code. Without cooking up a microcosm of my problem with little Foo's and Bar's, I'll just give my actual C, header file and Haskell code (or at least the relevant bits), and then the error. ------------- srilm.h ---------------- #ifdef __cplusplus extern "C" { #else typedef struct Ngram Ngram; /* dummy type to stand in for class */ #endif Ngram* bldLM(int order, const char* filename); void deleteLM(Ngram* ngram); float getSeqProb(Ngram* ngram, const char* ngramstr, unsigned order, unsigned length); #ifdef __cplusplus } #endif ----------------------------------------- ------------- srilm.c ---------------- // Initialize and read in the ngram model Ngram* bldLM(int order, const char* filename) { ... } ... // Delete the ngram model void deleteLM(Ngram* ngram) { delete srilm_vocab; delete ngram; } ... // Get the ngram probability of the given string, given n-gram order 'order' and string length // 'length'. float getSeqProb(Ngram* ngram, const char* ngramstr, unsigned order, unsigned length) { ...} ----------------------------- Next, the Haskell FFI specs and code that marshals data between Haskell and C. ---------------- LM.hs ---------------------- {-# INCLUDE "srilm.h" #-} {-# LANGUAGE ForeignFunctionInterface, EmptyDataDecls #-} ... module decl's, imports, etc. {- | A dummy placeholder for SRILM n-gram model thingies. -} data Ngram data NGModel = NGModel {ng :: !(ForeignPtr Ngram)} foreign import ccall "srilm.h bldLM" c_blm :: CInt -> CString -> Ptr Ngram foreign import ccall "srilm.h deleteLM" c_dlm :: FunPtr ((Ptr Ngram) -> IO ()) foreign import ccall "srilm.h getSeqProb" c_ngramProb :: Ptr Ngram -> CString -> CUInt -> CUInt -> CFloat {- | Given an n-gram model, an Int representing the n-gram order and a list of strings (word sequence), compute the n-gram probability of the sequence. -} scoreSequence :: NGModel -> Int -> [String] -> Float scoreSequence ngram order seq = unsafePerformIO $ do stringSeq <- newCString (unwords seq) let sc = c_ngramProb (unsafeForeignPtrToPtr $ ng ngram) stringSeq (fromIntegral order) (fromIntegral $ length seq) return (realToFrac sc) ... buildLM :: Int -> String -> NGModel buildLM order fname = NGModel $ unsafePerformIO $ do cFName <- newCString fname let ng = c_blm (fromIntegral order) cFName return $ unsafePerformIO $ newForeignPtr c_dlm ng -------------------------------------------- Now, I've defined a simple app that tries to use this: ------------------- Main.hs ------------------------- module Main where import SRILM.LM(scoreSequence, buildLM) main :: IO () main = do let lm = buildLM 5 "eng.kn.5g.lm" putStrLn $ show $ scoreSequence lm 5 ["the", "prime", "minister", "gave", "a", "speech", "."] ----------------------------------------------------------- But when I try to compile it (after having successfully compiled the C code with g++), I get: $ ghc --make Main.hs Linking Main ... LM.o: In function `r18k_info': (.text+0x122): undefined reference to `bldLM' LM.o: In function `r18m_info': (.text+0x14e): undefined reference to `deleteLM' LM.o: In function `r18o_info': (.text+0x28b): undefined reference to `getSeqProb' collect2: ld returned 1 exit status Any ideas? Note that I'm not confident that everything on the Haskell side is correct, but it seems that ghc can't find my C client of SRILM. As I said, I've compiled this code using g++, and it works when I interface with it through Python. Sorry for the long-windedness, but I figured I'd err on the side of TMI so that I don't have to keep posting more and more code snippets and error messages. Any help is greatly appreciated. (And I'd be happy to share my interface to SRILM to anyone who's interested, once I get it working -- and I get permission from Nitin Madnani to distribute a modified version of his code.) Thanks, Dennis -- View this message in context: http://old.nabble.com/FFI%2C-C-C%2B%2B-and-undefined-references-tp27139612p2... Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.
On Jan 13, 2010, at 00:57 , DNM wrote:
------------- srilm.c ---------------- // Initialize and read in the ngram model Ngram* bldLM(int order, const char* filename) { ... } ... // Delete the ngram model void deleteLM(Ngram* ngram) { delete srilm_vocab; delete ngram; } ... // Get the ngram probability of the given string, given n-gram order 'order' and string length // 'length'. float getSeqProb(Ngram* ngram, const char* ngramstr, unsigned order, unsigned length) { ...} -----------------------------
I think you need to `#include "srilm.h"' in the above, so C++ knows to export the functions with names callable from outside of C++. When you define a function in C++, the actual function symbol defined contains parameter type information unless previously declared in an extern "C" declaration.) While you've named the file with a .c extension, you have used C++-specific content (// comments, "delete" keyword) so I expect the file was compiled in C++ mode; otherwise it should have produced a syntax error from "delete" ("//" comments are a common enough extension that by themselves they probably work in much C code) and as a result the function symbols are mangled. If you add the #include, you bring the extern "C" declarations in scope, and C++ should produce non-mangled function definitions, which should be callable from Haskell. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH
Sorry. In my haste to paste in the .c file, I left out all the include statements. I do have #include "srilm.h" there (which to my non- C/C++ mind seems stupid -- why the hell would you need to import the header file for the code that it's a header *for*?) Still no dice. Thanks for your time, though. Sorry to waste it. --D.N. -- View this message in context: http://old.nabble.com/FFI%2C-C-C%2B%2B-and-undefined-references-tp27139612p2... Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.
On Jan 13, 2010, at 23:28 , DNM wrote:
Sorry. In my haste to paste in the .c file, I left out all the include statements. I do have #include "srilm.h" there (which to my non- C/C++ mind seems stupid -- why the hell would you need to import the header file for the code that it's a header *for*?)
Really, the only reason in this case is that there is no equivalent for `extern "C"' that you can apply to a function definition, only to a declaration. The rationale is that everything that works with the function, including its definition, needs to see that declaration, so rather than repeat it in the definition you #include the declaration. In GHC, this is the kind of thing that lands in the .hi file; the tradeoff is you need to have up to date .hi files for everything that needs to see that information, which can lead to dependency loops. GHC has a ".hs-boot" hack to work around this. No free lunch.... -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH
Hello Brandon, Thursday, January 14, 2010, 7:40:45 AM, you wrote:
Really, the only reason in this case is that there is no equivalent for `extern "C"' that you can apply to a function definition, only to a declaration
it works with GCC: extern "C" int c_szOpenArchive (TABI_ELEMENT* params) { .... } -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
Malcolm, I saw this suggestion somewhere else. Unfortunately, it didn't help either. I still get the "undefined reference" errors. I did eventually get ghc to compile Main.hs by putting the -c and -cpp flags after "--make Main.hs". Then it produces a Main.o file which (even with +x permissions on my Linux box) will not run. I just get the message "cannot run binary file", or some such message. No explanation given. Any ideas? Best, Dennis Malcolm Wallace wrote:
But when I try to compile it (after having successfully compiled the C code with g++), I get:
$ ghc --make Main.hs
You are not telling ghc to link against the C/C++ code, e.g. ghc --make Main.hs srilm.o
Regards, Malcolm
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
-- View this message in context: http://old.nabble.com/FFI%2C-C-C%2B%2B-and-undefined-references-tp27139612p2... Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.
I still get the "undefined reference" errors.
It is likely there is some combination of other mistakes as well then. Other responses have made suggestions of fixes you require in the C++ code for instance. You will need those as well.
I did eventually get ghc to compile Main.hs by putting the -c and -cpp flags after "--make Main.hs".
Then it produces a Main.o file which (even with +x permissions on my Linux box) will not run. I just get the message "cannot run binary file", or some such message.
The file Main.o is just an object file, not a complete executable. It still needs to be linked against some other (Haskell or C/C++) object files and libraries, and the Haskell runtime system, to form an executable that can be run. ghc is capable of doing all the linking, e.g. ghc -o myProg Main.o slirm.o -package base -package foo However, if you are unsure of which Haskell packages are needed, it is wise to let ghc work out the dependencies for you, e.g. with ghc --make Main.hs slirm.o It cannot work out the C/C++ dependencies though, so every time you get "undefined reference" linking errors, you must discover which C code provides those symbols, and add its object file to the commandline by hand. Regards, Malcolm
Which is weird, because 'srilm.o'/'srilm.h' are the files that define the mysterious "undefined references". I'll keep plugging away and report back when (or whether) I make some progress. In the meanwhile, if anyone has a clue, I'm all ears. Best, D.N. Malcolm Wallace wrote:
However, if you are unsure of which Haskell packages are needed, it is wise to let ghc work out the dependencies for you, e.g. with ghc --make Main.hs slirm.o
It cannot work out the C/C++ dependencies though, so every time you get "undefined reference" linking errors, you must discover which C code provides those symbols, and add its object file to the commandline by hand.
-- View this message in context: http://old.nabble.com/FFI%2C-C-C%2B%2B-and-undefined-references-tp27139612p2... Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.
Am Donnerstag 14 Januar 2010 20:42:42 schrieb DNM:
Which is weird, because 'srilm.o'/'srilm.h' are the files that define the mysterious "undefined references". I'll keep plugging away and report back when (or whether) I make some progress. In the meanwhile, if anyone has a clue, I'm all ears.
Best, D.N.
Just an idea. Are you on windows? If so, then your foreign calls would probably have to be foreign import stdcall "srilm.h whatever" ... instead of foreign import ccall "..."
Malcolm Wallace wrote:
However, if you are unsure of which Haskell packages are needed, it is wise to let ghc work out the dependencies for you, e.g. with ghc --make Main.hs slirm.o
It cannot work out the C/C++ dependencies though, so every time you get "undefined reference" linking errors, you must discover which C code provides those symbols, and add its object file to the commandline by hand.
On Thu, Jan 14, 2010 at 2:08 PM, Daniel Fischer
Just an idea. Are you on windows? If so, then your foreign calls would probably have to be
foreign import stdcall "srilm.h whatever" ...
instead of
foreign import ccall "..."
Yes, I came here to say that too. I was getting those errors on Windows. Paulo
Hello Daniel
On Windows, isn't stdcall vs ccall still dependent on the actual
library and what compiled it - commonly MSVC (stdcall) or gcc (ccall)
of course?
I could very easily be wrong...
Best wishes
Stephen
2010/1/14 Daniel Fischer
Am Donnerstag 14 Januar 2010 20:42:42 schrieb DNM:
Just an idea. Are you on windows? If so, then your foreign calls would probably have to be
foreign import stdcall "srilm.h whatever" ...
instead of
foreign import ccall "..."
Hello Daniel, Thursday, January 14, 2010, 11:08:24 PM, you wrote: i think you are wrong. stdcall used for std windows dlls, but gcc by default generates ccall things. and cl anyway useless here
Just an idea. Are you on windows? If so, then your foreign calls would probably have to be
foreign import stdcall "srilm.h whatever" ...
instead of
foreign import ccall "..."
Malcolm Wallace wrote:
However, if you are unsure of which Haskell packages are needed, it is wise to let ghc work out the dependencies for you, e.g. with ghc --make Main.hs slirm.o
It cannot work out the C/C++ dependencies though, so every time you get "undefined reference" linking errors, you must discover which C code provides those symbols, and add its object file to the commandline by hand.
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
-- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
Nope. Ubuntu Linux (Intrepid Ibex). I wish it were that simple. --D.N. Daniel Fischer-4 wrote:
Am Donnerstag 14 Januar 2010 20:42:42 schrieb DNM:
Which is weird, because 'srilm.o'/'srilm.h' are the files that define the mysterious "undefined references". I'll keep plugging away and report back when (or whether) I make some progress. In the meanwhile, if anyone has a clue, I'm all ears.
Best, D.N.
Just an idea. Are you on windows? If so, then your foreign calls would probably have to be
foreign import stdcall "srilm.h whatever" ...
instead of
foreign import ccall "..."
Malcolm Wallace wrote:
However, if you are unsure of which Haskell packages are needed, it is wise to let ghc work out the dependencies for you, e.g. with ghc --make Main.hs slirm.o
It cannot work out the C/C++ dependencies though, so every time you get "undefined reference" linking errors, you must discover which C code provides those symbols, and add its object file to the commandline by hand.
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
-- View this message in context: http://old.nabble.com/FFI%2C-C-C%2B%2B-and-undefined-references-tp27139612p2... Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.
Works fine here (Mac OS X 10.5): MigMit:ngram MigMit$ ghc --make Main.hs srilm.o [1 of 2] Compiling LM ( LM.hs, LM.o ) LM.hs:9:0: Warning: possible missing & in foreign import of FunPtr [2 of 2] Compiling Main ( Main.hs, Main.o ) Linking Main ... MigMit:ngram MigMit$ ls Main* Main* Main.hi Main.hs Main.hs~ Main.o MigMit:ngram MigMit$ cat Main.hs module Main where import LM(scoreSequence, buildLM) main :: IO () main = do let lm = buildLM 5 "eng.kn.5g.lm" putStrLn $ show $ scoreSequence lm 5 ["the", "prime", "minister", "gave","a", "speech", "."] MigMit:ngram MigMit$ cat LM.hs {-# INCLUDE "srilm.h" #-} {-# LANGUAGE ForeignFunctionInterface, EmptyDataDecls #-} module LM where import Foreign import Foreign.C data Ngram data NGModel = NGModel {ng :: !(ForeignPtr Ngram)} foreign import ccall "srilm.h bldLM" c_blm :: CInt -> CString -> Ptr Ngram foreign import ccall "srilm.h deleteLM" c_dlm :: FunPtr ((Ptr Ngram) -
IO ()) foreign import ccall "srilm.h getSeqProb" c_ngramProb :: Ptr Ngram -> CString -> CUInt -> CUInt -> CFloat scoreSequence :: NGModel -> Int -> [String] -> Float scoreSequence ngram order seq = unsafePerformIO $ do stringSeq <- newCString (unwords seq) let sc = c_ngramProb (unsafeForeignPtrToPtr $ ng ngram) stringSeq (fromIntegral order) (fromIntegral $ length seq) return (realToFrac sc) buildLM :: Int -> String -> NGModel buildLM order fname = NGModel $ unsafePerformIO $ do cFName <- newCString fname let ng = c_blm (fromIntegral order) cFName return $ unsafePerformIO $ newForeignPtr c_dlm ng MigMit:ngram MigMit$ cat srilm.h #ifdef __cplusplus extern "C" { class Ngram{}; #else typedef struct Ngram Ngram; #endif Ngram* bldLM(int order, const char* filename); void deleteLM(Ngram* ngram); float getSeqProb(Ngram* ngram, const char* ngramstr, unsigned order, unsigned length); #ifdef __cplusplus } #endif MigMit:ngram MigMit$ cat srilm.c #include "srilm.h" Ngram* bldLM(int order, const char* filename) { return 0; } void deleteLM(Ngram* ngram) {} float getSeqProb(Ngram* ngram, const char* ngramstr, unsigned order, unsigned length) { return 0;} MigMit:ngram MigMit$
Maybe you just need to recompile srilm.c or something. On 14 Jan 2010, at 23:39, DNM wrote:
Nope. Ubuntu Linux (Intrepid Ibex). I wish it were that simple.
--D.N.
Daniel Fischer-4 wrote:
Am Donnerstag 14 Januar 2010 20:42:42 schrieb DNM:
Which is weird, because 'srilm.o'/'srilm.h' are the files that define the mysterious "undefined references". I'll keep plugging away and report back when (or whether) I make some progress. In the meanwhile, if anyone has a clue, I'm all ears.
Best, D.N.
Just an idea. Are you on windows? If so, then your foreign calls would probably have to be
foreign import stdcall "srilm.h whatever" ...
instead of
foreign import ccall "..."
Malcolm Wallace wrote:
However, if you are unsure of which Haskell packages are needed, it is wise to let ghc work out the dependencies for you, e.g. with ghc --make Main.hs slirm.o
It cannot work out the C/C++ dependencies though, so every time you get "undefined reference" linking errors, you must discover which C code provides those symbols, and add its object file to the commandline by hand.
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
-- View this message in context: http://old.nabble.com/FFI%2C-C-C%2B%2B-and-undefined-references-tp27139612p2... Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Am Donnerstag 14 Januar 2010 22:19:08 schrieb Miguel Mitrofanov:
Works fine here (Mac OS X 10.5):
MigMit:ngram MigMit$ ghc --make Main.hs srilm.o [1 of 2] Compiling LM ( LM.hs, LM.o )
LM.hs:9:0: Warning: possible missing & in foreign import of FunPtr [2 of 2] Compiling Main ( Main.hs, Main.o ) Linking Main ...
Thanks Miguel. Yes, works here (openSuse 11.1), too (kind of): (move the typedef out of the #else clause in srilm.h, because my g++ doesn't know Ngram) $ g++ -c srilm.c $ ghc --make Main.hs srilm.o [1 of 2] Compiling LM ( LM.hs, LM.o ) LM.hs:1:11: Warning: -#include is deprecated: No longer has any effect LM.hs:13:0: Warning: possible missing & in foreign import of FunPtr [2 of 2] Compiling Main ( Main.hs, Main.o ) Linking Main ... $ ./Main 0.0 Speicherzugriffsfehler Fixing the two warnings (removing the {-# INCLUDE #-} pragma and changing the declaration of deleteLM to foreign import ccall "srilm.h &deleteLM" c_dlm :: FunPtr ((Ptr Ngram) -> IO ()) ), I get $ ghc -fforce-recomp --make Main.hs srilm.o [1 of 2] Compiling LM ( LM.hs, LM.o ) [2 of 2] Compiling Main ( Main.hs, Main.o ) Linking Main ... $ ./Main 0.0 $
Hello DNM, Thursday, January 14, 2010, 10:42:42 PM, you wrote: there is better way rather than playing with random bits. just find tutorial on FFI, and try it. once this example works, start modifying it to learn various aspects of ffi and add functionality you need it's one thing i've learned in those 20 years - go forward in small steps keeping working code instead of jumping at large distance and then spending days without any clue
Which is weird, because 'srilm.o'/'srilm.h' are the files that define the mysterious "undefined references". I'll keep plugging away and report back when (or whether) I make some progress. In the meanwhile, if anyone has a clue, I'm all ears.
Best, D.N.
Malcolm Wallace wrote:
However, if you are unsure of which Haskell packages are needed, it is wise to let ghc work out the dependencies for you, e.g. with ghc --make Main.hs slirm.o
It cannot work out the C/C++ dependencies though, so every time you get "undefined reference" linking errors, you must discover which C code provides those symbols, and add its object file to the commandline by hand.
-- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
2010/1/14 Bulat Ziganshin
there is better way rather than playing with random bits. just find tutorial on FFI, and try it. once this example works, start modifying it to learn various aspects of ffi and add functionality you need
Also binding to a C library is easier than binding to a C++ one, if you can think of another library rather than SRILM that will meet your needs... Best wishes Stephen
Stephen Tetley wrote:
Also binding to a C library is easier than binding to a C++ one, if you can think of another library rather than SRILM that will meet your needs...
Alas, SRILM really is the standard tool for this so there aren't other (worthwhile) options AFAIK. But it's pretty standard for people to bind to SRILM as though it were C, since C bindings are standardized and C++ is a nightmare. I haven't done the gory details myself but I sat next to someone who did. -- Live well, ~wren
Hello DNM, Wednesday, January 13, 2010, 8:57:45 AM, you wrote:
Note: I'm relatively new to Haskell, and my knowledge of C and C++ is basically pretty minimal -- I can read, modify and compile C/C++ programs (usually).
1. you use too much unsafePerformIO. since you need newCString, i suggest you to declare C functions as returning IO a so your code will be unsafePerformIO$ do withCString str $ \c_str -> do c_function c_str ... 2. if your function returns Ptr a - then hold in haskell types this Ptr a. no need to convert it back and forth to ForeignPtr 3. why c_dlm is FunPtr in your definition? it should be foreign import ccall "srilm.h deleteLM" c_dlm :: Ptr Ngram -> IO () 4. i don't looked in your code but if C functions defines *modifiable* datastructure - you should use it at Haskell side via imperatiove functions, i.e. those with return type IO a. using unsafePerformIO in this case will lead to Haskell compiler will consider this datatype as permanent and reorder operations on the will so,
data NGModel = NGModel {ng :: !(Ptr Ngram)}
foreign import ccall "srilm.h bldLM" c_blm :: CInt -> CString -> IO (Ptr Ngram)
foreign import ccall "srilm.h deleteLM" c_dlm :: Ptr Ngram -> IO ()
foreign import ccall "srilm.h getSeqProb" c_ngramProb :: Ptr Ngram -> CString -> CUInt -> CUInt -> IO CFloat
scoreSequence :: NGModel -> Int -> [String] -> IO Float scoreSequence ngram order seq = do withCString (unwords seq) $ \stringSeq -> do sc <- c_ngramProb (ng ngram) stringSeq (fromIntegral order) (fromIntegral $ length seq) return (realToFrac sc)
and so on -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
2. if your function returns Ptr a - then hold in haskell types this Ptr a. no need to convert it back and forth to ForeignPtr Yes, I thought of doing this, but then thought it was better to use a so-called "managed" foreign pointer via newForeignPtr. I thought this was the best way to have a foreign pointer that the Haskell garbage collector would eat up when it was no longer in use. I could be wrong. I convert from a ForeignPtr to a Ptr, because the FFI code wasn't compiling at all (nevermind the missing C/C++ reference
Bulat, Some very good suggestions. I will try to appease Ceiling Cat and reduce my (perhaps gratuitous) use of unsafePerformIO. I'm going to have to use it somewhere, since I want referentially transparent code (and I try to avoid the IO monad when possible, anyway). problem), as apparently a ForeignPtr isn't the sort of thing that an imported foreign function can take as an argument (or so said GHC). I just assumed that the back-and-forth between Ptr and ForeignPtr would be compiled away by GHC. I could be wrong, though. If performance starts to suffer, I'll manage the Ptr memory in my code directly.
4. i don't looked in your code but if C functions defines *modifiable* datastructure - you should use it at Haskell side via imperatiove functions, i.e. those with return type IO a. using unsafePerformIO in this case will lead to Haskell compiler will consider this datatype as permanent and reorder operations on the will
Good point. I would do this if I planned to train or update the language model from within Haskell, but, as it stands, I just want to train it once (at the command line, using the built-in mechanisms of SRILM) and then read in the ARPA-formatted language model file for use in Haskell.
3. why c_dlm is FunPtr in your definition? it should be
foreign import ccall "srilm.h deleteLM" c_dlm :: Ptr Ngram -> IO () No reason. Just because I don't know what I'm doing yet. Thanks for the correction.
Thanks for the help, Bulat. Much appreciated. Best, Dennis -- View this message in context: http://old.nabble.com/FFI%2C-C-C%2B%2B-and-undefined-references-tp27139612p2... Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.
Hello DNM, Thursday, January 14, 2010, 7:07:43 AM, you wrote:
Yes, I thought of doing this, but then thought it was better to use a so-called "managed" foreign pointer via newForeignPtr.
i recommend to use Ptr and switch to ForeignPtr only when you will study how to use it. overall, unsafe* functions are really unsafe, and using them without learning will lead to mysterious problems. it's like painting with eyes closed
Good point. I would do this if I planned to train or update the language model from within Haskell, but, as it stands, I just want to train it once (at the command line, using the built-in mechanisms of SRILM) and then read in the ARPA-formatted language model file for use in Haskell.
you may add problems by using unsafePerformIO. i recommend you to learn first how to manage FFI without it, make program work, and only then try to use it. eat elephant in small pieces! look into http://haskell.org/haskellwiki/IO_inside -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
OK. Before anyone expends any e-ink replying to my reply below -- the one where I demonstrate that I don't understand what -c, -cpp mean to 'ghc' (not that you can blame me, since there isn't any documentation in the 'ghc' man page) -- I see why the Main.o file doesn't run. It's an object file, not an executable (not being from the C/C++ world, being a distinction I did not have at the forefront of my mind). Anyhow, still no dice. Even when cleaning up my Haskell code, I can't get this to compile. --D.N. DNM wrote:
Note: I'm relatively new to Haskell, and my knowledge of C and C++ is basically pretty minimal -- I can read, modify and compile C/C++ programs (usually).
I'm trying to interface with some C++ code by writing a little bit of C code that uses that C++ code, and I'm getting "undefined reference" errors when I try to 'ghc --make' a client application to test it.
Actually, I'm modifying Nitin Madnani's (freely available) Python SRILM toolkit wrapper code. (SRILM, by the bye, is a C++-based toolkit for training and using statistical n-gram language models. I was surprised that no-one has tried to do this yet -- or at least not that they have shared with the rest of us.) Anyhow, I've verified that my modification of Madnani's C code works by compiling it and running it through a SWIG interface in Madnani's Python code, so I'm pretty confident the C client of SRILM is solid. The culprit is either my Haskell FFI code or the client of that code.
Without cooking up a microcosm of my problem with little Foo's and Bar's, I'll just give my actual C, header file and Haskell code (or at least the relevant bits), and then the error.
------------- srilm.h ---------------- #ifdef __cplusplus extern "C" { #else typedef struct Ngram Ngram; /* dummy type to stand in for class */ #endif
Ngram* bldLM(int order, const char* filename); void deleteLM(Ngram* ngram); float getSeqProb(Ngram* ngram, const char* ngramstr, unsigned order, unsigned length);
#ifdef __cplusplus } #endif -----------------------------------------
------------- srilm.c ---------------- // Initialize and read in the ngram model Ngram* bldLM(int order, const char* filename) { ... } ... // Delete the ngram model void deleteLM(Ngram* ngram) { delete srilm_vocab; delete ngram; } ... // Get the ngram probability of the given string, given n-gram order 'order' and string length // 'length'. float getSeqProb(Ngram* ngram, const char* ngramstr, unsigned order, unsigned length) { ...} -----------------------------
Next, the Haskell FFI specs and code that marshals data between Haskell and C.
---------------- LM.hs ---------------------- {-# INCLUDE "srilm.h" #-} {-# LANGUAGE ForeignFunctionInterface, EmptyDataDecls #-} ... module decl's, imports, etc. {- | A dummy placeholder for SRILM n-gram model thingies. -} data Ngram
data NGModel = NGModel {ng :: !(ForeignPtr Ngram)}
foreign import ccall "srilm.h bldLM" c_blm :: CInt -> CString -> Ptr Ngram
foreign import ccall "srilm.h deleteLM" c_dlm :: FunPtr ((Ptr Ngram) -> IO ())
foreign import ccall "srilm.h getSeqProb" c_ngramProb :: Ptr Ngram -> CString -> CUInt -> CUInt -> CFloat
{- | Given an n-gram model, an Int representing the n-gram order and a list of strings (word sequence), compute the n-gram probability of the sequence. -} scoreSequence :: NGModel -> Int -> [String] -> Float scoreSequence ngram order seq = unsafePerformIO $ do stringSeq <- newCString (unwords seq) let sc = c_ngramProb (unsafeForeignPtrToPtr $ ng ngram) stringSeq (fromIntegral order) (fromIntegral $ length seq) return (realToFrac sc) ... buildLM :: Int -> String -> NGModel buildLM order fname = NGModel $ unsafePerformIO $ do cFName <- newCString fname let ng = c_blm (fromIntegral order) cFName return $ unsafePerformIO $ newForeignPtr c_dlm ng --------------------------------------------
Now, I've defined a simple app that tries to use this:
------------------- Main.hs ------------------------- module Main where import SRILM.LM(scoreSequence, buildLM)
main :: IO () main = do let lm = buildLM 5 "eng.kn.5g.lm" putStrLn $ show $ scoreSequence lm 5 ["the", "prime", "minister", "gave", "a", "speech", "."] -----------------------------------------------------------
But when I try to compile it (after having successfully compiled the C code with g++), I get:
$ ghc --make Main.hs Linking Main ... LM.o: In function `r18k_info': (.text+0x122): undefined reference to `bldLM' LM.o: In function `r18m_info': (.text+0x14e): undefined reference to `deleteLM' LM.o: In function `r18o_info': (.text+0x28b): undefined reference to `getSeqProb' collect2: ld returned 1 exit status
Any ideas?
Note that I'm not confident that everything on the Haskell side is correct, but it seems that ghc can't find my C client of SRILM. As I said, I've compiled this code using g++, and it works when I interface with it through Python.
Sorry for the long-windedness, but I figured I'd err on the side of TMI so that I don't have to keep posting more and more code snippets and error messages. Any help is greatly appreciated. (And I'd be happy to share my interface to SRILM to anyone who's interested, once I get it working -- and I get permission from Nitin Madnani to distribute a modified version of his code.)
Thanks, Dennis
-- View this message in context: http://old.nabble.com/FFI%2C-C-C%2B%2B-and-undefined-references-tp27139612p2... Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.
Hi all, I got it to work...finally. Basically, I used Malcolm's suggestion of tracking down all the SRILM .o files needed. I need to run now, but I'll post the gory (oh, so gory) details soon. Thanks to all who helped. Best, D.N. -- View this message in context: http://old.nabble.com/FFI%2C-C-C%2B%2B-and-undefined-references-tp27139612p2... Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.
participants (9)
-
Brandon S. Allbery KF8NH -
Bulat Ziganshin -
Daniel Fischer -
DNM -
Malcolm Wallace -
Miguel Mitrofanov -
Paulo Tanimoto -
Stephen Tetley -
wren ng thornton