August 2001 - Glasgow-haskell-users

Missing functions in GHC's RTS?
by Ashley Yakeley 03 Aug '01

03 Aug '01

At 2001-08-03 00:41, I wrote (on the Haskell Cafe list): >When I make a function generator with "foreign export dynamic", GHC spits >out stub files which it apparently then needs for the link. What are >these for? OK, so I don't know what they are, but the link needs the _stub.o file. Unfortunately, the stub file includes these symbols: rts_getWord8 rts_getWord16 rts_getInt16 rts_getInt64 ...which do not appear to be in any library and indeed are not declared in RtsAPI.h, although corresponding rts_mk* functions are. This looks like a bug in GHC. $ ghc -v Glasgow Haskell Compiler, Version 5.00, for Haskell 98, compiled by GHC version 5.00 Using package config file: /usr/lib/ghc-5.00/package.conf Hsc static flags: -static -fignore-interface-pragmas -fomit-interface-pragmas -fdo-lambda-eta-expansion -flet-no-escape $ uname -a Linux server 2.2.19pre17 #1 Tue Mar 13 22:37:59 EST 2001 i686 unknown -- Ashley Yakeley, Seattle WA

2 1

RE: adding isWHNF primop to 5.00.2 native code generator
by Bernard James POPE 03 Aug '01

03 Aug '01

Hi all, Hack alert! I took Julian's advice and looked over the assembly output from ghc. In all the cases I tried, the info pointer in the closure header pointed to the end of the info table. Although a comment in ghc/includes/ClosureMacros.h worries me: "info pointer" The first word of the closure. Might point to either the end or the beginning of the info table, depending on whether we're using the mini interpretter or not. GET_INFO(c) retrieves the info pointer of a closure. Can someone please clarify when the info pointer will point to the beginning or end of the table? Basically I have assumed for the moment that it points to the end of the table, as that is what some other code in ghc seems to assume. This makes it easy (hack?) to find the "type" field of the info table, its just the bottom (or top, depending on which way you look at it) 16 bits prior to the address that the info pointer contains. Essentially dataToTag# gives you these bits. I don't think "dataToTag#" is a great name for this primop, something like: closureType#, or classifyClosure# might be better. My current strategy is to simply compare the type field with the constants from ClosureTypes.h, using the Haskellised names re-exported from SMRep.lhs as suggested by Simon Marlow. My problem is that this is a dodgy assumption since the design of the info table may change in the future, and you get a nasty unchecked dependency between the primops in the NCG and the design of the rts. It seems hard to get the kind of automatic offset calculation in the NCG as you get with the C version of primops. My current idea is to make some C primops that do the structure and array indexing for me, and then use them in the NCG. i.e. primops that essentially export 'closure_flags[type]' and 'get_itbl(c)->type' to the NCG. Does anyone know of a better solution? In any case I am happy for the moment with my hacky solution, since it gives the right results and it is what I need to write up my paper. Cheers, Bernie.

1 0

RE: adding isWHNF primop to 5.00.2 native code generator
by Simon Marlow 01 Aug '01

01 Aug '01

> \begin{code} > primCode [res] IsHNF [arg] > = let res' = amodeToStix res > arg' = amodeToStix arg > arg_info = StInd PtrRep arg' > word_32 = StInd WordRep (StIndex PtrRep > arg_info (StInt (-1))) > masked_le32 = StPrim SrlOp [word_32, StInt 16] > masked_be32 = StPrim AndOp [word_32, StInt 65535] > #ifdef WORDS_BIGENDIAN > ty_info = masked_le32 > #else > ty_info = masked_be32 > #endif > not_a_thunk = StPrim IntEqOp [ StPrim AndOp > [ty_info, StInt 0x10] > , StInt 0x0 > ] > -- ToDo: don't hardwire the value of > _THUNK from InfoTables.h > assign = StAssign IntRep res' not_a_thunk > in > returnUs (\ xs -> assign : xs) > \end{code} This code looks bogus - it is checking a bit in the info table, which would have been correct when we used to store flags in the info table, but that changed a while back. closure_THUNK() works by getting the type field (i.e. get_itbl(c)->type), and using that to index into the table of closure flags (see ghc/rts/ClosureFlags.c), and checking the "THU" flag. This is the portable way to do it. The hacky way to do it would be to compare against the closure types directly - fortunately you can use the names rather than the constants, see ghc/compiler/codeGen/SMRep.lhs. Cheers, Simon

1 0

RE: adding isWHNF primop to 5.00.2 native code generator
by Julian Seward (Intl Vendor) 01 Aug '01

01 Aug '01

| Sigbjorn gave me this suggestion, which is based on DataToTagOp. | | \begin{code} | primCode [res] IsHNF [arg] | = let res' = amodeToStix res | arg' = amodeToStix arg | arg_info = StInd PtrRep arg' | word_32 = StInd WordRep (StIndex PtrRep | arg_info (StInt (-1))) | masked_le32 = StPrim SrlOp [word_32, StInt 16] | masked_be32 = StPrim AndOp [word_32, StInt 65535] | #ifdef WORDS_BIGENDIAN | ty_info = masked_le32 | #else | ty_info = masked_be32 | #endif | not_a_thunk = StPrim IntEqOp [ StPrim AndOp [ty_info, StInt 0x10] | , StInt 0x0 | ] | -- ToDo: don't hardwire the value of | _THUNK from InfoTables.h | assign = StAssign IntRep res' not_a_thunk | in | returnUs (\ xs -> assign : xs) | \end{code} | | | I get different results with the version using the C macro | and the native code version. In particular I get the wrong | (unexpected result from the native code version). | | The C macro version works like this (uses these macros): | | #define closureFlags(c) (closure_flags[get_itbl(c)->type]) | -- in ghc/includes/InfoTables.h | | #define closure_THUNK(c) ( closureFlags(c) & _THU) | -- in ghc/includes/InfoTables.h | | #define get_itbl(c) (INFO_PTR_TO_STRUCT((c)->header.info)) | -- in ghc/includes/ClosureMacros.h | | #define INFO_PTR_TO_STRUCT(info) ((StgInfoTable *)(info) - 1) | -- in ghc/includes/ClosureMacros. | | The (-1) scares me a little bit, in INFO_PTR_TO_STRUCT. | | The array closure_flags[] is just a look up table, indexed by | the closure type. So the key part of the code above is: | | get_itbl(c)->type | | In the native code version above, the lines below act like | get_itbl(c): | | arg_info = StInd PtrRep arg' | word_32 = StInd WordRep (StIndex PtrRep arg_info (StInt (-1))) | | After this I am lost. It seems to be grabbing the top or | bottom 16 bits of word_32 (depending on endianness) and then | ANDing those bits with 0x10 (which is the bit mask for _THU), | and checking for 0. | | My feeling is that I should be finding out the "type" field | from the closure and then switching on its value. What you have started with looks plausible. If you can extract enough info from the NCG, you can usually find the problem without too much difficulty. I suggest you make friends with -ddump-stix, so that you can see directly the result of your extension to the primCode fn above. Then (if you are mad enough) the usual way to go about this is: 1. Write a plausible primCode case -- as you've already done. 2. Compile with -ddump-stix and see if it looks plausible in context. 3. Read the assembly code (-ddump-asm) and relate it to the Stix. 4. Try and relate (3) to the assembly code for the same when compiled with -fvia-C. This usually helps. (3) and (4) are a lot easier if you do it on x86 than sparc because x86 code is a lot less verbose. You do need to be clear about x86 addressing modes, tho. | I don't understand the native code generator well enough. I | am assuming that closures are laid out the same way as via-C, Yes. Code from the NCG and -fvia-C is 100% interoperable. | in particular that there is going to be some stuff in the | info table before the "type" field (this looks variable | depending on how things are built). Basically I'm looking for | a way to do the equivalent of "->type". Maybe there is a | better way to do it. Suggestion: find some other primop which uses the ->type field and which is implemented in the NCG (not all are). That might help. Finally let me point you at ghc/utils/debugNCG, a small but very useful program for debugging the NCG. I would never have got it working as well as it does without it. This should work -- but it might have slight bitrot -- it's very fragile and makes lots of assumptions about gcc's output. I haven't used it for a good few months. J

1 0

adding isWHNF primop to 5.00.2 native code generator
by Bernard James POPE 01 Aug '01

01 Aug '01

Hi all, More mind boggling questions on prim ops :) I am happy with isWHNF that is implemented with: #define isHNFzh(r,a) r=(! closure_THUNK((StgClosure *)a)) I will re-write for the IO Monad later. I want to mimic this behaviour in the native code generator. Sigbjorn gave me this suggestion, which is based on DataToTagOp. \begin{code} primCode [res] IsHNF [arg] = let res' = amodeToStix res arg' = amodeToStix arg arg_info = StInd PtrRep arg' word_32 = StInd WordRep (StIndex PtrRep arg_info (StInt (-1))) masked_le32 = StPrim SrlOp [word_32, StInt 16] masked_be32 = StPrim AndOp [word_32, StInt 65535] #ifdef WORDS_BIGENDIAN ty_info = masked_le32 #else ty_info = masked_be32 #endif not_a_thunk = StPrim IntEqOp [ StPrim AndOp [ty_info, StInt 0x10] , StInt 0x0 ] -- ToDo: don't hardwire the value of _THUNK from InfoTables.h assign = StAssign IntRep res' not_a_thunk in returnUs (\ xs -> assign : xs) \end{code} I get different results with the version using the C macro and the native code version. In particular I get the wrong (unexpected result from the native code version). The C macro version works like this (uses these macros): #define closureFlags(c) (closure_flags[get_itbl(c)->type]) -- in ghc/includes/InfoTables.h #define closure_THUNK(c) ( closureFlags(c) & _THU) -- in ghc/includes/InfoTables.h #define get_itbl(c) (INFO_PTR_TO_STRUCT((c)->header.info)) -- in ghc/includes/ClosureMacros.h #define INFO_PTR_TO_STRUCT(info) ((StgInfoTable *)(info) - 1) -- in ghc/includes/ClosureMacros. The (-1) scares me a little bit, in INFO_PTR_TO_STRUCT. The array closure_flags[] is just a look up table, indexed by the closure type. So the key part of the code above is: get_itbl(c)->type In the native code version above, the lines below act like get_itbl(c): arg_info = StInd PtrRep arg' word_32 = StInd WordRep (StIndex PtrRep arg_info (StInt (-1))) After this I am lost. It seems to be grabbing the top or bottom 16 bits of word_32 (depending on endianness) and then ANDing those bits with 0x10 (which is the bit mask for _THU), and checking for 0. My feeling is that I should be finding out the "type" field from the closure and then switching on its value. I don't understand the native code generator well enough. I am assuming that closures are laid out the same way as via-C, in particular that there is going to be some stuff in the info table before the "type" field (this looks variable depending on how things are built). Basically I'm looking for a way to do the equivalent of "->type". Maybe there is a better way to do it. Thanks once again, and apologies if I missed something obvious. Cheers, Bernie.

1 0

RE: help needed for adding isWHNF primop to 5.00.2
by Bernard James POPE 01 Aug '01

01 Aug '01

Hi all, Apologies for my large number of postings to this list of late. > the normal definition of WHNF applies only > to types containing _|_, i.e. pointed types. Unpointed types are never > represented by thunks, so I would be inclined to include them in WHNF. Yes, I agree here. It is useful for me since I am using WHNF to determine whether expressions are evaluated. Perhaps it is a slight abuse of the terminology, but it gives me what I want :) > You also have to deal with indirections, although you can assume (at the > moment) that an indirection will eventually lead to an object in WHNF. Good point. I did make this assumption. Well to be precise I made the assumption that the thunk and indirection closure flags were mututally exclusive. If things change I guess I will have to follow the indirections. currently my definition is: #define isHNFzh(r,a) r=(! closure_THUNK((StgClosure *)a)) I believe under this definition that unpointed types and indirections are considered to be in WHNF. My initial tests seem to indicate that this is working well. > You have to declare the primitive like this, > BTW: > > isWHNF# :: a -> (# State# RealWorld, Int# #) > > because the compiler doesn't know about the IO type and you can't return > a Bool directly. I'm not too fussed about the unboxed Int, although as Marcin points out it does look like some of the comparision primops can return a Bool. I couldn't figure out how they did it so I left it with the Int#. > There's no problem with the simplifier as long as you declare the > primitive to have the correct properties, i.e. that the polymorphic > argument is lazy. Great. > It sounds like you also need a way to take apart arbitrary objects and > look at their components. Yes indeed. I use type classes for this with deriving rules for user defined types. I'm in the midst of writing this up for IFL, in Stockholm, mid September. In particular, I am looking for "portable" ways of taking objects apart, that's why type classes are attractive. Of course, isWHNF cannot be written in Haskell, but if I can get it to work for Hugs, GHC and NHC, then that is good enough for me :) What I hope is that some of this can appear as a Meta-programming library for Haskell, which is more powerful than Read and Show. Something a bit similar to the semantic domain of Meta-ML. > I know that Andy Gill was also interested in > having similar facilities, perhaps he can help out. I wonder if Andy is reading this? Possibly there are other people around who are also interested in this. I think the HAT people at NHC headquarters are also interested in these things. There once was talk of starting a debugging group for Haskell, but I don't know what happened. Thanks heaps for your help. Bernie.

3 3

RE: FFI and Control-C
by Simon Marlow 01 Aug '01

01 Aug '01

> So, my questions are: > > 1) Does anyone know what exactly happens when control-C is > pressed during foreign function evaluation? Yes. Two cases: if the program hasn't requested that Ctrl-C be trapped (using the Posix library's installHandler), then the SIGINT handler just sets a flag which causes the scheduler to kill all the running threads and exit. If the program is in a foreign call, then the scheduler won't get to run until the foreign call returns. If the program has installed a handler for SIGINT (which GHCi does), then the SIGINT handler in the RTS sets a different flag. The next time the scheduler runs, it creates a new thread for the handler and puts it on the run queue. In the case of GHCi, the handler thread raises an exception in the main thread, which is caught by the GHCi loop. Again, if the main thread is in a foreign call, none of this will happen until the call returns. > 2) Does anyone know how this can be controlled in a more > finegrained way? We don't have a good solution for this at the moment. Where possible, don't do blocking foreign calls. The threadWaitRead/threadWaitWrite primitives can be used to avoid blocking if you know the file descriptor that the foreign call is going to block on. Cheers, Simon

1 0