- C2hs - Haskell.org

patch applied (c2hs): Rename parser state fields, change get/setInput type and tidy imports
by Duncan Coutts 19 Feb '07

19 Feb '07

Mon Feb 12 10:59:51 PST 2007 Duncan Coutts <duncan.coutts(a)worc.ox.ac.uk> * Rename parser state fields, change get/setInput type and tidy imports M ./c2hs/c/CLexer.x -12 +15 M ./c2hs/c/CParser.y -2 +3 M ./c2hs/c/CParserMonad.hs -24 +24

1 0

patch applied (c2hs): Split tokens and parser monad out of Lexer.x into their own modules
by Duncan Coutts 19 Feb '07

19 Feb '07

Mon Feb 12 04:20:37 PST 2007 Duncan Coutts <duncan.coutts(a)worc.ox.ac.uk> * Split tokens and parser monad out of Lexer.x into their own modules This allows the parser monad to be made a little more abstract. M ./c2hs/c/CLexer.x -382 +4 A ./c2hs/c/CParserMonad.hs A ./c2hs/c/CTokens.hs

1 0

a new C parser
by Duncan Coutts 19 Feb '07

19 Feb '07

All, I posted some time ago about a plan to make a new C parser that would correctly deal with the typedef problem and then to try some automated testing by making a gcc wrapper. I've now mostly done that. Currently I can parse almost all of the Linux kernel, glibc, GNU binutils, and the C code produced by ghc when it compiles itself. I can also parse most of the artificial nasty typedef examples: typedef int A, B(A); I think I correctly deal with scope of typedef names and typedef names being shadowed by local variable declarations: typedef int A; A main () { A A; -- declares local var A as having type A A = A + 1; -- A now refers to the local var, not the type; } A x; -- end of scope, A reverts to referring to the typedef name. A slightly more tricky example of the same thing is where function parameters redeclare names: typedef int A; A main (A A) { A = A + 1; -- A now refers to the local var, not the type; } An example I don't deal with yet is: typedef int A; int A = sizeof(A); that is we shadow the typename A with the local var A, so the A in the initialiser should now refer to the local var not the typename. However currently the parser only does the shadowing after the ';', not after the '=' as would be required. This is because the production rule looks like: declaring_list : declaration_specifier declarator initializer_opt {% do doDeclIdent $1 $2 return $ CDecl $1 [($2, $3, Nothing)]) } where as we'd really like to say: declaring_list : declaration_specifier declarator {% doDeclIdent $1 $2 } initializer_opt { CDecl $1 [($2, $4, Nothing)]) } that is to modify the typedef set immediately after the declarator and before parsing the initializer. This is exactly how people do it with YACC but unfortunately happy does not support this form where actions are interspersed in the production. It's probably possible to refactor the grammar to do this with happy, though it's not totally obvious to me how. Dealing with '__attribute__'s: ============================== GNU C attributes are a real pain. They seem to be able to appear almost anywhere in a declaration. However adding them to the grammar has to be done very carefully to avoid introducing ambiguities (ie shift/reduce or reduce/reduce conflicts). So for the moment I've taken the approach of not parsing them at all but recognising and ignoring attributes in the lexer and not passing them on to the parser at all. This effectively treats attributes as whitespace and they can appear anywhere. This isn't a long term solution because eventually we'll need to recognise some attributes because they can affect structure layout and we need that for accurately calculating the size of types and offsets of structure members. C99 extensions: =============== I'm not sure I've got 100% coverage of C99 features, but at least I cover the ones used in the code I've seen so far. These are just the ones that I can remember adding: * mixed declarations and statements in a compound statement * compound literals * for statements with declarations: for (int n = 0; n < m; n++) {}; * wide character character and string literals * unsigned long long numeric literals * _Bool and _Complex basic types GNU extensions: =============== Again, just some of the GNU extensions that I can remember adding to the new parser compared to the extensions the previous parser recognised: * anonymous structs and unions inside other structs/unions. * thread local storage qualifier * old style and array range compound literal member designator * case ranges * computed gotos and address of labels * conditional operator with missing then part: "x ? : y" * empty structs * allow redundant ';' in several places * asm expressions * several built-in 'functions' that take type names as parameters Various of these extensions require extensions in the AST. Testing: ======== I've made a cc-wrapper program that you can use as if it were gcc. It calls gcc with the same args as it was called with. Then if it can figure out what the gcc args meant and it looks like gcc was being asked to compile a .c file then it tries to pre-process and parse the same .c file. It outputs results to a log file and puts detailed reports on parse failures into separate files (in a directory specified via an environment variable). So (in theory at least) one ought to be able to compile vast amounts of C code and find out which extensions are really used and break the parser. At the moment the it doesn't check that it correctly parses the code, only that it parses it at all. Extra checks as I mentioned before could include checking if the sizes of types and offsets of structure members computed by c2hs match those computed by gcc. Another test would be to pretty-print the C code again and see if gcc can still parse it and if it produces the same object code (that'd require fully parsing attributes). As I said, I've been testing this on the linux kernel (using allyesconfig) glibc and ghc. Last time I tried it against linux it failed on 7 out of 4406 .c files. All those failures were due to the same bug which I think I've now fixed. In glibc, a few files fail to parse due to their use of nested function declarations which is a rarely used GNU extension that I've not implemented yet. Plan: ===== So there are a few things to do, one is to continue testing large amounts of C code. I intend to include cc-wrapper in c2hs (or perhaps do it as a c2hs mode via a flag) so that users can do their own testing and provide a way to give detailed bug reports. I haven't checked yet that the new C parser doesn't break things when doing the full .chs -> .hs translations. Also, I've not extended the guts of c2hs to understand any of these new language extensions so trying to bind things which really use them may not work. Then I need to start merging the new parser into the darcs version of c2hs. I'll try and do this as a series of small understandable patches. If anyone wants to play with the code I've got right now rather than wait for things to get merged then do say. I could put up a tarball. Duncan

3 3

Re: [Haskell-cafe] c2hs failed to process __extension__
by Duncan Coutts 18 Feb '07

18 Feb '07

On Mon, 2007-01-29 at 21:25 +0900, 向井淳 wrote: > c2hs fails to process __extension__? > > From the changelog, c2hs now can understand __extension. > Is this an exceptional case? Did you try the very latest darcs version of c2hs? Yesterday I added a couple patches that try to cope with more uses of __extension. Duncan

1 1

patch applied (c2hs): Remove the token number comments in the parser.
by Duncan Coutts 12 Feb '07

12 Feb '07

Fri Feb 9 05:16:12 PST 2007 Duncan Coutts <duncan.coutts(a)worc.ox.ac.uk> * Remove the token number comments in the parser. They were not accurate anyway. M ./c2hs/c/CParser.y -89 +89

1 0

patch applied (c2hs): Make the token constructors stricter
by Duncan Coutts 12 Feb '07

12 Feb '07

Fri Feb 9 05:15:17 PST 2007 Duncan Coutts <duncan.coutts(a)worc.ox.ac.uk> * Make the token constructors stricter Both in their semantic content and the position M ./c2hs/c/CLexer.x -89 +89

1 0

patch applied (c2hs): Make Position a proper (though not abstract) type
by Duncan Coutts 12 Feb '07

12 Feb '07

Fri Feb 9 05:13:08 PST 2007 Duncan Coutts <duncan.coutts(a)worc.ox.ac.uk> * Make Position a proper (though not abstract) type Also strictify and unpack the row and col numbers M ./base/errors/Errors.hs -2 +2 M ./base/general/Position.hs -26 +31 M ./base/syntax/Lexers.hs -8 +8 M ./c2hs/c/C.hs -2 +2 M ./c2hs/c/CLexer.x -5 +5 M ./c2hs/chs/CHS.hs -3 +3 M ./c2hs/chs/CHSLexer.hs -3 +3

1 0

patch applied (c2hs): Tidy up .cabal file
by Duncan Coutts 12 Feb '07

12 Feb '07

Fri Feb 9 04:48:58 PST 2007 Duncan Coutts <duncan.coutts(a)worc.ox.ac.uk> * Tidy up .cabal file M ./c2hs.cabal -3 +2

1 0

patch applied (c2hs): Eliminate need for ./configure
by Duncan Coutts 12 Feb '07

12 Feb '07

Fri Feb 9 04:45:07 PST 2007 Duncan Coutts <duncan.coutts(a)worc.ox.ac.uk> * Eliminate need for ./configure Move the version nick and date into the Version module directly (We might be able to get those into the cabal-generated module in future to make it more easily maintainable) Instead of using autoconf's host spec, use the equivalent from System.Info.(arch,os). Add host info for x86-64. Remove unused datadir var. ./c2hs/toplevel/C2HSConfig.hs.in -> ./c2hs/toplevel/C2HSConfig.hs ./c2hs/toplevel/Version.hs.in -> ./c2hs/toplevel/Version.hs M ./c2hs/toplevel/C2HSConfig.hs -14 +13 M ./c2hs/toplevel/Main.hs -4 +1 M ./c2hs/toplevel/Version.hs -7 +7 M ./configure.in -12 +1

1 0

patch applied (c2hs): Use Cabal' s mechanism to find the datadir rather than our own
by Duncan Coutts 12 Feb '07

12 Feb '07

Fri Feb 9 02:42:38 PST 2007 Duncan Coutts <duncan.coutts(a)worc.ox.ac.uk> * Use Cabal's mechanism to find the datadir rather than our own So remove the --data flag and the wrapper scripts. Cabal now installs the C2HS.hs file. Also means the Setup.hs no longer needs to install the wrapper script or C2HS.hs. ./c2hs/lib/C2HS.hs -> ./C2HS.hs M ./Setup.hs -26 +1 M ./c2hs.cabal +2 R ./c2hs/c2hs-inplace.in R ./c2hs/c2hs.in M ./c2hs/state/Switches.hs -2 M ./c2hs/toplevel/Main.hs -14 +3 R ./postInst.sh.in

1 0