Hi Carter,

Nick Partridge hit this recently and he's got PR's pending for Happy and Alex https://github.com/simonmar/alex/pull/37 and https://github.com/simonmar/happy/pull/12

I'm not using GCC. (In my previous message, I made a typo. "GCC" should be read as "clang"). So, I think that this is a different problem. --Kazu

Hi, I can finally build GHC head on Mavericks. I gave up on clang and installed GHC 4.8 with MacPorts. Then, I compiled alex and happy with GHC 7.6.3/GHC 4.8. With this, I can build GHC head. This step is a MUST. If alex and happy are complied GHC 7.6.3/clang-wrapper, I cannot build GHC head with GHC 7.6.3/GHC 4.8. --Kazu

Hi,

Adding the "-v" option, GHC displays:

---- *** Checking old interface for main:Lexer: *** Parser: *** Renamer/typechecker:

<built-in>:2:2: Not in scope: `#' ----

"-keep-tmp-files" keeps generated .hscpp file. The beginning of the generated .hscpp is:

---- # 1 "utils/genprimopcode/dist/build/Lexer.hs" # 1 "<built-in>" 1 # 1 "utils/genprimopcode/dist/build/Lexer.hs" 2 {-# LANGUAGE CPP,MagicHash #-} {-# LINE 2 "utils/genprimopcode/./Lexer.x" #-} ----

"<built-in>:2:2: Not in scope: `#'" in my previous message indicates this line 2.

I don't know why Renamer/typechecker cannot handle this .hsccp file.

--Kazu

...

Hi,

As I wrote in the following blog article, I could build GHC head on Mavericks 20 days ago.

http://d.hatena.ne.jp/kazu-yamamoto/20131028/1382921924

But I cannot build it recently:

---- "/usr/bin/ghc" -hisuf hi -osuf o -hcsuf hc -static -H32m -O -package-db libraries/bootstrapping.conf -i -iutils/genprimopcode/. -iutils/genprimopcode/dist/build -iutils/genprimopcode/dist/build/autogen -Iutils/genprimopcode/dist/build -Iutils/genprimopcode/dist/build/autogen -package array -no-user-package-db -rtsopts -odir utils/genprimopcode/dist/build -hidir utils/genprimopcode/dist/build -stubdir utils/genprimopcode/dist/build -c utils/genprimopcode/dist/build/Lexer.hs -o utils/genprimopcode/dist/build/Lexer.o

<built-in>:2:2: Not in scope: `#' make[1]: *** [utils/genprimopcode/dist/build/Lexer.o] Error 1 make[1]: *** Waiting for unfinished jobs.... shift/reduce conflicts: 46 reduce/reduce conflicts: 1 make: *** [all] Error 2 ----

I seems to me that linemarkers generated by GCC(CPP) is not correctly handled when alex is used. Note I can build GHC head on FreeBSD and Linux.

I'm using clang with clang-xcode5-wrapper.hs.

Any suggestions?

--Kazu

So did you find a bug in the current design of the clang-wrapper hack?

No. What I found is that the current design of the clang-wrapper hack is appropriate. "-x assembler-with-cpp" is necessary to rescue ---- {-# RULE whatever #-} ---

If so it'd be better to get it fixed there rather than in the Alex/happy setup HS files.

If we can do this, it's very good. But this is not good enough. happy/alex hack are also necessary to not generate ---- # 1 "<built-inxxx>" 1 # 1 "templates/GenericTemplatexxx.hs" 2 ---- --Kazu

Hi,

so what are you proposing?

I'm not proposing anything. I just want to share information so that we can find a good solution.

Are you agreeing that those patches work with clang wrapper, but also remarking that its unfortunate that such a hack is needed?

As I said, nkpart's patches are not necessary in my environment. But they do not side-effect, so it is fine with me to apply these patches. --Kazu

Yes. So we agree. Clang only works with ghc when passed that flag. Hence for building ghc, clang should only be called via a wrapper script if the host ghc is 7.6 or older. The whole point of the clang wrapper script is to pass that flag. Nicks patch to Alex and happy is to address a further quirk of clang cpp. Was there an issue if you used a wrapped clang and those patches or not? I'm really confused. Cheers -carter On Tuesday, November 19, 2013, Kazu Yamamoto wrote:

Hi,

...

No. What I found is that the current design of the clang-wrapper hack is appropriate.

"-x assembler-with-cpp" is necessary to rescue ---- {-# RULE whatever #-} ---

I read the source of clang. The argument of "-x" is defined in lib/Frontend/CompilerInvocation.cpp:

.Case("c", IK_C) .Case("cl", IK_OpenCL) .Case("cuda", IK_CUDA) .Case("c++", IK_CXX) .Case("objective-c", IK_ObjC) .Case("objective-c++", IK_ObjCXX) .Case("cpp-output", IK_PreprocessedC) .Case("assembler-with-cpp", IK_Asm) .Case("c++-cpp-output", IK_PreprocessedCXX) .Case("objective-c-cpp-output", IK_PreprocessedObjC) .Case("objc-cpp-output", IK_PreprocessedObjC) .Case("objective-c++-cpp-output", IK_PreprocessedObjCXX) .Case("objc++-cpp-output", IK_PreprocessedObjCXX) .Case("c-header", IK_C) .Case("cl-header", IK_OpenCL) .Case("objective-c-header", IK_ObjC) .Case("c++-header", IK_CXX) .Case("objective-c++-header", IK_ObjCXX) .Cases("ast", "pcm", IK_AST) .Case("ir", IK_LLVM_IR)

In my experiment, no argument except "assembler-with-cpp" can rescue " #-}".

--Kazu

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org javascript:; http://www.haskell.org/mailman/listinfo/ghc-devs

Ok. Got it. So some more patches were needed. Pardon the asking, was somehow getting confused earlier. On Tuesday, November 19, 2013, Kazu Yamamoto wrote:

...

Was there an issue if you used a wrapped clang and those patches or not? I'm really confused.

Even with GHC/clang-wrapper and alex/happy with Nick's patches, I cannot build GHC head. (I'm not using GCC 42 at all in this case.)

AlexTemplate (pre-processed by clang) has linemarkers like this:

---- # 1 "<built-in>" ----

So, Lexer.hs generated from Lexer.x by alex also includes this one.

Lexer.hs is CPP language extension. So, it is pre-processed again by clang. Since clang-wrapper specifies "-x assembler-with-cpp", generated Lexcer.hscpp contains:

---- # 1 "<built-in>" ----

GHC cannot handle this, so build fails.

My hack lets alex and happy not to produce linemarkers at all.

Is this explanation clear to you?

P.S.

If we modify clang-wrapper to not convert "-x c" to "-x assembler-with-cpp", some other files which contains " #-}" cannot be compiled.

--Kazu

I think these are difficult problems to talk about, because there are so many different versions of things flying about. But if I can sum up what I think has gone on here, I want to make sure we know what needs to happen next: * Kazu has verified that my patches to happy/alex allow us to build HEAD using GHC 7.6 + GCC 4.8. * There are separate patches needed for happy/alex to allow building of HEAD using GHC 7.6 + Clang (and as I see in the other thread, also a patch for ghc.mk) Kazu, can you put create pull requests for happy and alex with your required changes? If those changes play nice, hopefully we can then ship new versions of happy and alex for mac users shortly. - Nick On Wed, Nov 20, 2013 at 3:25 PM, Kazu Yamamoto wrote:

...

Ok. Got it. So some more patches were needed.

Nice.

...

Pardon the asking, was somehow getting confused earlier.

No problem. Maybe I explained too quickly.

--Kazu _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

I think he has forgotten, yes. But yeah do send the pull requests anyway, my last set got merged in. The missing code seems to just be the version bump and a new test case: http://hdiff.luite.com/cgit/alex/commit/?id=f0f1f332d56b3fd21930d771b15b7c0d... On Thu, Nov 21, 2013 at 3:25 PM, Kazu Yamamoto wrote:

Hi,

...

Kazu, can you put create pull requests for happy and alex with your required changes?

I have one concern.

The version of alex on Hackage is 3.1.2. But that of github is 3.1.1:

https://github.com/simonmar/alex/blob/master/alex.cabal

# This inconsistency exists in happy, too.

Did SimonM forget to push his local changes?

Should I send pull requests on github, anyway?

--Kazu

On 21/11/13 06:54, Kazu Yamamoto (山本和彦) wrote:

...

I think he has forgotten, yes. But yeah do send the pull requests anyway, my last set got merged in.

Done: https://github.com/simonmar/happy/pull/13 https://github.com/simonmar/alex/pull/38

Sorry, I closed your pull requests without seeing this discussion. But I'm not sure about this fix. I'm happy to merge in fixes for Mac in general, but in this case we seem to lose the line pragmas, which are actually useful (error messages will point to the original source, instead of some preprocessed file). Cheers, Simon

On 21/11/2013 13:05, Kazu Yamamoto (山本和彦) wrote:

Hi SimonM,

...

But I'm not sure about this fix. I'm happy to merge in fixes for Mac in general, but in this case we seem to lose the line pragmas, which are actually useful (error messages will point to the original source, instead of some preprocessed file).

Yes. If we can find another solution which keeps line markers, it is much better, anyway.

But this patch effects templates only. When a template is included into an HS file (genearted from a '.x' file), the position is shifted. So, I'm not sure that line markers are important as you think.

Isn't this why the line pragmas are important? If GHC reports an error in the template code, we want the error message to point to the template source (GenericTemplate.hs), not the generated source file. This works right now, it'd be a shame to lose it. Cheers, Simon

great sleuthing! thanks Kazu On Thu, Nov 21, 2013 at 11:48 PM, Kazu Yamamoto wrote:

Hi SimonM,

...

...

But this patch effects templates only. When a template is included into an HS file (genearted from a '.x' file), the position is shifted. So, I'm not sure that line markers are important as you think.

Isn't this why the line pragmas are important? If GHC reports an error in the template code, we want the error message to point to the template source (GenericTemplate.hs), not the generated source file. This works right now, it'd be a shame to lose it.

This is what I did not understand. Thank you for this explanation.

I was wondering why you are talking about line pragmas while I'm talking about line markers. And I noticed the real problem.

It is the format change of clang's line markers. GCC's line marker is as follows:

---- # 1 "<built-in>" ----

But clang's one is as follows:

---- # 1 "<built-in>" 1 ----

Since this kind of line markers consist of four tokens, Setup of alex/happy cannot convert to GHC's line pragmas. So, line markers are left in templates as is, resulting compile errors with GHC/clang-wrapper later.

I sent pull requests to fix this: https://github.com/simonmar/alex/pull/39 https://github.com/simonmar/happy/pull/15

I confirmed that I can build GHC head with this patched alex/happy.

The patches are redundant. SimonM may want to simplify them by defining local functions.

--Kazu _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Hi Simon,

...

...

I think he has forgotten, yes. But yeah do send the pull requests anyway, my last set got merged in.

Done: https://github.com/simonmar/happy/pull/13 https://github.com/simonmar/alex/pull/38

I've merged these and made new releases of Happy (1.19.2) and Alex (3.1.3).

I verified that this happy and alex can build GHC head on Mavericks. Thank you! --Kazu

Hey Sven: the simplest solution is to have users install and use GCC rather than clang. On Jan 2, 2014 1:16 PM, "Sven Panne" wrote:

Although it is not really GHC-related, this thread is sufficiently close to the problem described in https://github.com/haskell-opengl/OpenGLRaw/issues/18: AFAICT, Mac OS X 10.9's clang doesn't really honor -traditional, so what can I do to make things work with recent Macs without breaking all other platforms? I guess the right #if in

https://github.com/haskell-opengl/OpenGLRaw/blob/master/include/HsOpenGLRaw.... will do the trick, but I don't have access to a Mac. Hints are highly appreciated, the whole current Mac situation is a bit of a mystery to me...

Cheers, S. _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

sven,http://www.haskell.org/platform/mac.html has a wrapper script that makes clang play nice with CPP, though a simpler alternative one can be found on manuel's page here http://justtesting.org/post/64947952690/the-glasgow-haskell-compiler-ghc-on-.... That said, it also doesnt work for all use case, hence why i generally recommend people also have gcc installed. there are plans post 7.8 release to give ghc a haskell native CPP analogue, but thats not happening for 7.8 HP + ghc 7.6 users have to use a clang wrapper script like the above, probably http://justtesting.org/post/64947952690/the-glasgow-haskell-compiler-ghc-on-... the simplest one you an recommend, because it'll worth with haskell platform and none haskell platform systems. On Thu, Jan 2, 2014 at 2:37 PM, Sven Panne wrote:

2014/1/2 Carter Schonwald :

...

Hey Sven: the simplest solution is to have users install and use GCC rather than clang.

Hmmm, if clang is the default what people get on Mac OS X, I consider that a non-solution... :-) Furthermore: How will the upcoming Haskell Platform release handle these problems (clang vs. GCC, my OpenGLRaw problem)?

oh right, that is a problem with clang no matter what. lens hit this issue, https://github.com/ekmett/lens/issues/357 I think the solution is "don't do that" if you want to support clang being the C compiler GHC uses, though you'll have to look at the diffs on that ticket closely to see how they addressed it On Thu, Jan 2, 2014 at 4:31 PM, Sven Panne wrote:

2014/1/2 Carter Schonwald :

...

sven,http://www.haskell.org/platform/mac.html has a wrapper script that makes clang play nice with CPP, though a simpler alternative one can be found on manuel's page [...]

I've seen the wrappers before, but do they really solve the problem for OpenGLRaw (concatenation via /**/ and replacement in strings)? As I said, I don't have access to a Mac, but the mangled options don't look like if they have anything to do with that. Can somebody confirm that?

2014/1/2 Carter Schonwald :

it looks like their work around is using ## rather than /**/

Well, actually lens is bypassing the problem by using cpphs, not the C preprocessor. :-P OpenGLRaw is part of the Haskell Platform, and cpphs is not, so I can't simply depend on it. (Licensing issues IIRC?) "Don't do that" is not an option, either, at least not until the binding is auto-generated. If I see this correctly, I really have to do some preprocessor magic (slightly simplified output): ----------------------------------------------------------------------------- svenpanne@svenpanne:~$ cat preprocess.hs #define FOO(x) bla/**/x "x" #define BAR(x) bla##x #x FOO(baz) BAR(boo) svenpanne@svenpanne:~$ gcc -traditional -E -x c preprocess.hs blabaz "baz" bla##boo #boo svenpanne@svenpanne:~$ gcc -E -x c preprocess.hs bla baz "x" blaboo "boo" svenpanne@svenpanne:~$ clang -traditional -E -x c preprocess.hs bla baz "x" bla##boo #boo svenpanne@svenpanne:~$ clang -E -x c preprocess.hs bla baz "x" blaboo "boo" ----------------------------------------------------------------------------- If -traditional is not used, things are simple and consistent, and we can simply use ## and #. Alas, -traditional *is* used, so we can't use ## and # with gcc an we are out of luck with clang. This really sucks, and I consider the clang -traditional behavior a bug: How can you do concatenation/stringification with clang -traditional? One can detect clang via defined(__clang__) and the absence of -traditional via defined(__STDC__), but this doesn't really help here. Any suggestions? I am testing with a local clang 3.4 version (trunk 193323), but I am not sure if this matters.