Which raises the question then: what would be needed to make this happen by 7.10, if not sooner? naively, I'd imagine one part of the challenge is that Template haskell induces needing a ghc that can build libraries for *BOTH* host and target, because we have arbitrary execution at Template Haskell time, right? So does that mean fixing up GHC's architecture so that it can be built with support for more than 1 target? (or at least running libs on *host* in GHCI but otherwise compiling them for target? ) What would be needed in the restricted case of "We just want TH for our cross compiler" rather than "general multi target goodness"? Because If I squint, it sort of sounds like the cross compiler would need to be Stage 2, and that the UNsound but simplest approach would then be to take the various "libways", and produce both host and target variants as appropriate (or that "build HOST Way" has to explicitly flagged). I"m not sure what implications this or another approach would have on all the various build tooling needed etc etc. This is me just thinking out loud / trying to understand *why* and whats needed, and how complex the work would be to do it well, or at least decently in a way that *improves* (or doesn't complicate much more) the associated parts that would need to be fixed up. Basically: whats needed to make cross compiler GHC first class haskell wrt TH support? (I imagine that its not worth working on till the new TH work lands / stabilizes, but id like to understand what work has to be done to lead up to it). (if I can figure out ways I can directly or indirectly support such work happening, subject to having the time and/or resources over the next year, i'd like to try and do so) -Carter On Thu, Jun 13, 2013 at 8:28 PM, Manuel M T Chakravarty < chak@cse.unsw.edu.au> wrote:

Carter,

As you can imagine, I would be extremely happy about cross-compilation support for TH. However, I don't know of anybody working on it at the moment, unfortunately.

Manuel

...

Hey All, Whats the story planned for template haskell + cross compiler support come ghc 7.8? I understand theres a lot of Template Haskell design underway, some of which will help support tools like Manuel's Inline-Objective-C work. Does

Carter Schonwald : this mean that factored out within this reorganization is a better story for cross compilation?

...

Especially since one kill "app" for the Inline-Objective-C template

haskell work would be writing IOS applications. Though I guess that also touches on the need to sort out supporting "FAT" ARM binaries too, right?

...

This intersects with a few different large subsets of tickets, so i'm

not sure if any single ticket is the right fora for this question.

...

thanks! -Carter _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

On 6/14/13, Carter Schonwald wrote:

Which raises the question then: what would be needed to make this happen by 7.10, if not sooner?

naively, I'd imagine one part of the challenge is that Template haskell induces needing a ghc that can build libraries for *BOTH* host and target, because we have arbitrary execution at Template Haskell time, right?

From what I understood, executing (interpreted) bytecode (just like GHCi) should be enough unless you want to call target-only libs from TH.

Here is an older thread, speculating about the problems involved. http://comments.gmane.org/gmane.comp.lang.haskell.ghc.devel/771 Cheers, Gabor

So does that mean fixing up GHC's architecture so that it can be built with support for more than 1 target? (or at least running libs on *host* in GHCI but otherwise compiling them for target? )

What would be needed in the restricted case of "We just want TH for our cross compiler" rather than "general multi target goodness"?

Because If I squint, it sort of sounds like the cross compiler would need to be Stage 2, and that the UNsound but simplest approach would then be to take the various "libways", and produce both host and target variants as appropriate (or that "build HOST Way" has to explicitly flagged). I"m not sure what implications this or another approach would have on all the various build tooling needed etc etc.

This is me just thinking out loud / trying to understand *why* and whats needed, and how complex the work would be to do it well, or at least decently in a way that *improves* (or doesn't complicate much more) the associated parts that would need to be fixed up.

Basically: whats needed to make cross compiler GHC first class haskell wrt TH support? (I imagine that its not worth working on till the new TH work lands / stabilizes, but id like to understand what work has to be done to lead up to it).

(if I can figure out ways I can directly or indirectly support such work happening, subject to having the time and/or resources over the next year, i'd like to try and do so)

-Carter

On Thu, Jun 13, 2013 at 8:28 PM, Manuel M T Chakravarty < chak@cse.unsw.edu.au> wrote:

...

Carter,

As you can imagine, I would be extremely happy about cross-compilation support for TH. However, I don't know of anybody working on it at the moment, unfortunately.

Manuel

...

Hey All, Whats the story planned for template haskell + cross compiler support come ghc 7.8? I understand theres a lot of Template Haskell design underway, some of which will help support tools like Manuel's Inline-Objective-C work. Does

Carter Schonwald : this mean that factored out within this reorganization is a better story for cross compilation?

...

Especially since one kill "app" for the Inline-Objective-C template

haskell work would be writing IOS applications. Though I guess that also touches on the need to sort out supporting "FAT" ARM binaries too, right?

...

This intersects with a few different large subsets of tickets, so i'm

not sure if any single ticket is the right fora for this question.

...

thanks! -Carter _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Awesome! Please let us know how we can help get those into good shape for being merged in! That leaves the other question: how could we support the "Fat Binary" builds for IOS devices? I assume that this would only be supported via -fllvm, but work would be needed to support such? Is it a tractable small project to deal with that, or easy, or hard? (I'm not yet familiar enough with the relevant engineering in GHC, but I understand this is important for folks who wish to develop applications that target multiple IOS device variants, which apparently use different ARM variants hand in hand with that!) -Carter On Fri, Jun 14, 2013 at 4:11 PM, Luite Stegeman wrote:

We've had TH working in GHCJS [1] for a few months now, compiling once for the host platform, and once for the target.

Our GHCJS GHC patch [2] (that we hope to get merged before 7.8) adds, among other things, WayCustom to the Ways in DynFlags, which just adds a custom token to the filename extension. This, uhm, way (sorry), we generate native files ".o" and ".hi", and ArchJavaScript files ".js_o" and ".js_hi" in the same directory. TH always uses the native files.

We had to patch Cabal [3] to install the extra files (but we needed to patch Cabal anyway to add the GHCJS compiler flavour), but other than that it seems to work. GHC loads the native .a files when running TH, we do our own JavaScript linking.

Another thing we patched is the GHC.Prim interface, which is different for 32 and 64 bit code. When we generate code for ArchJavaScript, we load a custom GHC.Prim for 32 bit, while we use the regular GHC.Prim when we generate the native code.

It's all a bit hacky, but maybe this approach can be used as a basis for getting TH working in a real GHC. We could take some shortcuts to make a pseudo-mutltitarget compiler by just tacking an extra code generator onto a non-crosscompiling GHC. I'd be happy to help or discuss to make it more general.

Luite

[1] https://github.com/ghcjs / http://weblog.luite.com/wordpress/?p=14 [2] https://github.com/ghcjs/ghcjs.github.com/blob/master/patches/ghc-ghcjs.patc... [3] https://github.com/ghcjs/ghcjs.github.com/blob/master/patches/cabal-ghcjs.pa...

On Thu, Jun 13, 2013 at 11:43 PM, Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Hey All, Whats the story planned for template haskell + cross compiler support come ghc 7.8? I understand theres a lot of Template Haskell design underway, some of which will help support tools like Manuel's Inline-Objective-C work. Does this mean that factored out within this reorganization is a better story for cross compilation?

Especially since one kill "app" for the Inline-Objective-C template haskell work would be writing IOS applications. Though I guess that also touches on the need to sort out supporting "FAT" ARM binaries too, right?

This intersects with a few different large subsets of tickets, so i'm not sure if any single ticket is the right fora for this question.

thanks! -Carter

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Luite Via HWN I read your GHCJS posthttp://weblog.luite.com/wordpress/?p=14&date=2013-06-03 this morning. Great work. It's silly that you should need to patch GHC. One alternative is to absorb your patchhttps://github.com/ghcjs/ghcjs.github.com/blob/master/patches/ghc-ghcjs.patc..., or a successor. I think it mostly just extends the FFI and target types with new cases. The patch doesn't actually include any JS generation, so I'm far from sure how that happens. But it would be even better to make it easier to people to add new backends without modifying GHC, via some kind of plugin interface. I'm underwater with POPL right now, but do discuss on ghc-devs any details that would help you. Simon From: ghc-devs-bounces@haskell.org [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Luite Stegeman Sent: 14 June 2013 21:12 To: Carter Schonwald Cc: ghc-devs@haskell.org Subject: Re: status of template haskell + cross compiling plans for 7.8? We've had TH working in GHCJS [1] for a few months now, compiling once for the host platform, and once for the target. Our GHCJS GHC patch [2] (that we hope to get merged before 7.8) adds, among other things, WayCustom to the Ways in DynFlags, which just adds a custom token to the filename extension. This, uhm, way (sorry), we generate native files ".o" and ".hi", and ArchJavaScript files ".js_o" and ".js_hi" in the same directory. TH always uses the native files. We had to patch Cabal [3] to install the extra files (but we needed to patch Cabal anyway to add the GHCJS compiler flavour), but other than that it seems to work. GHC loads the native .a files when running TH, we do our own JavaScript linking. Another thing we patched is the GHC.Prim interface, which is different for 32 and 64 bit code. When we generate code for ArchJavaScript, we load a custom GHC.Prim for 32 bit, while we use the regular GHC.Prim when we generate the native code. It's all a bit hacky, but maybe this approach can be used as a basis for getting TH working in a real GHC. We could take some shortcuts to make a pseudo-mutltitarget compiler by just tacking an extra code generator onto a non-crosscompiling GHC. I'd be happy to help or discuss to make it more general. Luite [1] https://github.com/ghcjs / http://weblog.luite.com/wordpress/?p=14 [2] https://github.com/ghcjs/ghcjs.github.com/blob/master/patches/ghc-ghcjs.patc... [3] https://github.com/ghcjs/ghcjs.github.com/blob/master/patches/cabal-ghcjs.pa... On Thu, Jun 13, 2013 at 11:43 PM, Carter Schonwald mailto:carter.schonwald@gmail.com> wrote: Hey All, Whats the story planned for template haskell + cross compiler support come ghc 7.8? I understand theres a lot of Template Haskell design underway, some of which will help support tools like Manuel's Inline-Objective-C work. Does this mean that factored out within this reorganization is a better story for cross compilation? Especially since one kill "app" for the Inline-Objective-C template haskell work would be writing IOS applications. Though I guess that also touches on the need to sort out supporting "FAT" ARM binaries too, right? This intersects with a few different large subsets of tickets, so i'm not sure if any single ticket is the right fora for this question. thanks! -Carter _______________________________________________ ghc-devs mailing list ghc-devs@haskell.orgmailto:ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

I recently came across this: https://github.com/neurocyte/ghc-android The "build" script there together with the patches in the 'patch' directory seem to take care of the issues mentioned on this thread. Of course, that's quite a hacky way to make this work, but maybe Luite's work and this one could inspire a good long-term solution. On Wed, Jun 19, 2013 at 11:44 AM, Simon Marlow wrote:

The issue that we hit sooner than this is that when we build a cross-compiler, we build all the libs for the target platform and not the host platform (see http://hackage.haskell.org/** trac/ghc/wiki/CrossCompilationhttp://hackage.haskell.org/trac/ghc/wiki/CrossCompilation **). So to make TH work, we have to change the way we build cross compilers, and make the build system able to build and install two sets of libs. This is probably a big job.

I think it might be possible to avoid needing a multi-target GHC, as long as you have two identical GHCs, one that produces code for the host platform (host-GHC) and one for the target platform (cross-GHC). You compile all the code once with host-GHC, and then again with the cross-GHC, the second time using the .o files produced by host-GHC to run the TH code.

Cheers, Simon

On 17/06/13 14:52, Simon Peyton-Jones wrote:

...

I have not been following the details of this debate, but there is a good reason why TH doesn’t work on a cross compiler. Specifically, suppose module M imports module X, which defines a function mkD that M calls in a splice, thus $(mkD “wobble”).

Currently, we compile X to X.o, and when compiling M we dynamically link X.o (and all the other libraries it relies on) into GHC so that we can call foo. Obviously X.o has to be runnable on the machine doing the compiling, so if X.o is for some other architecture that’s not going to work.

There is no reason in principle why one could not compile X into X.bytecode (along with all its dependencies), where X.bytecode is architecture independent. Then X.bytecode could be interpreted on any platform. But GHC has no mechanism for doing this at all. I’ve always take the view that if you want X.bytecode, you may as well load X.hs and translate it into bytecode. A bit slower, to be sure, but maybe fast enough. But don’t forget those libraries.

Anyway that’s the state of play. Have fun!

Simon

*From:*ghc-devs-bounces@**haskell.org [mailto:ghc-devs-bounces@**haskell.org ] *On Behalf Of *Carter Schonwald *Sent:* 13 June 2013 22:44 *To:* ghc-devs@haskell.org *Subject:* status of template haskell + cross compiling plans for 7.8?

Hey All,

Whats the story planned for template haskell + cross compiler support come ghc 7.8?

I understand theres a lot of Template Haskell design underway, some of which will help support tools like Manuel's Inline-Objective-C work. Does this mean that factored out within this reorganization is a better story for cross compilation?

Especially since one kill "app" for the Inline-Objective-C template haskell work would be writing IOS applications. Though I guess that also touches on the need to sort out supporting "FAT" ARM binaries too, right?

This intersects with a few different large subsets of tickets, so i'm not sure if any single ticket is the right fora for this question.

thanks!

-Carter

______________________________**_________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/**mailman/listinfo/ghc-devshttp://www.haskell.org/mailman/listinfo/ghc-devs

______________________________**_________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/**mailman/listinfo/ghc-devshttp://www.haskell.org/mailman/listinfo/ghc-devs

-- Alp Mestanogullari