I'd be willing to help out at least as a backup mentor for this. I'd prefer if we could find someone who is more familiar with the GHC peculiarities, but I can speak to the JVM and Java bytecode end and know the general STG execution model pretty well. -Edward On Mon, May 2, 2016 at 11:48 PM, Carter Schonwald wrote:

Woah this is some prodigious work!

1) would initially targeting only newer jvm variants make the work targets for the summer easier ?

2) it's worth talking with ghc folks on #ghc and the gsoc admins at #haskell-gsoc on freenode IRC about gsoc fit and possible mentors.

3) it sounds like the work so far is worth sharing on the r/Haskell sub Reddit. If you want. I do recommend IRC for talking with some of the active ghc and gsoc folk

On Monday, May 2, 2016, Rahul Muttineni wrote:

...

Hi GHC Developers,

I've started working on a JVM backend for GHC [1] and I'd love to work on it as my Summer of Haskell project.

Currently, the build system is setup using a mix of Shake (for the RTS build) and Stack (for the main compiler build) and I ensure that most commits build successfully. I have ported the core part of the scheduler and ported over the fundamental types (Capability, StgTSO, Task, StgClosure, etc.) taking advantage of OOP in the implementation when I could.

Additionally, I performed a non-trivial refactor of the hs-java package adding support for inner classes and fields which was very cumbersome to do in the original package. On the frontend, I have tapped into the STG code from the GHC 7.10.3 library and setup a CodeGen monad for generating JVM bytecode. The main task of generating the actual bytecode, porting the more critical parts of the RTS, and adding support for the threaded RTS remain.

The strategy for compilation is as follows: - Intercept the STG code in the GHC pipeline - Convert from STG->JVM bytecode [2] in a similar manner as STG->Cmm preserving semantics as best as possible [3] - Port the GHC RTS (normal & threaded) to Java [4] - Put all the generated class files + RTS into a single jar to be run directly by the JVM.

My objectives for the project during the summer are: - To implement the compilation strategy mentioned above - Implement the Java FFI for foreign imports. [5] - Implement the most important [6] PrimOps that GHC supports. - Port the base package replacing the C FFI imports with equivalent Java FFI imports. [7]

A little bit about myself: I spent a lot of time studying functional language implementation by reading SPJ's famous book and reading research papers on related topics last summer as self-study.

I took a break and resumed a couple months ago where I spent a lot of time plowing through the STG->Cmm code generator as well as the RTS and going back and forth between them to get a clear understanding of how everything works.

Moreover, I compiled simple Haskell programs and observed the STG, Cmm, and assembly output (by decompiling the final executable with objdump) to understand bits of the code generator where the source code wasn't that clear.

I also spent a great deal of time studying the JVM internals, reading the JVM spec, looking for any new features that could facilitate a high performance implementation [8].

It would be great if someone with an understanding of nuances of the RTS and code generator could mentor me for this project. It has been a blast so far learning all the prerequisites and contemplating the design. I'd be very excited to take this on as a summer project.

Also, given that I have hardly 5 days remaining, does anyone have suggestions on how I can structure the proposal without getting into too many details? There are still some parts of the design I haven't figured out, but I know I could find some solution when I get to it during the porting process.

Thanks, Rahul Muttineni

[1] http://github.com/rahulmutt/ghcvm

[2] I intend to organically derive an IR at a later stage to allow for some optimizations by looking at the final working implementation without an IR and looking for patterns of repeated sequences of bytecode and assigning each sequence its own instruction in the IR.

[3] Obviously, the lack of control of memory layouts (besides allocating off the JVM heap using DirectByteBuffers) and lack of general tail calls makes it tough to match the semantics of Cmm, but there are many solutions around it, as can be found in the few papers on translating STG to Java/JVM bytecode.

[4] This is the GHC RTS without GC and profiling since the JVM has great support for those already. Also, lots of care must be taken to ensure that the lock semantics stays in tact during the port.

[5] foreign exports will be dealt at a later stage, but I am taking care of naming the closures nicely so that in the future you don't have to type long names like the labels GHC compiles to call a Haskell function in Java.

[6] Basically all the PrimOps that would be required to provide plumbing for the Prelude functions that can compile beginner-level programs found in books such as Learn You a Haskell for Great Good.

[7] I know that it's a lot more complicated than just replacing FFI calls. I'd have to change around a lot of the code in base as well.

[8] I found that the new "invokedynamic" instruction as well as the MethodHandle API (something like function pointers) that were introduced in JDK 7 could fit the bill. But as of now, I want to get a baseline implementation that is compatible with Java 5 so I will not be utilizing these newer features.

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

@Carter: 1) Actually, that would be harder. I've given the implementation without the new features a lot of thought and it's simpler to get done. I've only briefly given a thought on how the newer JVM features could be used, but I figured as I port more of the GHC's RTS/code generator, I'll know enough of the nuances to be able to judge where to apply these new features effectively. 2) Thanks for the suggestion. I don't use IRC that often, but if I could reach more people there then I'll give it a try. 3) Right now it doesn't do anything that is visible to the user so I'll hold off on that until I've ported a tiny but usable subset of the code generator. My target is to get the basic Java FFI and I/O working so that there's something new and interesting to play with. @Edward: Thanks a lot! That would be great. @Karel: Yeah it is, but the reward of being able to run Haskell anywhere is definitely worth it! I'm aware of the LambdaVM project. I didn't mention it because I wanted to focus on the present. I sent Brian Alliet an email asking for guidance on architectural decisions a while back, but have received no response. I'd be interested in having a copy of the LambdaVM source, thanks. There are a couple of places in the design I'm iffy on, so it'd be nice to have a source of inspiration. Frege is a great project in its own right, but GHC Haskell has become the standard and I find many of the extensions to be useful in eliminating many more classes of bugs at compile-time. Thanks guys, Rahul Muttineni On Mon, May 2, 2016 at 9:59 PM, Karel Gardas wrote:

Rahul,

a lot of work in front of you. There was a similar project in the past, LambdaVM was it's name, author Brian Alliet -- unfortunately google can't reveal working site with it nor web.archive.org. What I can offer you are some of the latest sources of this project. It's based on GHC 6.7 version so really an old base, but perhaps something would be usable even for your own work and even after all those years? Let me know if you are interested and I'll pack that and upload somewhere -- well assuming you have not heart about it since otherwise I would expect it to be noted somewhere in your email. Just let me know if you are interested...

Anyway, I'm keeping my finger crossed for this your work! I'm personally very interested in seeing Haskell hosted in JVM and inter operable with Java...(yes, I know about Frege...)

Cheers, Karel

On 05/ 2/16 05:26 PM, Rahul Muttineni wrote:

...

Hi GHC Developers,

I've started working on a JVM backend for GHC [1] and I'd love to work on it as my Summer of Haskell project.

Currently, the build system is setup using a mix of Shake (for the RTS build) and Stack (for the main compiler build) and I ensure that most commits build successfully. I have ported the core part of the scheduler and ported over the fundamental types (Capability, StgTSO, Task, StgClosure, etc.) taking advantage of OOP in the implementation when I could.

Additionally, I performed a non-trivial refactor of the hs-java package adding support for inner classes and fields which was very cumbersome to do in the original package. On the frontend, I have tapped into the STG code from the GHC 7.10.3 library and setup a CodeGen monad for generating JVM bytecode. The main task of generating the actual bytecode, porting the more critical parts of the RTS, and adding support for the threaded RTS remain.

The strategy for compilation is as follows: - Intercept the STG code in the GHC pipeline - Convert from STG->JVM bytecode [2] in a similar manner as STG->Cmm preserving semantics as best as possible [3] - Port the GHC RTS (normal & threaded) to Java [4] - Put all the generated class files + RTS into a single jar to be run directly by the JVM.

My objectives for the project during the summer are: - To implement the compilation strategy mentioned above - Implement the Java FFI for foreign imports. [5] - Implement the most important [6] PrimOps that GHC supports. - Port the base package replacing the C FFI imports with equivalent Java FFI imports. [7]

A little bit about myself: I spent a lot of time studying functional language implementation by reading SPJ's famous book and reading research papers on related topics last summer as self-study.

I took a break and resumed a couple months ago where I spent a lot of time plowing through the STG->Cmm code generator as well as the RTS and going back and forth between them to get a clear understanding of how everything works.

Moreover, I compiled simple Haskell programs and observed the STG, Cmm, and assembly output (by decompiling the final executable with objdump) to understand bits of the code generator where the source code wasn't that clear.

I also spent a great deal of time studying the JVM internals, reading the JVM spec, looking for any new features that could facilitate a high performance implementation [8].

It would be great if someone with an understanding of nuances of the RTS and code generator could mentor me for this project. It has been a blast so far learning all the prerequisites and contemplating the design. I'd be very excited to take this on as a summer project.

Also, given that I have hardly 5 days remaining, does anyone have suggestions on how I can structure the proposal without getting into too many details? There are still some parts of the design I haven't figured out, but I know I could find some solution when I get to it during the porting process.

Thanks, Rahul Muttineni

[1] http://github.com/rahulmutt/ghcvm

[2] I intend to organically derive an IR at a later stage to allow for some optimizations by looking at the final working implementation without an IR and looking for patterns of repeated sequences of bytecode and assigning each sequence its own instruction in the IR.

[3] Obviously, the lack of control of memory layouts (besides allocating off the JVM heap using DirectByteBuffers) and lack of general tail calls makes it tough to match the semantics of Cmm, but there are many solutions around it, as can be found in the few papers on translating STG to Java/JVM bytecode.

[4] This is the GHC RTS without GC and profiling since the JVM has great support for those already. Also, lots of care must be taken to ensure that the lock semantics stays in tact during the port.

[5] foreign exports will be dealt at a later stage, but I am taking care of naming the closures nicely so that in the future you don't have to type long names like the labels GHC compiles to call a Haskell function in Java.

[6] Basically all the PrimOps that would be required to provide plumbing for the Prelude functions that can compile beginner-level programs found in books such as Learn You a Haskell for Great Good.

[7] I know that it's a lot more complicated than just replacing FFI calls. I'd have to change around a lot of the code in base as well.

[8] I found that the new "invokedynamic" instruction as well as the MethodHandle API (something like function pointers) that were introduced in JDK 7 could fit the bill. But as of now, I want to get a baseline implementation that is compatible with Java 5 so I will not be utilizing these newer features.

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-- Rahul Muttineni

Hi Rahul, This is pretty neat stuff. I'd be really interested in tracking this project and maybe making a few commits, though I'm not in a good position to be a full blown mentor. While it's not mainline Java/bytecode yet, you might have some future opportunities for better FFI and control of memory layout with the work being done in the OpenJDK Panama and Valhalla projects (resp.). Some of the features in these projects will probably be in future JDKs and might be worth harmonizing with on some level. I think you're on the right track with MethodHandles and invokedynamic. --Ian On Mon, May 2, 2016 at 12:15 PM Karel Gardas wrote:

On 05/ 2/16 07:55 PM, Rahul Muttineni wrote:

...

@Karel: Yeah it is, but the reward of being able to run Haskell anywhere is definitely worth it! I'm aware of the LambdaVM project. I didn't mention it because I wanted to focus on the present. I sent Brian Alliet an email asking for guidance on architectural decisions a while back, but have received no response.

I'd be interested in having a copy of the LambdaVM source, thanks. There are a couple of places in the design I'm iffy on, so it'd be nice to have a source of inspiration.

http://uloz.to/x3q5Mmi1/lambdavm-tar-bz2 -- this is 50 MB. I've not `gmake clean' it so you see what's compiled in jar/classes and what's not. Pity, if you'd like to have it running (besides already precompiled examples) you would need either Solaris 11 (my host) or "risk" build yourself.

...

Frege is a great project in its own right, but GHC Haskell has become the standard and I find many of the extensions to be useful in eliminating many more classes of bugs at compile-time.

Indeed, I'm looking forward to seeing/testing your results.

Karel

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Hi Karel, On Tue, May 3, 2016 at 12:45 AM, Karel Gardas wrote: http://uloz.to/x3q5Mmi1/lambdavm-tar-bz2 -- this is 50 MB. I've not `gmake

clean' it so you see what's compiled in jar/classes and what's not. Pity, if you'd like to have it running (besides already precompiled examples) you would need either Solaris 11 (my host) or "risk" build yourself.

That's fine, it was no problem. Thanks for taking the time and uploading it! It seems like it has dependencies and some more of Brian's packages (of which one has a module named Brianweb.Java), which again are nowhere to be found on the internet. Do you happen to have sources to his other packages too?

Indeed, I'm looking forward to seeing/testing your results.

Karel

Awesome, I'll be sure you keep you updated. Thanks, Rahul Muttineni

By the time it has made it down to Cmm there are a lot of assumptions about layout in memory -- everything is assumed to be a flat object made out of 32-bit or 64-bit slots. These assumptions aren't really suitable for the JVM. -Edward On Sat, May 7, 2016 at 11:32 AM, Thomas Jakway wrote:

This is a strange coincidence. I'm definitely no expert GHC hacker but I started (highly preliminary) work on a JVM backend for GHC a few weeks ago. It's here: https://github.com/tjakway/ghcjvm/tree/jvm/compiler/jvmGen/Jvm (The memory runtime is here: https://github.com/tjakway/lljvm)

I'm very new to this so pardon my ignorance, but I don't understand what the benefit is of intercepting STG code and translating that to bytecode vs. translating Cmm to bytecode (or Jasmin assembly, as I'd prefer)? It seems like Cmm is designed for backends and the obvious choice. Or have I got this really mixed up?

I hope this isn't out of line considering my overall lack of experience but I think I can give some advice:

read the JVM 7 spec cover-to-cover. I highly suggest outputting Jasmin assembly instead of raw bytecode. The classfile format is complicated and you will have to essentially rewrite Jasmin in Haskell if you don't want to reuse it. Jasmin is also the de facto standard assembler and much more thoroughly tested than any homegrown solution we might make. read the LLVM code generator. This project is more like the LLVM backend than the native code generator. Don't go for speed. The approach that I've begun is to emulate a C stack and memory system the RTS can run on top of (https://github.com/tjakway/lljvm/blob/master/src/main/java/lljvm/runtime/Mem...). This will make getting something working much faster and also solves the problem of how to deal with memcpy/memset/memmove on the JVM. This will of course be very slow (I think) and is not a permanent solution. Can't do everything at once. Any other approach will probably require rewriting the entire RTS from the beginning. I don't think Frege is especially useful to this project, though I'd love to be proven wrong. Frege's compilation model is completely different from GHC's: they compile Haskell to Java and then send that to javac. Porting GHC to the JVM is really more like writing a Cmm to JVM compiler.

I've heard of the LambdaVM project but couldn't find the actual code anywhere. The site where it was hosted appears to be offline. I'd certainly like to look at it if anyone knows where to find it.

Information on Jasmin: http://web.mit.edu/javadev/packages/jasmin/doc/ http://web.mit.edu/javadev/packages/jasmin/doc/instructions.html http://web.mit.edu/javadev/packages/jasmin/doc/about.html

Once you've tried manually dealing with constant pools you'll appreciate Jonathan Meyer's work!

I forked davidar's extended version of Jasmin. The differences versus the original Jasmin are detailed here. Some nice additions:

supports invokedynamic supports .annotation, .inner, .attribute, .deprecated directives better handling of the ldc_w instruction multi-line fields .debug directives signatures for local variables .bytecode directive to specify bytecode version (most importantly, I think): support for the StackMap attribute. If we eventually want to use new JVM instructions like invokedynamic, we need stack map frames or the JVM will reject our bytecode. JVM 7 has options to bypass this (but it's a hack), but they're deprecated and I believe not optional going forward. Alternatively we can stick with older bytecode versions indefinitely and not use the new features.

(Just to be clear, I forked it in case it was deleted. I didn't write those features, the credit belongs to him).

I think the biggest risk is taking too much on at once. Any one of these subtasks, writing a bytecode assembler, porting the RTS, etc. could consume the whole summer if you're not careful.

I'd love to help out with this project!

Sincerely, Thomas Jakway

-------

Woops, after scrolling back through the emails it looks like someone sent out the LambdaVM source. I'll have to take a look at that.

On 05/02/2016 11:26 AM, Rahul Muttineni wrote:

Hi GHC Developers,

I've started working on a JVM backend for GHC [1] and I'd love to work on it as my Summer of Haskell project.

Currently, the build system is setup using a mix of Shake (for the RTS build) and Stack (for the main compiler build) and I ensure that most commits build successfully. I have ported the core part of the scheduler and ported over the fundamental types (Capability, StgTSO, Task, StgClosure, etc.) taking advantage of OOP in the implementation when I could.

Additionally, I performed a non-trivial refactor of the hs-java package adding support for inner classes and fields which was very cumbersome to do in the original package. On the frontend, I have tapped into the STG code from the GHC 7.10.3 library and setup a CodeGen monad for generating JVM bytecode. The main task of generating the actual bytecode, porting the more critical parts of the RTS, and adding support for the threaded RTS remain.

The strategy for compilation is as follows: - Intercept the STG code in the GHC pipeline - Convert from STG->JVM bytecode [2] in a similar manner as STG->Cmm preserving semantics as best as possible [3] - Port the GHC RTS (normal & threaded) to Java [4] - Put all the generated class files + RTS into a single jar to be run directly by the JVM.

My objectives for the project during the summer are: - To implement the compilation strategy mentioned above - Implement the Java FFI for foreign imports. [5] - Implement the most important [6] PrimOps that GHC supports. - Port the base package replacing the C FFI imports with equivalent Java FFI imports. [7]

A little bit about myself: I spent a lot of time studying functional language implementation by reading SPJ's famous book and reading research papers on related topics last summer as self-study.

I took a break and resumed a couple months ago where I spent a lot of time plowing through the STG->Cmm code generator as well as the RTS and going back and forth between them to get a clear understanding of how everything works.

Moreover, I compiled simple Haskell programs and observed the STG, Cmm, and assembly output (by decompiling the final executable with objdump) to understand bits of the code generator where the source code wasn't that clear.

I also spent a great deal of time studying the JVM internals, reading the JVM spec, looking for any new features that could facilitate a high performance implementation [8].

It would be great if someone with an understanding of nuances of the RTS and code generator could mentor me for this project. It has been a blast so far learning all the prerequisites and contemplating the design. I'd be very excited to take this on as a summer project.

Also, given that I have hardly 5 days remaining, does anyone have suggestions on how I can structure the proposal without getting into too many details? There are still some parts of the design I haven't figured out, but I know I could find some solution when I get to it during the porting process.

Thanks, Rahul Muttineni

[1] http://github.com/rahulmutt/ghcvm

[2] I intend to organically derive an IR at a later stage to allow for some optimizations by looking at the final working implementation without an IR and looking for patterns of repeated sequences of bytecode and assigning each sequence its own instruction in the IR.

[3] Obviously, the lack of control of memory layouts (besides allocating off the JVM heap using DirectByteBuffers) and lack of general tail calls makes it tough to match the semantics of Cmm, but there are many solutions around it, as can be found in the few papers on translating STG to Java/JVM bytecode.

[4] This is the GHC RTS without GC and profiling since the JVM has great support for those already. Also, lots of care must be taken to ensure that the lock semantics stays in tact during the port.

[5] foreign exports will be dealt at a later stage, but I am taking care of naming the closures nicely so that in the future you don't have to type long names like the labels GHC compiles to call a Haskell function in Java.

[6] Basically all the PrimOps that would be required to provide plumbing for the Prelude functions that can compile beginner-level programs found in books such as Learn You a Haskell for Great Good.

[7] I know that it's a lot more complicated than just replacing FFI calls. I'd have to change around a lot of the code in base as well.

[8] I found that the new "invokedynamic" instruction as well as the MethodHandle API (something like function pointers) that were introduced in JDK 7 could fit the bill. But as of now, I want to get a baseline implementation that is compatible with Java 5 so I will not be utilizing these newer features.

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Hi Alois, I can see a nice feedback loop between GHCVM and Kuna that can help accelerate both projects. Let's this discuss offline so that this thread isn't flooded with a discussion that will probably get lost in time later? We'll try to keep the results of our discussions public, either in the GHCVM Wiki or elsewhere so that people know what's going on. Thanks, Rahul On Wed, May 11, 2016 at 4:40 PM, Aloïs Cochard wrote:

Hi Rahul,

Thanks for the feedback, and good to hear you are thinking about using it as an assembler! First thing I'll have to do is to extract the assembler itself from "Kuna", so it can be reused by yourself. I'll be happy to give support and implement new features as needed, I know some important operations are still missing but have a clear idea how to ad dthem.

Then a bit of history, where this project comes from, and why it had no commit since December...

Long story short, I got trapped working on a FP library in Scala, and had left this project aside in the meantime (had strong interest from folks asking for me to look into this Scala stuff, and at the time not much interest in Kuna... which the opposite of my personal preference... so I was extremely happy to read your initial message here!).

Now back to our common interest, the reason I "chose" Core, it's because it seems fundamentally impossible to avoid having an interpreter in order to get stack safe execution of STG on the JVM. That is a fundamental limitation, that is the core thing I want to experiment with... Creating a JVM assembler and a minimal Lambda core calculus was kind of just an excuse to get to this core issue (I'm thinking about control flow analysis and complex program transformation/optimization in order to get there).

That being said, the path I would like to take is quite complex, full of unknowns, I would love to share the ideas I have in mind, but I fear it might be out of scope of your actual goal: getting a workable implementation ASAP.

So, I suppose you are fine writing a small (hopefully performant) STG-like Java interpreter used during runtime (that's basically what I'm trying to avoid with my experiment), which I think is a reasonable path to follow in order to get a GHC JVM implementation soon. I would be very interested to help you on this, first by giving support on the JVM assembler (that will be extracted from Kuna), then also by proposing myself as a mentor. I'm not sure exactly how things goes there, how the process is handled... but I'll learn about it, I did notify Edward Kmett of my interest.

My gut feeling is that it should be fine to use STG with the approach you follow, I also feel some technics could be shared between GHC JVM and the Kuna experiment (where the later could be seen as the a sandbox/test-bed for the former).

Anyway as you said: "Again, all these issues can be looked at once a performance baseline has been established.".

Totally agree, that's why I don't want to share too much of my experimental ideas atm, let's focus on a realistic initial implementation (for which I have lot of ideas to share as well). Again, I'm ready to assist you as much as possible, implementation missing features in the assembler, I already read so much of this specification..... ;-)

In case you are using IRC/IM/Gitter let me knows, so we could discuss more directly.

Looking forward hearing from you,

Aloïs

On 10 May 2016 at 04:43, wrote:

...

Hi Alois,

I just checked out Kuna, and it looks like a great project. For others the link to the repo is https://github.com/aloiscochard/kuna. I think I'll go with it since not having to implement StackMapTables will save me a lot of time.

I'm interested in your approach, can you explain more, especially the stack-safe bytecode part? And I noticed the last commit was in December. Is there any particular reason you stopped the project?

I chose STG over Core because Core has an embedded language of coercions which complicates the code generation (or maybe they can simply be ignored?), and the terms are not in administrative normal form which requires more effort to manage. But a thought did cross my mind that certain Core optimizations might actually need to be turned off ‎because the resulting STG might be in a form that might not get translated to the most efficient JVM bytecode. Again, all these issues can be looked at once a performance baseline has been established.

Thanks, Rahul Muttineni

Sent from my BlackBerry 10 smartphone. *From: *Alois Cochard *Sent: *Monday 9 May 2016 10:21 PM *To: *Edward Kmett *Cc: *ghc-devs@haskell.org *Subject: *Re: Mentor for a JVM backend for GHC

Totally agree, Cmm is too late.

My aim in Kuna was to start the transformation from Core, targeting stack-safe JVM bytecode without using Graal or the like.

Yes, I am quite an optimistic person ;-) but I believe it's the path to take. I'm not interested in Frege like approach for various reasons, performance being one of them. On 7 May 2016 6:19 pm, "Edward Kmett" wrote:

...

By the time it has made it down to Cmm there are a lot of assumptions about layout in memory -- everything is assumed to be a flat object made out of 32-bit or 64-bit slots. These assumptions aren't really suitable for the JVM.

-Edward

...

This is a strange coincidence. I'm definitely no expert GHC hacker but I started (highly preliminary) work on a JVM backend for GHC a few weeks ago. It's here: https://github.com/tjakway/ghcjvm/tree/jvm/compiler/jvmGen/Jvm (The memory runtime is here: https://github.com/tjakway/lljvm)

I'm very new to this so pardon my ignorance, but I don't understand what the benefit is of intercepting STG code and translating that to bytecode vs. translating Cmm to bytecode (or Jasmin assembly, as I'd prefer)? It seems like Cmm is designed for backends and the obvious choice. Or have I got this really mixed up?

I hope this isn't out of line considering my overall lack of experience but I think I can give some advice:

read the JVM 7 spec cover-to-cover. I highly suggest outputting Jasmin assembly instead of raw bytecode. The classfile format is complicated and you will have to essentially rewrite Jasmin in Haskell if you don't want to reuse it. Jasmin is also the de facto standard assembler and much more thoroughly tested than any homegrown solution we might make. read the LLVM code generator. This project is more like the LLVM backend than the native code generator. Don't go for speed. The approach that I've begun is to emulate a C stack and memory system the RTS can run on top of ( https://github.com/tjakway/lljvm/blob/master/src/main/java/lljvm/runtime/Mem... ). This will make getting something working much faster and also solves

...

problem of how to deal with memcpy/memset/memmove on the JVM. This will of course be very slow (I think) and is not a permanent solution. Can't do everything at once. Any other approach will probably require rewriting the entire RTS from the beginning. I don't think Frege is especially useful to this project, though I'd love to be proven wrong. Frege's compilation model is completely different from GHC's: they compile Haskell to Java and then send that to javac. Porting GHC to the JVM is really more like writing a Cmm to JVM compiler.

I've heard of the LambdaVM project but couldn't find the actual code anywhere. The site where it was hosted appears to be offline. I'd certainly like to look at it if anyone knows where to find it.

Information on Jasmin: http://web.mit.edu/javadev/packages/jasmin/doc/ http://web.mit.edu/javadev/packages/jasmin/doc/instructions.html http://web.mit.edu/javadev/packages/jasmin/doc/about.html

Once you've tried manually dealing with constant pools you'll appreciate Jonathan Meyer's work!

I forked davidar's extended version of Jasmin. The differences versus

...

original Jasmin are detailed here. Some nice additions:

supports invokedynamic supports .annotation, .inner, .attribute, .deprecated directives better handling of the ldc_w instruction multi-line fields .debug directives signatures for local variables .bytecode directive to specify bytecode version (most importantly, I think): support for the StackMap attribute. If we eventually want to use new JVM instructions like invokedynamic, we need stack map frames or the JVM will reject our bytecode. JVM 7 has

...

bypass this (but it's a hack), but they're deprecated and I believe not optional going forward. Alternatively we can stick with older bytecode versions indefinitely and not use the new features.

(Just to be clear, I forked it in case it was deleted. I didn't write

...

features, the credit belongs to him).

I think the biggest risk is taking too much on at once. Any one of

...

subtasks, writing a bytecode assembler, porting the RTS, etc. could consume the whole summer if you're not careful.

I'd love to help out with this project!

Sincerely, Thomas Jakway

-------

Woops, after scrolling back through the emails it looks like someone sent out the LambdaVM source. I'll have to take a look at that.

On 05/02/2016 11:26 AM, Rahul Muttineni wrote:

Hi GHC Developers,

I've started working on a JVM backend for GHC [1] and I'd love to work on it as my Summer of Haskell project.

Currently, the build system is setup using a mix of Shake (for the RTS build) and Stack (for the main compiler build) and I ensure that most commits build successfully. I have ported the core part of the scheduler and ported over the fundamental types (Capability, StgTSO, Task, StgClosure, etc.) taking advantage of OOP in the implementation when I could.

Additionally, I performed a non-trivial refactor of the hs-java package adding support for inner classes and fields which was very cumbersome to do in the original package. On the frontend, I have tapped into the STG code from the GHC 7.10.3 library and setup a CodeGen monad for generating JVM bytecode. The main task of generating the actual bytecode, porting the more critical parts of the RTS, and adding support for the threaded RTS remain.

The strategy for compilation is as follows: - Intercept the STG code in the GHC pipeline - Convert from STG->JVM bytecode [2] in a similar manner as STG->Cmm preserving semantics as best as possible [3] - Port the GHC RTS (normal & threaded) to Java [4] - Put all the generated class files + RTS into a single jar to be run directly by the JVM.

My objectives for the project during the summer are: - To implement the compilation strategy mentioned above - Implement the Java FFI for foreign imports. [5] - Implement the most important [6] PrimOps that GHC supports. - Port the base package replacing the C FFI imports with equivalent Java FFI imports. [7]

A little bit about myself: I spent a lot of time studying functional language implementation by reading SPJ's famous book and reading research papers on related topics last summer as self-study.

I took a break and resumed a couple months ago where I spent a lot of time plowing through the STG->Cmm code generator as well as the RTS and going back and forth between them to get a clear understanding of how everything works.

Moreover, I compiled simple Haskell programs and observed the STG, Cmm, and assembly output (by decompiling the final executable with objdump) to understand bits of the code generator where the source code wasn't that clear.

I also spent a great deal of time studying the JVM internals, reading

...

JVM spec, looking for any new features that could facilitate a high performance implementation [8].

It would be great if someone with an understanding of nuances of the RTS and code generator could mentor me for this project. It has been a blast so far learning all the prerequisites and contemplating the design. I'd be very excited to take this on as a summer project.

Also, given that I have hardly 5 days remaining, does anyone have suggestions on how I can structure the proposal without getting into too many details? There are still some parts of the design I haven't

...

out, but I know I could find some solution when I get to it during the porting process.

Thanks, Rahul Muttineni

[1] http://github.com/rahulmutt/ghcvm

[2] I intend to organically derive an IR at a later stage to allow for some optimizations by looking at the final working implementation without an IR and looking for patterns of repeated sequences of bytecode and assigning each sequence its own instruction in the IR.

[3] Obviously, the lack of control of memory layouts (besides allocating off the JVM heap using DirectByteBuffers) and lack of general tail calls makes it tough to match the semantics of Cmm, but there are many solutions around it, as can be found in the few papers on translating STG to Java/JVM bytecode.

[4] This is the GHC RTS without GC and profiling since the JVM has great support for those already. Also, lots of care must be taken to ensure

...

the lock semantics stays in tact during the port.

[5] foreign exports will be dealt at a later stage, but I am taking care of naming the closures nicely so that in the future you don't have to type long names like the labels GHC compiles to call a Haskell function in Java.

[6] Basically all the PrimOps that would be required to provide

On Sat, May 7, 2016 at 11:32 AM, Thomas Jakway wrote: the the options to those these the figured that plumbing for

...

the Prelude functions that can compile beginner-level programs found in books such as Learn You a Haskell for Great Good.

[7] I know that it's a lot more complicated than just replacing FFI calls. I'd have to change around a lot of the code in base as well.

[8] I found that the new "invokedynamic" instruction as well as the MethodHandle API (something like function pointers) that were introduced in JDK 7 could fit the bill. But as of now, I want to get a baseline implementation that is compatible with Java 5 so I will not be utilizing these newer features.

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-- *Λ\ois* http://twitter.com/aloiscochard http://github.com/aloiscochard

-- Rahul Muttineni