True multi stage Haskell

After many years of hoping someone else would do this, I would like to make GHC into a true multi-stage programming language. Here is how I thought I might approach this. 1) Use the GHC as a library module. 2) Use the LLVM backend. I have no experience with either of these tools. Lets start simple, How would I write functions like compile :: String -> IO PtrToLLVMCode -- where the string is a small Haskell program. llvmCompile :: PtrToLLVMCode -> IO PtrToMachineCode jumpTo:: PtrToMachineCode -> IO ans -- where ans is the "type" of the string. Any thoughts on how to get started? What papers to read, examples to look at? I'd love to move to some more disciplined input type, a sort of (mono) typed program representation (with similar complexity) to Template Haskell Exp type. where (Exp t) is a data structure representing a Haskell program of type t. All offers of advice accepted. I'm trying to get started soon, and good advice about what to avoid is especially welcome. If any one wanted to help with this, that would be great. Tim Sheard

Hi Tim,
When you say "multi-stage programming language" do you have a specific
calculus in mind? I think this can mean lots of different things to
different people.
This is a topic I have been interested in recently.
Cheers,
Matt
On Fri, Nov 17, 2017 at 6:05 PM, Tim Sheard
After many years of hoping someone else would do this, I would like to make GHC into a true multi-stage programming language. Here is how I thought I might approach this.
1) Use the GHC as a library module. 2) Use the LLVM backend.
I have no experience with either of these tools. Lets start simple, How would I write functions like
compile :: String -> IO PtrToLLVMCode -- where the string is a small Haskell program. llvmCompile :: PtrToLLVMCode -> IO PtrToMachineCode jumpTo:: PtrToMachineCode -> IO ans -- where ans is the "type" of the string.
Any thoughts on how to get started? What papers to read, examples to look at?
I'd love to move to some more disciplined input type, a sort of (mono) typed program representation (with similar complexity) to Template Haskell Exp type.
where (Exp t) is a data structure representing a Haskell program of type t.
All offers of advice accepted. I'm trying to get started soon, and good advice about what to avoid is especially welcome. If any one wanted to help with this, that would be great.
Tim Sheard
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Hello Tim, I don't know the answers to your questions exactly, but I've played around with something that I think might be related, so I thought I'd share. The example code is here: https://github.com/GaloisInc/galua/blob/master/galua-jit/src/Galua/Micro/Com... This module contains a function that compiles (at runtime) a Haskell module represented as a String to the value of whatever `main` happened to be defined in this module. Actually, in this case the expression is taken as a `Doc` , which is the result of pretty printing something, but this doc is simply "show"-ed to make a string (see `modText`). The function works in roughly two steps: 1. Compile the module: 1.1. Setup some compiler flags, this has to do with the dependencies of the module (i.e., what packages it depends on, etc) 1.2. Define a "target": this is what GHC will be compiling. There is a bit of weirdness here: GHC can take the input to compile either from a text file, or from an in-memory buffer. The second is what we want here. Unfortunately, for some reason GHC still insists on there being a file around (its contents is completely irrelevant) I think it just want to check when it was last modified :-) This is why we create an empty file. This is something that should probably be fixed in GHC. 1.3 Load all the targets: this is what will actually compile the module 2. Get the value of `main` in this module: this is done in the `Succeed` branch of the case, where we create an expression `M.main` and tell GHC to evaluate it. Depending on how you need things to be setup, just the second step might be enough (i.e., call `compileExpr`). You'd still need to do enough to tell GHC in what context it will be compiling the expression though. I am not an expert on this topic, but I'd be happy to try to answer more questions if I can. -Iavor On Fri, Nov 17, 2017 at 10:42 AM Matthew Pickering < matthewtpickering@gmail.com> wrote:
Hi Tim,
When you say "multi-stage programming language" do you have a specific calculus in mind? I think this can mean lots of different things to different people.
This is a topic I have been interested in recently.
Cheers,
Matt
On Fri, Nov 17, 2017 at 6:05 PM, Tim Sheard
wrote: After many years of hoping someone else would do this, I would like to make GHC into a true multi-stage programming language. Here is how I thought I might approach this.
1) Use the GHC as a library module. 2) Use the LLVM backend.
I have no experience with either of these tools. Lets start simple, How would I write functions like
compile :: String -> IO PtrToLLVMCode -- where the string is a small Haskell program. llvmCompile :: PtrToLLVMCode -> IO PtrToMachineCode jumpTo:: PtrToMachineCode -> IO ans -- where ans is the "type" of the string.
Any thoughts on how to get started? What papers to read, examples to look at?
I'd love to move to some more disciplined input type, a sort of (mono) typed program representation (with similar complexity) to Template Haskell Exp type.
where (Exp t) is a data structure representing a Haskell program of type t.
All offers of advice accepted. I'm trying to get started soon, and good advice about what to avoid is especially welcome. If any one wanted to help with this, that would be great.
Tim Sheard
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Hi Tim, About using GHC as a library module, have you see the ongoing work on "native metaprogramming" in GHC at https://ghc.haskell.org/trac/ghc/wiki/NativeMetaprogramming and a child project at https://ghc.haskell.org/trac/ghc/wiki/ImplementingTreesThatGrow There, online code generation is not of the highest priority, rather we are trying to open up the compiler from top to down: making available GHC's AST and then the parser and the typechecker have priority over the other functionalities down the pipeline (like the code generator).
If any one wanted to help with this, that would be great.
I am interested to hear more about the approach you have in mind, and would
be happy to help.
Yours,
Shayan
On Fri, Nov 17, 2017 at 6:05 PM, Tim Sheard
After many years of hoping someone else would do this, I would like to make GHC into a true multi-stage programming language. Here is how I thought I might approach this.
1) Use the GHC as a library module. 2) Use the LLVM backend.
I have no experience with either of these tools. Lets start simple, How would I write functions like
compile :: String -> IO PtrToLLVMCode -- where the string is a small Haskell program. llvmCompile :: PtrToLLVMCode -> IO PtrToMachineCode jumpTo:: PtrToMachineCode -> IO ans -- where ans is the "type" of the string.
Any thoughts on how to get started? What papers to read, examples to look at?
I'd love to move to some more disciplined input type, a sort of (mono) typed program representation (with similar complexity) to Template Haskell Exp type.
where (Exp t) is a data structure representing a Haskell program of type t.
All offers of advice accepted. I'm trying to get started soon, and good advice about what to avoid is especially welcome. If any one wanted to help with this, that would be great.
Tim Sheard
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Hi Tim, Am Freitag, den 17.11.2017, 13:05 -0500 schrieb Tim Sheard:
After many years of hoping someone else would do this, I would like to make GHC into a true multi-stage programming language. Here is how I thought I might approach this.
1) Use the GHC as a library module. 2) Use the LLVM backend.
I have no experience with either of these tools.
Lets start simple, How would I write functions like
compile :: String -> IO PtrToLLVMCode -- where the string is a small Haskell program. llvmCompile :: PtrToLLVMCode -> IO PtrToMachineCode jumpTo:: PtrToMachineCode -> IO ans -- where ans is the "type" of the string.
maybe not quite what you are looking for, but veggies, at https://github.com/nomeata/veggies uses the GHC API build a “new” Haskell compiler with its own Core→LLVM pass. Cheers, Joachim -- Joachim Breitner mail@joachim-breitner.de http://www.joachim-breitner.de/

Hi Tim,
Several years ago I wrote a proof of concept of one way to implement this
in Haskell,
http://johnlato.blogspot.com/2012/10/runtime-meta-programming-in-haskell.htm...
.
I used TH to automatically lift expressions into a newtype-wrapped ExpQ
that could be used for staged evaluation. More than one stage would
probably have been tedious though, and a major problem with any non-trivial
code was getting all the imports in place. I do think it would have been
possible to automatically track imports though.
Of course that was 5 years ago, so state of the art has moved on. TH has
better facilities for this now, and the ghc API probably has something
better too.
On Fri, Nov 17, 2017, 10:06 Tim Sheard
After many years of hoping someone else would do this, I would like to make GHC into a true multi-stage programming language. Here is how I thought I might approach this.
1) Use the GHC as a library module. 2) Use the LLVM backend.
I have no experience with either of these tools. Lets start simple, How would I write functions like
compile :: String -> IO PtrToLLVMCode -- where the string is a small Haskell program. llvmCompile :: PtrToLLVMCode -> IO PtrToMachineCode jumpTo:: PtrToMachineCode -> IO ans -- where ans is the "type" of the string.
Any thoughts on how to get started? What papers to read, examples to look at?
I'd love to move to some more disciplined input type, a sort of (mono) typed program representation (with similar complexity) to Template Haskell Exp type.
where (Exp t) is a data structure representing a Haskell program of type t.
All offers of advice accepted. I'm trying to get started soon, and good advice about what to avoid is especially welcome. If any one wanted to help with this, that would be great.
Tim Sheard
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
participants (6)
-
Iavor Diatchki
-
Joachim Breitner
-
John Lato
-
Matthew Pickering
-
Shayan Najd
-
Tim Sheard