On 26 January 2012 16:33, JP Moresmau <jpmoresmau@gmail.com> wrote:
Thomas, thank you for that explanation about the different type of
identifiers in the different phases of analysis. I've never seen that
information so clearly laid out before, can it be added to the wikis
(in http://hackage.haskell.org/trac/ghc/wiki/Commentary/Compiler/API
or http://www.haskell.org/haskellwiki/GHC/As_a_library maybe)? I think
it would be helpful to all people that want to dive into the GHC API.

Will do.
 

On a side note, I'm going to do something very similar in my
BuildWrapper project (which is now the backend of the EclipseFP IDE
plugins): instead of going back to the API every time the user
requests to know the type of "something" in the AST, I'm thinking of
sending the whole typed AST to the Java code. Maybe that's something
Christopher could use. Both the BuildWrapper code and Thomas's scion
code are available on GitHub, as they provide examples on how to use
the GHC API.

I really don't think you want to do much work on the front-end as that will just need to be duplicated for each front-end.  That was the whole point of building Scion in the first place.  I understand, of course, that Scion is not useful enough at this time.

Well, I currently don't have much time to work on Scion, but the plan is as follows:

  - Scion becomes a multi-process architecture.  It has to be since it's not safe to run multiple GHC sessions inside the same process.  Even if that were possible, you wouldn't be able to, say, have a profiling compiler and a release compiler in the same process due to how static flags work.  Separate processes have the additional advantage that you can kill them if they use too much memory (e.g., because you can't unload loaded interfaces).

  - Scion will be based on Shake and GHC will mostly be used in one-shot mode (i.e., not --make).  This makes it easier to handle preprocessed files.  It also allows us to generate and update meta-information on demand.  I.e., instead of parsing and typechecking a file and then caching the result for the current file, Scion will simply generate meta information whenever it (re-)compiles a source file and writes that meta information to a file.  Querying or caching that meta information then is completely orthogonal to generating it.  The most basic meta information would be a type-annotated version of the compiled AST (possibly + warnings and errors from the last time it was compiled).  Any other meta information can then be generated from that.

 - The GHCi debugger probably needs to be treated specially.  There also should be automatic detection of files that aren't supported by the bytecode compiler (e.g., those using UnboxedTuples) and force compilation to machine code for those.

 - The front-end protocol should be specified somewhere.  I'm thinking about using protobuf specifications and then use ways to generate custom formats from that (e.g., JSON, Lisp S-Expressions, XML?).  And if the frontend supports protocol buffers, then it can use that and be fast.  That also means that all serialisation code can be auto-generated.

I won't have time to work on this before the ICFP deadline (and only very little afterwards), but Scion is not dead (just hibernating).
 

JP


On Thu, Jan 26, 2012 at 2:31 PM, Thomas Schilling
<nominolo@googlemail.com> wrote:
>
>
> On 26 January 2012 09:24, Christopher Brown <cmb21@st-andrews.ac.uk> wrote:
>> Hi Thomas,
>>
>> By static semantics I mean use and bind locations for every name in the
>> AST.
>
> Right, that's what the renamer does in GHC.  The GHC AST is parameterised
> over the type of identifiers used.  The three different identifier types
> are:
>
> RdrName: is the name as it occurred in source code. This is the output of
> the parser.
> Name: is basically RdrName + unique ID, so you can distinguish two "x"s
> bound at different locations (this is what you want). This is the output of
> the renamer.
> Id: is Name + Type information and consequently is the output of the type
> checker.
>
> Diagram:
>
>    String  --parser-->  HsModule RdrName  --renamer-->  HsModule Name
>  --type-checker-->  HsBinds Id
>
> Since you can't hook in-between renamer and type checker, it's perhaps more
> accurately depicted as:
>
>    String  --parser-->  HsModule RdrName  --renamer+type-checker-->
>  (HsModule Name,  HsBinds Id)
>
> The main reasons why it's tricky to use the GHC API are:
>
> You need to setup the environment of packages etc.  E.g., the renamer needs
> to look up imported modules to correctly resolve imported names (or give a
> error).
> The second is that the current API is not designed for external use.  As I
> mentioned, you cannot run renamer and typechecker independently, there are
> dozens of invariants, there are environments being updated by the various
> phases, etc.  For example, if you want to generate code it's probably best
> to either generate HsModure RdrName or perhaps the Template Haskell API
> (never tried that path).
>
>
> / Thomas
>
> --
> Push the envelope. Watch it bend.
>



--
JP Moresmau
http://jpmoresmau.blogspot.com/



--
Push the envelope. Watch it bend.