proposal: standardize interface to Haskell' implementations

Claus Reinke

12 Feb 2006 12 Feb '06

5:42 p.m.

[an innocent question on ghc-users just reminded me of another missed opportunity in previous Haskell definitions: by chosing to ignore the very idea of implementations, they have left tool implementors in a limbo.] these days, there is some momentum for providing Haskell with various tools for refactoring, documentation, profiling, tracing, instance generators, analyzers, pre-processors for extensions, editor modes, interactive interfaces (textual, graphical, visual), .. and even first ides, but all that is hampered by the lack of a standardized interface to functionality that exists in every single Haskell implementation: parsing, type checking, AST, (pretty printing,) semantic information, (evaluation). that functionality is common to most tools, hard (and unneccessary) work to reimplement, and almost impossible to keep up to date, diverting precious man-power from small teams trying to provide much needed tools. all we have at the moment is the haskell-src part of the libraries Language.Haskell.{Parser,Pretty,Syntax), and some Hugs- or GHC-specific APIs (Hugs Server API, package GHC), or other frontends like Programatica, etc.. compare that with Lisp, Smalltalk, SML/NJ and the like, where reflection support means that tool builders get a fantastic head start by being able to reuse available implementation functionality. if you think that isn't the job of a language standard, think again; for instance, there is the ADA Semantic Interface Specification (ASIS) - "an interface between an Ada environment (as defined by ISO/IEC 8652:1995) and any tool or application requiring information from it." http://www.acm.org/sigada/wg/asiswg/ASIS_FAQ.html http://www.acm.org/sigada/wg/asiswg/ I am aware that the precise form of, say, the GHC API is still in flux, but I think Haskell' is a great chance to make a start on standardizing the basics(*), so that those tools will be available for *all* conforming Haskell' implementations, and so that the tool implementers are freed to focus on their core functionality! cheers, claus (*) a standard haskell' api providing the commands of ghci/hugs style interactive systems would be a start, together with an annotated AST, parser/typer/pretty printer. more detailed specifications could be left for future revisions.

Show replies by date

Neil Mitchell

12 Feb 12 Feb

5:52 p.m.

...

[an innocent question I know this was me, and various things I do would be a LOT easier if this standard interface did exist, but I don't think its possible.

To compare GHC, Hugs and Yhc - they have very little in common. Hugs is written in C, the other two in Haskell, so for a start there is no easy consistent interface to the bits of them that are required. Hugs and GHC use Yacc/Happy while Yhc uses parser combinators. GHC has a really complex type system, while in Yhc its impossible to even figure out the types by the time you get down to Core. Even the actual stages in the compiler are different in some places - Yhc has no kind inference for example. Because of all this, if you make a standard like this, you basically dictate a large part of the implementation, and it seems no one wants to follow the same implementation path... Maybe, as all the various implementations stabalise and start to provide some API's, in the future it will be possible to write a standard library that translates between the various API's and then you would have a standard interface. Unfortunately, I think thats still years and years away. And as someone who uses Ada on a weekly basis, I'm opposed to doing anything similar to Ada in any way ;) Thanks Neil

Claus Reinke

13 Feb 13 Feb

5:29 a.m.

eeek! how negative!-) would all pessimists and nay-sayers please return to their seats, fasten their seat-belts and refrain from smoking - the haskell' process is preparing for lift-off! by cs standards, Haskell is _old_. it already was old when java entered the scene, and java is not exactly the new kid on the block. the *only* reason why such things would be impossible now would be if we once again decided not to do anything about them. when java was new, they completely ignored reflection support. even when it turned out to be badly needed in practice, it was only added in half-hearted and incomplete ways. iirc, it was only at about version 1.2 that reflection support was beginning to be taken seriously, and the transition pains, incomplete and unstable apis where considerable. but by today, we find ourselves in a situation where _proposals_ for the advanced research vehicle Haskell refer to _current practice_ in the dull industry standard Java. where does that leave us? you're right that not all haskell implementations are the same. perhaps you even underestimate the differences: even those implementations that already have an interactive loop and are implemented in haskell will not usually have the ast/type info in whatever form the standard will require. the point is to standardise an api to functionality that all haskell implementations will need in some form or other and that all haskell tools should be able to depend on. whether that api is implemented as a separate haskell library like a haskell-src-exts package extended with type info, or whether that api is implemented by converting internal formats to standardised structures is a completely different matter (do you remember "hmake interactive"?-). i was *not* suggesting to standardise how things are *implemented*, but how things are *used* and *presented*. you will notice that my main list was limited to facilities needed in every implementation, with features that might only be partially available (pretty-printing, evaluation) in parentheses. it does not matter whether your optimizer is implemented by posting sources to the programming language shootout, whether your parser is based on neural net hardware, or whether your full type system is going to puzzle type theoreticians for years to come. what matters is that you are trying to implement haskell', and you have to parse and type-check source code and produce user-readable error messages. and if it type-checks, there will presumably be some means to run the thing, right? these things need to be implemented, but they should not have to be implemented again and again in every tool. that is exactly what a standard is for! even (n+1) implementations (n proper haskell' implementations and 1 haskell'-in-haskell' standard library) would be an improvement over the situation we have today. naturally, the haskell' implementers would be _very_ keen to optimize their version of the haskell'-in-haskell' library _if_ they want all those tools to be useable with _their_ implementation..

...

...
[an innocent question I know this was me, and various things I do would be a LOT easier if this standard interface did exist, but I don't think its possible.

as I was trying to explain, you are far from the first person in that situation. but if Haskell' makes a start on the issue, you might become one of the last persons with that problem!

...

Maybe, as all the various implementations stabalise and start to provide some API's, in the future it will be possible to write a standard library that translates between the various API's and then you would have a standard interface. Unfortunately, I think thats still years and years away.

that's what I was thinking at around 1997, when I first felt the need for this kind of API. unless we change our thinking into something more optimistic, this kind of thing is never going to happen! cheers, claus

...

And as someone who uses Ada on a weekly basis, I'm opposed to doing anything similar to Ada in any way ;)

oh, i'm only using it when i want to refer to voluminous standards!-)

Christopher Brown

6:29 a.m.

Claus, I totally agree with you on this. With experience of developing HaRe and realising the limitations of using one compiler front end over the over. It would be so much easier to have a standard API - so that projects such as HaRe could be easily ported to another system if needs be - instead of a complete rewrite of the entire system. I'm not quite sure I fully understand how you intend on making things like the AST standard? Have a standard AST structure? It certainly worth some serious thought though... Chris. On 13 Feb 2006, at 10:29, Claus Reinke wrote:

...

eeek! how negative!-)

would all pessimists and nay-sayers please return to their seats, fasten their seat-belts and refrain from smoking - the haskell' process is preparing for lift-off!

by cs standards, Haskell is _old_. it already was old when java entered the scene, and java is not exactly the new kid on the block.

the *only* reason why such things would be impossible now would be if we once again decided not to do anything about them.

when java was new, they completely ignored reflection support. even when it turned out to be badly needed in practice, it was only added in half-hearted and incomplete ways. iirc, it was only at about version 1.2 that reflection support was beginning to be taken seriously, and the transition pains, incomplete and unstable apis where considerable. but by today, we find ourselves in a situation where _proposals_ for the advanced research vehicle Haskell refer to _current practice_ in the dull industry standard Java. where does that leave us?

you're right that not all haskell implementations are the same. perhaps you even underestimate the differences: even those implementations that already have an interactive loop and are implemented in haskell will not usually have the ast/type info in whatever form the standard will require.

the point is to standardise an api to functionality that all haskell implementations will need in some form or other and that all haskell tools should be able to depend on. whether that api is implemented as a separate haskell library like a haskell-src-exts package extended with type info, or whether that api is implemented by converting internal formats to standardised structures is a completely different matter (do you remember "hmake interactive"?-).

i was *not* suggesting to standardise how things are *implemented*, but how things are *used* and *presented*. you will notice that my main list was limited to facilities needed in every implementation, with features that might only be partially available (pretty-printing, evaluation) in parentheses.

it does not matter whether your optimizer is implemented by posting sources to the programming language shootout, whether your parser is based on neural net hardware, or whether your full type system is going to puzzle type theoreticians for years to come. what matters is that you are trying to implement haskell', and you have to parse and type-check source code and produce user-readable error messages. and if it type-checks, there will presumably be some means to run the thing, right?

these things need to be implemented, but they should not have to be implemented again and again in every tool. that is exactly what a standard is for! even (n+1) implementations (n proper haskell' implementations and 1 haskell'-in-haskell' standard library) would be an improvement over the situation we have today. naturally, the haskell' implementers would be _very_ keen to optimize their version of the haskell'-in-haskell' library _if_ they want all those tools to be useable with _their_ implementation..

...
...
[an innocent question I know this was me, and various things I do would be a LOT easier if this standard interface did exist, but I don't think its possible.

as I was trying to explain, you are far from the first person in that situation. but if Haskell' makes a start on the issue, you might become one of the last persons with that problem!

...
Maybe, as all the various implementations stabalise and start to provide some API's, in the future it will be possible to write a standard library that translates between the various API's and then you would have a standard interface. Unfortunately, I think thats still years and years away.

that's what I was thinking at around 1997, when I first felt the need for this kind of API. unless we change our thinking into something more optimistic, this kind of thing is never going to happen!

cheers, claus

...
And as someone who uses Ada on a weekly basis, I'm opposed to doing anything similar to Ada in any way ;)

oh, i'm only using it when i want to refer to voluminous standards!-)

_______________________________________________ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime

Christopher Brown Ph.D. Student University of Kent. UK. http://www.cs.kent.ac.uk/people/rpg/cmb21/index.html

Gabriel Dos Reis

10:09 a.m.

"Claus Reinke" writes: [...] | the point is to standardise an api to functionality that all | haskell implementations will need in some form or other and that all | haskell tools should be able to depend on. something in line of Template Haskell? -- Gaby

Claus Reinke

10:41 a.m.

...

| the point is to standardise an api to functionality that all | haskell implementations will need in some form or other and that all | haskell tools should be able to depend on.

something in line of Template Haskell?

not really. it would be nice to see TH standardized (at least, I'd like to see some -separate- discussion about that here), but it is slighly different, and more complex. TH would be a likely client of such an Haskell' API, but would probably be limited by its facilities to look somewhat less well integrated than it is in its current form. TH does provide an interface to some of the functionality people would want from a Haskell' API, and TH has been used instead of separate frontends, so it would be worthwhile to look at both side-by-side, for inspirations;-) in other words, the ultimate Haskell'''''''' API and the ultimate TH might look similar, but for now, it might be better to think of them as separate, though related issues. in the meantime, perhaps someone might implement a "poor man's TH" on top of a Haskell' API, and GHC's TH and GHC API might drift towards a convergence with such. but first, we need one!-) claus

Gabriel Dos Reis

10:05 a.m.

Neil Mitchell writes: [...] | Because of all this, if you make a standard like this, you basically | dictate a large part of the implementation, and it seems no one wants | to follow the same implementation path... Indeed. I'm not sure ASIS is as successful as it was intended to. Previous experience with DIANA (still Ada) was no more successful. -- Gaby

Jan-Willem Maessen

10:05 a.m.

On Feb 12, 2006, at 5:42 PM, Claus Reinke wrote:

...

... these days, there is some momentum for providing Haskell with various tools for refactoring, documentation, profiling, tracing, instance generators, analyzers, pre-processors for extensions, editor modes, interactive interfaces (textual, graphical, visual), .. and even first ides, but all that is hampered by the lack of a standardized interface to functionality that exists in every single Haskell implementation: parsing, type checking, AST, (pretty printing,) semantic information, (evaluation).

that functionality is common to most tools, hard (and unneccessary) work to reimplement, and almost impossible to keep up to date, diverting precious man-power from small teams trying to provide much needed tools.

I agree strongly with the need for a standard parser/AST/typechecker to enable tools and extensions. That's why Fortress contains an AST specification! So naturally I'd love it if Haskell had one, too. It should probably include: * Parser * Standard AST types (which might be extended in particular implementations) * Type checker which produces a decorated AST (again, extended in particular implementations) * A renamer turns out to be awfully useful/necessary; this raises the sticky question of how imports are specified. It'd be nice *not* to have to dredge up the old .hi files, as they tended to require compilers to extend the .hi format in really non-standard ways. -Jan-Willem Maessen

Malcolm Wallace

10:30 a.m.

Jan-Willem Maessen writes:

...

* A renamer turns out to be awfully useful/necessary; this raises the sticky question of how imports are specified. It'd be nice *not* to have to dredge up the old .hi files, as they tended to require compilers to extend the .hi format in really non-standard ways.

On this page: http://haskell.galois.com/trac/haskell-prime/wiki/ModuleSystem the suggestion "Permit Signatures in Export Lists" would place the necessary interface information directly in the source files, rather than in separate .hi files. Would that solve the renaming problem? Regards, Malcolm

John Meacham

14 Feb 14 Feb

4:04 p.m.

On Mon, Feb 13, 2006 at 03:30:47PM +0000, Malcolm Wallace wrote:

...

Jan-Willem Maessen writes:

...
* A renamer turns out to be awfully useful/necessary; this raises the sticky question of how imports are specified. It'd be nice *not* to have to dredge up the old .hi files, as they tended to require compilers to extend the .hi format in really non-standard ways.

On this page: http://haskell.galois.com/trac/haskell-prime/wiki/ModuleSystem the suggestion "Permit Signatures in Export Lists" would place the necessary interface information directly in the source files, rather than in separate .hi files. Would that solve the renaming problem?

There is no naming problem other than that ghc just hasn't implemented it. jhc and helium both do cross module renaming properly and it is quite well specified. http://www.cse.ogi.edu/~diatchki/hsmod/ we might want to take some of the text from this paper and integrate it into the report rather than the current informal treatment. John -- John Meacham - ⑆repetae.net⑆john⑈

John Meacham

4 p.m.

On Mon, Feb 13, 2006 at 10:05:00AM -0500, Jan-Willem Maessen wrote:

...

I agree strongly with the need for a standard parser/AST/typechecker to enable tools and extensions. That's why Fortress contains an AST specification! So naturally I'd love it if Haskell had one, too. It should probably include: * Parser * Standard AST types (which might be extended in particular implementations) * Type checker which produces a decorated AST (again, extended in particular implementations) * A renamer turns out to be awfully useful/necessary; this raises the sticky question of how imports are specified. It'd be nice *not* to have to dredge up the old .hi files, as they tended to require compilers to extend the .hi format in really non-standard ways.

This seems like it would be a useful portable library, and it is something I hoped to spin off of jhc. but probably not the sort of thing that should go into a language specification. There is a world of difference between libraries that are part of the language spec and libraries that are portable. There is nothing that particularly depends on an implementation here anyway. ghc can compile jhc's type checker and use it just fine for instance obviously. a portable type checking AST library is completly possible. John -- John Meacham - ⑆repetae.net⑆john⑈

7036

Age (days ago)

7038

Last active (days ago)

List overview

Download

10 comments

7 participants

participants (7)

Christopher Brown
Claus Reinke
Gabriel Dos Reis
Jan-Willem Maessen
John Meacham
Malcolm Wallace
Neil Mitchell

proposal: standardize interface to Haskell' implementations

tags

participants (7)