ANN: Efficient, dynamically compiled, lazy functional semantics on JVM, having tight integration with the Java language
Greetings denizens of the Haskell mailing list! Though this is not strictly an announcement about Haskell, it might be of interest to a number of folk here. The Research Group here at Business Objects has been beavering away since 1999 on a suite of technologies to allow us to represent certain kinds of business logic as reusable, composable pieces. One of the primary requirements was that a general runtime for such objects be embeddable within the Java environment (i.e. pure Java). These pieces of business logic (which we called "Gems" to give them a nice friendly face) are declarative 'functional' objects. So, the side effect of focusing on our goals was the creation of a high performance, general purpose functional language and runtime for the Java platform. Here are a few 'highlights' from our feature list: - A lazily evaluated, strictly-typed language called CAL, with many similarities to Haskell - A compiler capable of generating Java bytecode with efficient schemes - Many optimisations specific to targeting a dynamic, OO language platform, such as the JVM - A graphical language and tooling for interactively developing and testing Gems (also a good teaching tool) - Debugging features, with value inspection that doesn't force evaluation - Eclipse integration (still in progress) - Metadata on CAL entities - CALDoc (source embedded documentation) - Fully multithreaded compiler and runtime - Control of evaluation from Java, if required (suspension, resumption, exploring different parts of the result) - Dynamic compilation - use SDK to create Gems at runtime, or create ad hoc compositions that might represent specific data flows in an application - Easy integration to use Java types in CAL and to call regular Java logic - Exception support - Localisation and Internationalisation support Over the next few weeks, we will be publishing more technical information about our framework (codenamed Quark). As Business Objects is not a language or development tools vendor, we are investigating the possibilities around open sourcing the general language machinery. As part of figuring out the best way to approach this, we would be interested in hearing from anyone with a possible interest in this sort of thing (for teaching, application development, research, etc.). While we gather useful collateral, I have made a couple of videos available that we had to hand and could share quickly. Some of the content is designed for those new to the functional paradigm, but the videos do show quite a range of tool and language features. To take a look at these, point your browser at: http://homepage.mac.com/luke_e In precis, the videos demonstrate the following: Intro to Gems (parts 1 and 2) Some of the regular things you expect functional languages to do, created interactively in our graphical language designer called the Gem Cutter. We find this tool is used quite a bit by developers for experimentation and orientation when reasoning from scratch about problems. In the demo you can see real-time typing, Intellicut (our system for proposing legal compositions), a graphical way to handle partial application and recursion and a number of other features that may be interesting. CAL and ICE Shell This video is a whistlestop tour through some CAL language features as well as some of the features of the framework (as exposed through the interactive shell, ICE). In this demonstration, you can see some aspects of CAL syntax, the interface to Java types and methods, a quick demo of some tracing/debugging, a look at CALDoc, records/tuples, refactoring and some brief discussion on a range of other topics. I may post other videos over the next few days and weeks, along with papers, manuals and other artifacts, but I'll spare everyone from further announcements here lest that bugs people. [Videos are Quicktime/H264, if you aren't in the habit of watching Quicktime I believe most browsers will help you to get and install the plug-in. I can send people other formats on request, or indeed, at the original resolution which is easier to see!] Regards to all, Luke Evans Chief Scientist, VP Research. Business Objects
On Tue, Sep 26, 2006 at 09:22:21PM -0700, Luke Evans wrote:
Here are a few 'highlights' from our feature list: - A lazily evaluated, strictly-typed language called CAL, with many similarities to Haskell
Do you think that CAL would be a good target for a Haskell compiler? In other words, would it be a good idea to use CAL as an intermediate language for a Haskell -> Java byte-code compiler? Best regards Tomasz
It could be for a subset of Haskell (probably a large one). Haskell has some features that CAL does not (many just syntactic sugar, some not) we¹re actually working on a short paper to summarise the differences, primarily for people on this list. Even without a one-to-one correspondence, this might work in many situations where special lower level CAL could be generated in lieu of higher-level Haskell features. Still, you would probably be able to go a very long way with not much more than straightforward syntax transformations. Of course, use of Haskell libraries would either have to be mapped to CAL ones, or the Haskell library functions converted themselves (with this treatment applied recursively to dependees). Lwe On 9/26/06 10:52 PM, "Tomasz Zielonka" <tomasz.zielonka@gmail.com> wrote:
On Tue, Sep 26, 2006 at 09:22:21PM -0700, Luke Evans wrote:
Here are a few 'highlights' from our feature list: - A lazily evaluated, strictly-typed language called CAL, with many similarities to Haskell
Do you think that CAL would be a good target for a Haskell compiler? In other words, would it be a good idea to use CAL as an intermediate language for a Haskell -> Java byte-code compiler?
Best regards Tomasz
Well, we actually looked at doing just that in 1999. There are a number of reasons why we didn't go that route... 1. Java This was critical to us because it looked like it was becoming a very important platform for business applications. We also wanted easy cross-platform support, and to be able to leverage Java connectivity and libraries. We have since experimented with .NET and believe that Quark could be hosted on this platform too (delegating of course to its own primitive/core types). However this hasn't been a focus for us, to date. 2. Dynamic compilation and interaction with the functional subsystem Our goals required that everything ran fast and that logic could be assembled piecemeal under application control, from modules defining the abstractions of a domain language/model for whatever purpose the application was using Quark. As you can see in the videos, the Gem Cutter is truly interactive, with type information being computed on-the-fly and new functional entities created quickly and smoothly. Yet the Gem Cutter is not special in that regard. Any Java application can discover functional components (Gems) and snap these together programmatically into logic to represent something useful that is immediately executable. This is a fundamental feature we desired. 3. Simple syntax, appropriate features The original intention was to not actually focus so much on the syntax of the language, because we were more interested in the runtime semantics under control of Java, with programmatic construction of new functional logic. We had CAL from the very beginning as a convenient way to define our 'Gems', but very much had a design principle to keep the language as simple as possible. Haskell really has different goals here, because it is catering to a wide range of requirements and the need for different styles. It's also an excellent place to experiment and develop the genre, and benefits from a rich palette of syntactic forms. Anyway, as time went on, we realised the need to develop CAL into a proper "developer's language", so it grew a somewhat richer syntax and of course this was driven by our particular priorities and requirements. For example, because the Quark framework is motivated by the desire to create reusable business logic, and because this is frequently record-oriented, a powerful record capability was a fairly early necessity. Similarly, our requirements around tight integration with Java and Java's own data connectivity (and other libraries) meant that our 'foreign function' interface had to be as simple as possible (which also makes it reasonably sophisticated when it comes to the compiler features that really make it work). In terms of syntax, if we had decided to adopt Haskell from the beginning, we would almost certainly now still have a language that wasn't truly Haskell, because of the forces of our primary concerns acting on some quite fundamental language features. Of course, we may have had something more similar in some areas, and therefore more able to compile certain code without transforming anything, but this is a matter of degrees. Though moot, the extensions over a Haskell 98 base that we would have deemed necessary would also likely have been based on different choices to those emerging in the intervening time and possibly being considered for adoption in future standard Haskell. While I hope the foregoing gives something of a background to our choice, I'm pleased to offer a document that deals with these issues in a little more detail. My colleague, Bo Ilic, has completed a paper entitled "CAL for Haskell Programmers", which is available here: http://homepage.mac.com/luke_e/FileSharing13.html. Lwe On 9/28/06 5:13 AM, "Paul Hudak" <paul.hudak@yale.edu> wrote:
I suspect many of us are dying to ask: Why not just use Haskell?
-Paul
Luke Evans wrote:
Re: [Haskell] ANN: Efficient, dynamically compiled, lazy functional semantics on JVM, having tight integration with the Java language It could be for a subset of Haskell (probably a large one). Haskell has some features that CAL does not (many just syntactic sugar, some not) we¹re actually working on a short paper to summarise the differences, primarily for people on this list. Even without a one-to-one correspondence, this might work in many situations where special lower level CAL could be generated in lieu of higher-level Haskell features. Still, you would probably be able to go a very long way with not much more than straightforward syntax transformations. Of course, use of Haskell libraries would either have to be mapped to CAL ones, or the Haskell library functions converted themselves (with this treatment applied recursively to dependees).
Lwe
On 9/26/06 10:52 PM, "Tomasz Zielonka" <tomasz.zielonka@gmail.com> <mailto:tomasz.zielonka@gmail.com> wrote:
On Tue, Sep 26, 2006 at 09:22:21PM -0700, Luke Evans wrote:
Here are a few 'highlights' from our feature list: - A lazily evaluated, strictly-typed language called CAL, with many similarities to Haskell
Do you think that CAL would be a good target for a Haskell compiler? In other words, would it be a good idea to use CAL as an intermediate language for a Haskell -> Java byte-code compiler?
Best regards Tomasz
I'd like to summarize why you chose to create a new language instead of a Haskell->JVM backend and to throw in ideas which address these points in Haskell.
1. Java 2) Why did we create CAL instead of just using Haskell? 2)b) CAL is a low-risk choice for business applications 2)c) Programs do not need to be entirely written, or even primarily written in CAL 2)d) CAL seeks to be as comfortable as possible for mainstream developers to use 3. Simple syntax, appropriate features
The functional stuff should be integrated as tightly as possible with Java which means a foreign function interface that blurs the lines and using the JVM runtime and its garbage collector. When it goes to marshaling, the Haskell FFI does not blur anything. Perhaps this could be remedied by adding a foreign data interface: foreign data c-struct Foo { bar :: Int, bar2 :: Bool } foreign export foobar :: Foo -> Int foobar f = (if bar2 f then id else negate) $ bar f foreign data java-object Foo { bar :: Int, bar2 :: Bool, foobar :: Foo -> Int} etc. so that foreign data things become first class values. In the paper "Strongly typed memory areas", something similar is done for memory areas. The type of such areas gets a special kind different from *. For conservative reasons, this could be done here, too. With some form of kind polymorphism, one could trash all CInt and the like and have the kind annotation (Int :: forall * . *). The function (+) f.i. could then be specialized for (CInt :: C) and Haskell (Int :: *). The main problem is an embedding of System F into Java (or any other foreign language, strictly speaking -fvia-C already provides such an embedding) with the additional property that "normal functions" are exported *without marshaling*. Viewed as a mathematical map, the embedding should be a projection, so to speak. Maybe a kind system can help: everything that can be exported transparently gets a "simple" kind and is automatically exported/imported. So (foreign export) and (foreign import) should be dropped altogether. Those things using System F features (polymorphism, rank-n-types etc.) get a complicated kind which clearly marks them as "to be marshaled". One tricky point is how to marshal laziness and it looks like Business Objects solved this in a very pragmatic way (i.e. Java objects are always strict)? The point
3. CAL performance also falls under this category: no marshaling means compiling to loops etc.
To summarize, there is a need for a very sophisticated foreign language interface / foreign run-time system integration.
2. Dynamic compilation
With lambdabot, "Dynamic applications from the ground up" and hs-plugins, there are some things underway. Clean seems to offer a potent solution, where one can pattern match against types: foo :: Dynamic -> Dynamic -> Dynamic foo (Dyn f :: a -> b) (Dyn x :: a) = Dyn (f a) :: b foo _ _ = error "foo falls through" Values of type Dynamic are/can be serialized which allows one to serialize functions (!). I don't know exactly, but I think its even possible to use the Clean compiler as a library and compile things at runtime to values of type Dynamic. The paper "Towards a strongly typed operating system" mentions some things in that direction. Maybe there is no need for another language construct to allow the pattern matching of types, some approach to generic programming à la Hinze already could do things like that. Somehow, it becomes a problem of representing and manipulating types at runtime.
Leaving aside the syntactic differences, I think that these two points are actually the essence of what separates CAL from Haskell? A minor motivation why busy enterprise business integration applications need the two mentioned features might be: 1. Java is the "official machine architecture of business applications". Point. At least, so it seems to me. The need for a foreign language interface (software reuse!) is obvious, then. 2. Software for a banking-house that handles all the financial transactions just cannot be shut down for bug fixing or rewrites. The same goes for lambdabot :) Regards, apfelmus
Just a quick addendum to this. In order not to be posting too much (essentially non-Haskell stuff) to this list, I have created a blog at: http://www.quarkframework.blogspot.com/ I intend to post most news there as we get technical material (and the software!) together to share with the community. We¹ll probably still post some Haskell comparison stuff here. This blog is likely to be temporary, but helps us with early discussion and dissemination of news prior to anything more permanent being set up. Lwe On 9/26/06 9:22 PM, "Luke Evans" <luke.evans@businessobjects.com> wrote:
Greetings denizens of the Haskell mailing list!
Though this is not strictly an announcement about Haskell, it might be of interest to a number of folk here.
The Research Group here at Business Objects has been beavering away since 1999 on a suite of technologies to allow us to represent certain kinds of business logic as reusable, composable pieces. One of the primary requirements was that a general runtime for such objects be embeddable within the Java environment (i.e. pure Java). These pieces of business logic (which we called "Gems" to give them a nice friendly face) are declarative 'functional' objects. So, the side effect of focusing on our goals was the creation of a high performance, general purpose functional language and runtime for the Java platform.
Here are a few 'highlights' from our feature list: - A lazily evaluated, strictly-typed language called CAL, with many similarities to Haskell - A compiler capable of generating Java bytecode with efficient schemes - Many optimisations specific to targeting a dynamic, OO language platform, such as the JVM - A graphical language and tooling for interactively developing and testing Gems (also a good teaching tool) - Debugging features, with value inspection that doesn't force evaluation - Eclipse integration (still in progress) - Metadata on CAL entities - CALDoc (source embedded documentation) - Fully multithreaded compiler and runtime - Control of evaluation from Java, if required (suspension, resumption, exploring different parts of the result) - Dynamic compilation - use SDK to create Gems at runtime, or create ad hoc compositions that might represent specific data flows in an application - Easy integration to use Java types in CAL and to call regular Java logic - Exception support - Localisation and Internationalisation support
Over the next few weeks, we will be publishing more technical information about our framework (codenamed Quark). As Business Objects is not a language or development tools vendor, we are investigating the possibilities around open sourcing the general language machinery. As part of figuring out the best way to approach this, we would be interested in hearing from anyone with a possible interest in this sort of thing (for teaching, application development, research, etc.).
While we gather useful collateral, I have made a couple of videos available that we had to hand and could share quickly. Some of the content is designed for those new to the functional paradigm, but the videos do show quite a range of tool and language features. To take a look at these, point your browser at: http://homepage.mac.com/luke_e
In precis, the videos demonstrate the following:
Intro to Gems (parts 1 and 2) Some of the regular things you expect functional languages to do, created interactively in our graphical language designer called the Gem Cutter. We find this tool is used quite a bit by developers for experimentation and orientation when reasoning from scratch about problems. In the demo you can see real-time typing, Intellicut (our system for proposing legal compositions), a graphical way to handle partial application and recursion and a number of other features that may be interesting.
CAL and ICE Shell This video is a whistlestop tour through some CAL language features as well as some of the features of the framework (as exposed through the interactive shell, ICE). In this demonstration, you can see some aspects of CAL syntax, the interface to Java types and methods, a quick demo of some tracing/debugging, a look at CALDoc, records/tuples, refactoring and some brief discussion on a range of other topics.
I may post other videos over the next few days and weeks, along with papers, manuals and other artifacts, but I'll spare everyone from further announcements here lest that bugs people.
[Videos are Quicktime/H264, if you aren't in the habit of watching Quicktime I believe most browsers will help you to get and install the plug-in. I can send people other formats on request, or indeed, at the original resolution which is easier to see!]
Regards to all,
Luke Evans Chief Scientist, VP Research. Business Objects
_______________________________________________ Haskell mailing list Haskell@haskell.org http://www.haskell.org/mailman/listinfo/haskell
participants (4)
-
apfelmus＠quantentunnel.de -
Luke Evans -
Paul Hudak -
Tomasz Zielonka