On Sat, Jun 16, 2007 at 06:14:38PM -0400, Christopher Lane Hinson wrote:

While we're on the topic of IDE features.

I wish to have an editor that ran GHC[I] every few seconds or so, and underlined sites of syntax errors in red. This would save me a lot of back-and-forth. If an editor did this, I would switch (from kate) in a heartbeat.

This has been mentioned before in only one place that I can find:

http://compilers.iecc.com/comparch/article/99-09-067

Yeah, I could do it myself.

Shim *almost* does this; it has a key command to run GHC and highlight all the syntax and type errors, on mouseover displaying the error message. Someone experienced with Emacs could probably set up the automation easily... Stefan

Indeed, that's what I forgot to mention, Resharper in Visual Studio 2005 does that for C#, IntelliJ & Eclipse for Java. You rarely need compilation, its syntax checker runs inplace and incrementally and shows you the errors and warning in the right margin. That saves you a lot of time. For Haskell, the Eclipse plugin should do something like that, at least every time you save. http://eclipsefp.sourceforge.net But I never got it to work with GHC (only GHCI/Hugs), I tried on 3 different machines (on Windows). I also guess this should not be too difficult to implement in Emacs... for someone how knows Emacs that is. -----Original Message----- From: haskell-cafe-bounces@haskell.org [mailto:haskell-cafe-bounces@haskell.org] On Behalf Of Christopher Lane Hinson Sent: Sunday, June 17, 2007 12:15 AM To: haskell-cafe@haskell.org Subject: Re: Re[2]: [Haskell-cafe] IDE? While we're on the topic of IDE features. I wish to have an editor that ran GHC[I] every few seconds or so, and underlined sites of syntax errors in red. This would save me a lot of back-and-forth. If an editor did this, I would switch (from kate) in a heartbeat. This has been mentioned before in only one place that I can find: http://compilers.iecc.com/comparch/article/99-09-067 Yeah, I could do it myself. --Lane _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Hi

And that's why IMHO for Windows users, one needs a friendly IDE to get started with Haskell in a modern way. And the Windows version should comply to the Windows styleguides.

I use TextPad and WinHugs, you might find Visual Studio meets your needs better. Both those options are properly Windows native. Thanks Neil

Thanks for the nice info. I'm going to give it another try then... When I said I don't want to learn Emacs, I meant not learning its LISP architecture with the goal of creating my own custom Emacs... OT: The reasons I'm looking at Haskell are: - the object oriented approach failed for me when working on large projects in medium to large teams. OO works reasonably if you have a good design and follow some rules, but no language enforces those rules (yet), so aliasing and unwanted side effects leak all over and you spend a lot of time debugging. - I noticed I was using the "immutable" pattern a lot for solving problems lately... So I got steered automatically towards the FP world, which is immutable by definition. That is, excluding the IO "world" I guess, but as far as I understand, monadic IO is also pure, in the sense it has no aliasing, unless using unsafePerformIO. But since I'm a Haskell newbie, I'm not going to claim I understand monads! ;-) - After 20 years of hitting keyboards like a madman to reach yet another crazy deadline, I got myself RSI in both arms (Workrave helps, but it's too late :\) So I want to spend more time thinking than typing, and I certainly don't want to type boiler plate code. But most OO languages are FULL of boiler plate code, and are much more verbose than Haskell. - A last but not least, if all goes well, I will be teaching an undergrad "applied mathematics for videogame development" course. This will be very basic and practical mathematics. These students love games, so I want to let then *play* with the math using a "mathematical" programming language. If the school approves it (which is unlikely because the students will also be learning C++ => confusing), this will most likely be very basic Haskell. Being an old school C/C++ developer, I would find it unfortunate that the students don't get to see some FP, because I believe FP can play an important part in the future of videogame and even business software development (even the "giants" in my industry seem to believe that in some degree: see e.g. http://blogs.msdn.com/charlie/archive/2007/01/26/anders-hejlsberg-on-linq-an d-functional-programming.aspx and http://www.st.cs.uni-sb.de/edu/seminare/2005/advanced-fp/docs/sweeny.pdf) So, if I decide to use Haskell, I want my students to get playing right away using a nice IDE, because the young videogame generation is even more spoiled and impatient than I am :) Peter V No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.472 / Virus Database: 269.9.0/852 - Release Date: 17/06/2007 08:23

Having just presented a case for the possible rationality of the irrational decision of creating an Emacs-like IDE in Haskell, I wonder if we should not be even more irrational and contemplate the possibility of using Haskell to create a radically different kind of IDE. New technologies are often used to imitate and reimplement the artefacts of previous technologies. In my house, I have a gas-powered fireplace that goes out of its way to present itself as a Victorian coal-powered fireplace that in turn was just a clone of the wood-powered fireplace of yore. The killer application of natural gas as a home heating technology is not, however, fake fireplaces but rather something completely new and way more convenient: clean, reliable and affordable central heating. Emacs was designed in a very different age, with technical limitations and therefore design constrains quite different from the current ones: Individual versus collective development: In the 70/80ties, cooperative development meant having an expert spending months writing a program in C and then distributing the source on tape to a few hand-picked peers. These days we create whole encyclopaedias cooperatively combining the skills of both traditional academic experts and high school students with too much time on their hands. In the Emacs-age the emphasis was necessarily on supporting highly-skilled individual programmers. Now, it should be on supporting distributed, large-scale, communal development; keeping in mind that not all potential contributors measure 1000 milliOlegs or milliSimons on the Haskell IQ scale. Local versus ubiquitous access: In the 70/80ties, people were lucky to have access to a computer terminal, applications were naturally designed to be run and configured locally. These days we tend to have multiple computing devices and we expect to be able to access our data and software from everywhere. Local versus distributed execution: Application these days are often run in a distributed environment. Web applications for example are multi-tier with often one or more servers per tier. Simply copying the Emacs model might therefore not do much to address the needs of a different technological age. So where should we look for inspiration? Maybe not very far, the seeds of a possible revolution might have already been sown. LambdaBot, hpaste and hoogle as well as the Haskell Wiki and indeed even this mailing list can all be thought as elements of a cooperative, ubiquitous, customisable, zero-installation, distributed and integrated (that should be enough buzzwords for anybody) development system. What we need is to connect the dots and recognise the possibilities that might arise from an integration of all these tools in a World-Wide Functional Web. So, if we let our imagination loose and dream about an IDE for the Internet age, what would it look like? My personal dream goes a bit like this: Simon wakes up in the morning, with an idea for a new GHC extension to facilitate parallel computing. He starts up his Web Browser to access the Haskell Integrated Development Environment. Visually, it looks pretty much like Eclipse. He can choose among a set of customisable perspectives for development, editing, browsing, etc. Each perspective is composed by multiple views displaying different kind of data (Haskell modules, stack traces, etc). Internally the IDE is based on a 'remote functional application framework': a simple, flexible way of composing statically-typed functions in a networked environment. Functions (or possibly first-class modules) are made available on the Internet as source code and/or endpoints (opaque network-accessible implementations). Functions can take as parameters additional functions. The IDE window manager is one such function that takes as parameters the functions that implements the perspectives. In turn, perspectives take as parameters the functions that implement the views and so on recursively. The Haskell editor, for example, take as a parameter a set of String -> [HaskellSymbol] functions that are used to provide code completion functionality. One such implementation is provided remotely by the Hoogle search engine. Simon has also configured its IDE to use two different function compositions to compile its code, one is made of stable versions of the GHC modules (parser, Haskell to Core-Haskell translator, code generators) , another uses more experimental versions of the same modules. Being a social, well-meaning guy, Simon marks his new module as 'public'. It then immediately appears on the Haskell Code Wiki. The code Wiki is a combination of a classic Wiki, in the sense that it allows for unrestricted Web based editing, and a revision control system as it supports multiple branches per page and the definition of named sets of specific versions of multiple Wiki pages (know as “releases” in the pre-distributed IDE age). Oleg notices the appearance of Simon's module, as it appears in the “Current Haskell News” view of his Haskell Browsing perspective and in a matter of a few hours publishes on the Wiki a new implementation that demonstrates that a compiler extension is not really required as, making use of a clever Peano encoding of type-level delimited-continuations, he can provide the same functionality and then some with a 40 lines library running on top of the existing GHC compiler. The publishing of his code causes a great commotion and induces many would-be Haskell programmers to give up in desperation and go back to Visual Basic. Some are so intellectually humiliated that give up programming entirely and move into the used car sale business. Luckily, later in the day (PhD students do not wake up early), Neil makes available its version that has half of the functionality of Simon's, not to mention Oleg's, but is explained in plain English, requires only Haskell 98 and compiles under YHC. As he is fond of Windows, he also makes a point of replacing all the "/"s in the source code with "\"s. A few days later, Titto, a perpetual Haskell beginner, takes Neil's version, as it is the only one that he can almost make sense of, changes the "\"s back to "/"s and compiles it in his GHC-based environment. Titto runs a Web site that is implemented using the same remote functional application framework used by his IDE. The Web site is made of a set of function/modules (Web front-end, application server, persistence back end) distributed on top of a few servers, for performance and reliability. With a few clicks, he replaces one of the existing Web site modules with a new implementation based on Neil's parallel library and upgrade its system without any of his users noticing. Up to you now, what is your dream? Best, titto

On Monday 18 June 2007 16:13:02 you wrote:

I just did a quick read through of your dream and I'm not going to say either way with it. But I would like to point out, just to make sure you've considered it, that my dream--or maybe my reality--involves being able to code without the requirement of a network connection.

Bryan

That's a very fair point. An "ubiquitous" IDE should also work when disconnected. Luckily, Web applications can be designed to work offline. In effect, modern AJAX based web apps already execute most of their code on the client side, going back to the server only when absolutely necessary and there are now technologies like Google Gear to efficiently store/access information locally so further reducing dependency on remote services. Best, titto

Hi Claus, On Monday 18 June 2007 18:14:58 Claus Reinke wrote:

...

Having just presented a case for the possible rationality of the irrational decision of creating an Emacs-like IDE in Haskell, I wonder if we should not be even more irrational and contemplate the possibility of using Haskell to create a radically different kind of IDE.. New technologies are often used to imitate and reimplement the artefacts of previous technologies.

don't underestimate those "previous" technologies, though. given your you're almost certain to find this interesting: Croquet is a powerful open source software development environment

Thanks for the reference. I actually knew about Croquet but I thought of it mostly as an "open-source second life" because of its emphasys on shared 3D worlds but you are quite right, it might also be useful for cooperative software development. I must admit that my dream doesn't go so far, I was more thinking about Web/Web services kind of technology to integrate distributed traditional development text-based tools (editors, compilers, etc.) plus a configurable Web based UI.

if you look closely, you'll see that croquet is implemented in squeak, which in turn is a re-implementation of one of the ancient smalltalks.

squeak is by no means the ideal implementation language for this kind of project, nor am i completely convinced by the synchronous approach used for croquet. but while implementation of croquet in squeak is obviously doable, i see various difficulties for doing the same in haskell.

where squeak is too dynamic/imperative/flexible, haskell is too static/unreflective/limited (ever tried to pass functions through haskell's i/o interface?

Is this really a limitation of the language proper or just of its implementations? Is there any fundamental reasons why Haskell functions/closures cannot be serialised? I believe that this is precisely what the distributed version of GHC used to do. Most languages, even Java, have a reflection capability to dynamically inspect an object. It is surprising that Haskell doesn't offer it. ...

persistence [many starts, no finish

Have you checked the prevayler-inspired approach implemented in HAppS ?

, but see Clean's first class i/o]

What advantages does it provide?

reflection/meta-programming [Data/Typeable, template haskell..; meta ml?]). one dream would be successors to haskell and croquet so that croquet' could be implemented in haskell''.

Is just the lack of reflection in Haskell that you miss? Or there are other things as well? best, titto

hi titto,

I actually knew about Croquet but I thought of it mostly as an "open-source second life" because of its emphasys on shared 3D worlds but you are quite right, it might also be useful for cooperative software development.

the first corporation betting its money on croquet is Qwaq: http://croquetconsortium.org/index.php/Current_Projects#Qwaq_Projects ..proprietary Croquet-based collaborative spaces enabling our customers to integrate their existing 2D media, 3D models and simulations. These spaces enable interlinked virtual conference rooms dedicated to each area of corporated interest.. cooperative software development is just one instance of that (i seem to recall a reference to ubuntu linux using croquet as a coordination tool, but my own network connection is highly uncooperative at the moment..; sorry, seems i have to send this email with without adding the necessary references/urls below; most of the keywords should show up on google, but most connections are timing out on me..).

...

where squeak is too dynamic/imperative/flexible, haskell is too static/unreflective/limited (ever tried to pass functions through haskell's i/o interface?

Is this really a limitation of the language proper or just of its implementations?

both, though the/some implementations are further ahead than the "language proper". but if those implementation branches are not consolidated into the main implementations and the language that is used to write the standard libraries, they will bitrot (some have).

Is there any fundamental reasons why Haskell functions/closures cannot be serialised?

no, and that is part of the problem: the language would need to be extended, but the academically interesting issues have been tackled, all that is left is a lot of work (that is why these things would be so valuable: complex implementation machinery, controlled by very small language extensions, sometimes even language simplifications, such as lifting existing restrictions on i/o), preferably with very good planning, so that all the work does not become useless right after it is finished. there isn't much hope that this issue is going to be settled via the usual academic funding sources.

I believe that this is precisely what the distributed version of GHC used to do.

yes, distributed systems have some of the same issues. but note that you said "version of ghc", not "ghc", not "haskell". even what is there hasn't been folded back into the mainline, nor does it keep up with other mainline developments.

Most languages, even Java, have a reflection capability to dynamically inspect an object. It is surprising that Haskell doesn't offer it.

it has to be done with care, or it will invalidate *all* your nice reasoning about haskell programs. random example reify (f . g) == [| f . g |] =/= [| \x-> f (g x) |] == reify (\x-> f (g x)) reification is not a referentially transparent program context.

...

persistence [many starts, no finish

Have you checked the prevayler-inspired approach implemented in HAppS ?

no, do you have a reference? but i meant "orthogonal persistence", as in all program parts can persist, including functions, thunks, types,.. once you start going down that route, the rigid globally static/dynamic distinction quickly becomes meaningless (instead one has locally static/dynamic phases of evaluating program parts, ie, one does a dynamic type check in each static phase, and if that succeeds, the immediately following dynamic phase will be type correct without further "runtime" type checks). if you can store, retrieve, visualise and edit collections of functions, you already have first class modules (there was an early thesis on a persistent algol, with a title roughly like "persistent first-class procedures are enough"). and the classical distinctions between language and programming environment begin to blur considerably.

...

, but see Clean's first class i/o]

What advantages does it provide?

please see their papers. but in essence, they approach the same problem from a different direction: if one doesn't limit i/o to fully evaluated non- functions, as haskell does, one can make anything persist, just by writing it to a file (which thereby becomes a typed object rather than a string of bytes), and one can distribute anything, including code, by writing it accross the network. it raises all the same issues as orthogonal persistence: type Dynamic is needed when checking that input is of the type it claims to be, types need to be stored/validated via some clearing house system (there's no guarantee that the program that reads an expression has compatible types to the program that wrote it, types and compilers evolve, ..).

...

reflection/meta-programming [Data/Typeable, template haskell..; meta ml?]). one dream would be successors to haskell and croquet so that croquet' could be implemented in haskell''.

Is just the lack of reflection in Haskell that you miss? Or there are other things as well?

there is more, but i thought my list was already long and fundamental enough?-) sorry again about the lack of references in this email, claus

Prevayler is an efficient and very simple way of providing application state persistency.

Essentially: - all the state is kept in memory, in native language data structures - whenever a write transaction is performed the system automatically serialises a description of the transaction so that it can be replayed later. - occasionally the whole state can be serialised to produce a snapshot - when the system restarts it reads in the last snapshot and replays all transaction since the last snapshot.

thanks for the explanation. but this means that there is one process permanently running and holding on to all the application state in main memory. if you want to shut down, move, or replicate that process, you need to get hold of its data, and if you can't communicate its data to worker processes, you have to communicate the work into the repository process. all limited by serialisation, unless that aspect is kept implicit by accross-the-board support for persistence and first-class communications. its a bit like pointers in haskell: just about everything a haskell program touches is a pointer (unboxing or shared constants are optimisations), but usually, haskell programs talk about the data being referenced, not about the references. if you want to share or forget the data, you move the references, and the system handles memory management behind the scenes. with orthogonal persistence, everything a program touches might persist, but usually, programs talk about the data being persistet (?), not about whether that data is currently temporary or in long-term storage. if you want to move such data between processes or storage areas, you move the reference, and the system handles serialisation/ communication/deserialisation behind the scenes.

Is this orthogonal persistence?

it doesn't seem so. but it does try to address the same issues, in an environment limited by its lack of support for orthogonal persistence. i used to have a long list of references to the area here: http://www.cs.kent.ac.uk/people/staff/cr3/bib/bookshelf/Persistence.html which might still give some overview of titles, but note that almost all of the urls are out-of-date or have disappeared entirely. lets see whether i can find some of them again: St.Andrews seems to have preserved some of its older groups' information, including publication lists (some of the publications are online as well): http://www-old.cs.st-andrews.ac.uk/research/publications.php?keyword=persist... http://www-old.cs.st-andrews.ac.uk/research/publications.php?keyword=reflect... and has a wiki, too: http://www-systems.cs.st-andrews.ac.uk/wiki/Persistence http://www-systems.cs.st-andrews.ac.uk/wiki/Linguistic_Reflection and there were some survey papers, including: Orthogonally Persistent Object Systems (1995) Malcolm Atkinson, Ronald Morrison; VLDB Journal http://citeseer.ist.psu.edu/atkinson95orthogonally.html many of the topics talked about in those papers will be more familiar to todays haskellers than to the general programming language audience of the time (first class procedures, type abstraction, existentials, runtime code generation, ...). but note that most of the material is old, so code might be in terms of algol variants, not haskell, compile-time reflection, not template haskell, runtime reflection, not meta-ml, reflection where generic programming would be sufficient, and so on.. and todays haskellers are certainly aware of the issues addressed in that old work on unifying database and programming language research: getting the advantages of databases into production code without the hassle of dealing with two separate and incompatible semantics and systems. instead of seeing databases as external to programming languages, they become integrated structures/features of the language. hth, claus

...

with orthogonal persistence, everything a program touches might persist, but usually, programs talk about the data being persistet (?), not about whether that data is currently temporary or in long-term storage. if you want to move such data between processes or storage areas, you move the reference, and the system handles serialisation/ communication/deserialisation behind the scenes.

This is interesting, could you elaborate on it? How would you get data to move around by moving its reference?

more elaboration than the various papers, surveys, and phd theses listed in the references i provided?-) the idea is that i give you the reference, and you take care of looking at the data behind it, without me having to serialise the contents;-) but ok, lets see whether i can get the idea accross by example: a) suppose you want to move some x from list a to list b do you get the type of x, devise a type-specific traversal to serialise x from the source, move the flattened data from a to b, and deserialise x in the target? or do you just write: test = move ([1..4],[3..5]) move (x:as) b = (as,x:bs) b) suppose you want to move some x from concurrent haskell process a to concurrent haskell process b do you get the type of x, devise a type-specific traversal to serialise x from the source, move the flattened data from a to b, and deserialise x in the target? or do you write something like: test = do { av<-newEmptyMVar; bv<-newEmptyMVar; forkIO (putMVar av [1..]); forkIO (takeMVar bv >>= print . take 10); move av bv } move av bv = takeMVar av >>= putMVar bv c) suppose you want to move some x from os process a to os process b do you get the type of x, devise a type-specific traversal to serialise x from the source, move the flattened data from a to b, and deserialise x in the target? yes. and if the type is not serialisable, you're stuck. d) suppose you want to move some x from os process a to an os file, for later retrieval in process b do you get the type of x, devise a type-specific traversal to serialise x from the source, move the flattened data from a to b, and deserialise x in the target? yes. and if the type is not serialisable, you're stuck. now, why are c/d so much more troublesome than a/b? i don't care whether the x to be moved is an integer, a matrix, a function, or the list of primes - i just want it to be moved from a to b. or rather, i move the reference to x, and the runtime system moves whatever representation is behind that, if a move is necessary, and without ever exposing that internal representation. and if i happen to move x into a long-term storage area, it will persist there for future reference, without further ado. much more about that idea in the papers i mentioned. or, if you prefer something more recent, have a look at the Clean papers: http://www.st.cs.ru.nl/Onderzoek/Publicaties/publicaties.html a selection of entries related to dynamics and first-class i/o: 1997: 4. Pil, Marco, First Class File I/O 2003: 7. Arjen van Weelden and Rinus Plasmeijer. Towards a Strongly Typed Functional Operating System. 2003: 6. Martijn Vervoort and Rinus Plasmeijer. Lazy Dynamic Input/Output in the lazy functional language Clean 2004: 21. Arjen van Weelden, Rinus Plasmeijer. A Functional Shell that Dynamically Combines Compiled Code. hth, claus

Hi everybody, What is the situation with respect to reification of function/thunks in Haskell? Does any current implementation support it ? And, is there any plan for GHC to support it? Claus's comments on this, follow. titto On Monday 18 June 2007 23:45:23 Claus Reinke wrote:

...

Is there any fundamental reasons why Haskell functions/closures cannot be serialised?

no, and that is part of the problem: the language would need to be extended, but the academically interesting issues have been tackled, all that is left is a lot of work (that is why these things would be so valuable: complex implementation machinery, controlled by very small language extensions, sometimes even language simplifications, such as lifting existing restrictions on i/o), preferably with very good planning, so that all the work does not become useless right after it is finished. there isn't much hope that this issue is going to be settled via the usual academic funding sources.

...

Most languages, even Java, have a reflection capability to dynamically inspect an object. It is surprising that Haskell doesn't offer it.

it has to be done with care, or it will invalidate *all* your nice reasoning about haskell programs. random example

reify (f . g) == [| f . g |] =/= [| \x-> f (g x) |] == reify (\x-> f (g x))

reification is not a referentially transparent program context.

Hi Bulat, do you mean that as the type information is used only at compilation time and then thrown away there is no way of getting it back at execution time? best, titto On Wednesday 20 June 2007 16:33:12 Bulat Ziganshin wrote:

Hello Pasqualino,

Wednesday, June 20, 2007, 11:30:32 AM, you wrote:

...

...

...

Most languages, even Java, have a reflection capability to dynamically inspect an object. It is surprising that Haskell doesn't offer it.

how about asm? :) there are no OOP objects in Haskell, each name is just an address of memory area. all operations are checked statically (at compile time). reflection capabilities may be only handmade - you can get any type info via hidden class dictionary (see http://homepages.inf.ed.ac.uk/wadler/papers/class/class.ps.gz for details of type classes implementation)

Pasqualino 'Titto' Assini wrote:

Is there any fundamental reasons why Haskell functions/closures cannot be serialised?

I believe that this is precisely what the distributed version of GHC used to do.

Most languages, even Java, have a reflection capability to dynamically inspect an object. It is surprising that Haskell doesn't offer it.

Inspecting functions is not referentially transparent. In Haskell, function equality is extensional, i.e. two functions are equal when their results are equal on all arguments. Intensional equality would mean that functions are equal when they have the same representation. If you allow a function serialize :: (Int -> Int) -> String that can give different results on intensionally different functions, you may not expect equations like f (*3) == f (\n -> n+n+n) to hold anymore (because f might inspect its argument). Also, having "serialize" somehow check whether intensionally different arguments are extensionally the same and should have a unique serialization is no option because this problem is undecidable. Regards, apfelmus

On Thu, 21 Jun 2007, Pasqualino 'Titto' Assini wrote:

Thanks for the explanation.

But, doesn't this simply mean that the correct signature would be:

serialize :: (Int -> Int) -> IO String

to take in account the fact that serialise really use 'external' information that is not in the domain of pure Haskell functions?

I'm afraid not. The beauty of the IO monad is that it permits equational reasoning over I/O operations. E.g. because of the definition print = putStrLn . show the compiler can freely inline calls to print. Although there's I/O involved, equational reasoning is still valid: the inlined call will behave in the same way as the original code. Hence, the compiler does not have to be aware of IO and treat it in a different way. Your serialize function does not have that property. You don't want its argument to be inlined or otherwise replaced with an equivalent expression. Tom

Having "serialize" in the IO monad would do no harm as usually one serialise precisely to output a value :-)

So, is it correct to conclude that there is no theoretical reason why Haskell cannot have a built-in reification/serialisation facility?

titto

On Wednesday 20 June 2007 17:05:04 apfelmus wrote:

...

Pasqualino 'Titto' Assini wrote:

...

Is there any fundamental reasons why Haskell functions/closures cannot be serialised?

I believe that this is precisely what the distributed version of GHC used to do.

Most languages, even Java, have a reflection capability to dynamically inspect an object. It is surprising that Haskell doesn't offer it.

Inspecting functions is not referentially transparent. In Haskell, function equality is extensional, i.e. two functions are equal when their results are equal on all arguments. Intensional equality would mean that functions are equal when they have the same representation. If you allow a function

serialize :: (Int -> Int) -> String

that can give different results on intensionally different functions, you may not expect equations like

f (*3) == f (\n -> n+n+n)

to hold anymore (because f might inspect its argument). Also, having "serialize" somehow check whether intensionally different arguments are extensionally the same and should have a unique serialization is no option because this problem is undecidable.

Regards, apfelmus

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- Tom Schrijvers Department of Computer Science K.U. Leuven Celestijnenlaan 200A B-3001 Heverlee Belgium tel: +32 16 327544 e-mail: tom.schrijvers@cs.kuleuven.be

On Thu, 21 Jun 2007, Pasqualino 'Titto' Assini wrote:

Hi Tom,

On Thursday 21 June 2007 08:59:42 Tom Schrijvers wrote:

...

On Thu, 21 Jun 2007, Pasqualino 'Titto' Assini wrote:

...

Thanks for the explanation.

But, doesn't this simply mean that the correct signature would be:

serialize :: (Int -> Int) -> IO String

to take in account the fact that serialise really use 'external' information that is not in the domain of pure Haskell functions?

I'm afraid not. The beauty of the IO monad is that it permits equational reasoning over I/O operations. E.g. because of the definition

print = putStrLn . show

the compiler can freely inline calls to print. Although there's I/O involved, equational reasoning is still valid: the inlined call will behave in the same way as the original code. Hence, the compiler does not have to be aware of IO and treat it in a different way.

Your serialize function does not have that property. You don't want its argument to be inlined or otherwise replaced with an equivalent expression.

Is it so?

I understand that, depending on what the compiler does the result of :

do let f = (*) 2 print $ serialise f

might differ as, for example, the compiler might have rewritten f as \n -> n+n.

But, why would that make equational reasoning on serialise not valid?

Isn't that true for all functions in the IO monad that, even when invoked with the same arguments, they can produce different results?

Not if you take the ``state of the world" to be part of the arguments. If two programs behave differently for the same arguments and the same state of the world, then they're not equivalent. You do want your compiler to preserve equivalence, don't you? Tom -- Tom Schrijvers Department of Computer Science K.U. Leuven Celestijnenlaan 200A B-3001 Heverlee Belgium tel: +32 16 327544 e-mail: tom.schrijvers@cs.kuleuven.be

Tom Schrijvers wrote:

...

...

I understand that, depending on what the compiler does the result of :

do let f = (*) 2 print $ serialise f

might differ as, for example, the compiler might have rewritten f as \n -> n+n.

But, why would that make equational reasoning on serialise not valid?

Isn't that true for all functions in the IO monad that, even when invoked with the same arguments, they can produce different results?

Not if you take the ``state of the world" to be part of the arguments. If two programs behave differently for the same arguments and the same state of the world, then they're not equivalent. You do want your compiler to preserve equivalence, don't you?

You can put the internal representation of the argument into the "state of the world".

That wouldn't make a difference. If, from the pure Haskell point of view we can't tell the difference between two expressions that denote the same function, then operations in the IO monad should not be able to do so either. Otherviews a whole lot of program transformations based on rewriting of expressions would be invalid. How do you account for that? Tom -- Tom Schrijvers Department of Computer Science K.U. Leuven Celestijnenlaan 200A B-3001 Heverlee Belgium tel: +32 16 327544 e-mail: tom.schrijvers@cs.kuleuven.be

On Thursday 21 June 2007, Tom Schrijvers wrote:

...

That wouldn't make a difference. If, from the pure Haskell point of view we can't tell the difference between two expressions that denote the same function, then operations in the IO monad should not be able to do so either.

This doesn't make any sense to me. IO is a non-determinism monad (among many other things). That's already true --- randomIO is one example; Control.Exception.evaluate is another (and is already dependent on the particular compilation choices made). I think of Haskell `values' as sets of legal representations anyway --- why shouldn't serialize be free to make a non-deterministic choice from among those sets, just like evaluate can make a non-deterministic choice from the set of exceptions an expression can throw?

Good point. I agree, if the specification is non-deterministic, this should be alright. Tom -- Tom Schrijvers Department of Computer Science K.U. Leuven Celestijnenlaan 200A B-3001 Heverlee Belgium tel: +32 16 327544 e-mail: tom.schrijvers@cs.kuleuven.be

On Thu, Jun 21, 2007 at 04:37:20PM +0200, Tom Schrijvers wrote:

That wouldn't make a difference. If, from the pure Haskell point of view we can't tell the difference between two expressions that denote the same function, then operations in the IO monad should not be able to do so either. Otherviews a whole lot of program transformations based on rewriting of expressions would be invalid. How do you account for that?

In the same way we don't mind if "const getChar" can "tell a difference" between the value () (and itself). Also think of imprecise exceptions - really the pure code hitting the exception does all the usual stuff with stack walking, but semantically the pure expression evaluates to a set of exception, catch makes a nondeterministic choice of which one to handle, and it just happens that catch always picks the one that you run into first given the evaluation order that happend in this particular execution. I think it would be safe to treat serialize the same way. Brandon

Tom Schrijvers wrote:

On Thu, 21 Jun 2007, Pasqualino 'Titto' Assini wrote:

...

But, doesn't this simply mean that the correct signature would be:

serialize :: (Int -> Int) -> IO String

to take in account the fact that serialise really use 'external' information that is not in the domain of pure Haskell functions?

I'm afraid not. The beauty of the IO monad is that it permits equational reasoning over I/O operations. E.g. because of the definition

print = putStrLn . show

the compiler can freely inline calls to print. Although there's I/O involved, equational reasoning is still valid: the inlined call will behave in the same way as the original code. Hence, the compiler does not have to be aware of IO and treat it in a different way.

Your serialize function does not have that property. You don't want its argument to be inlined or otherwise replaced with an equivalent expression.

I don't agree, I think that serialize in the IO-monad is perfectly fine and a good idea :) I mean, a "similar" implementation would be serialize :: (Int -> Int) -> IO String serialize f = randomIO which isn't a treat to equational reasoning as well. Of course, serialize can only live due to his brother compile :: String -> Maybe (Int -> Int) with the property fmap compile . serialize = return . Just That being said, serialization of function values is a practical problem. GHC's internal representation changes often and it would be very tedious to keep serialization working. But Clean can serialize function values. Regards, apfelmus

On Thu, Jun 21, 2007 at 10:27:58AM +0200, Tomasz Zielonka wrote:

On Mon, Jun 18, 2007 at 10:05:40PM +0100, Pasqualino 'Titto' Assini wrote:

...

Most languages, even Java, have a reflection capability to dynamically inspect an object.

...

It is surprising that Haskell doesn't offer it.

I'll just add that I've never consciously used reflection, so I probably overlook some features of Haskell that give a part of reflection functionality. Best regards Tomek

Hi, On Thursday 21 June 2007 09:27:58 Tomasz Zielonka wrote:

I think the reasons are mostly insufficient resources and not enough interest to justify the effort. I think an interesting lesson about this comes from the effort that went into Template Haskell (which, BTW, offers some kind of compile time reflection) - this was probably very hard to design and implement, but my impression is that it's not widely used, despite being "cool" and useful.

I wonder: would it be possible to use the compile time reflection facilities of TH to write a 'serialise' function, keeping the TH AST so that it can be used at run-time? titto

Hi Bulat, I was thinking of something like this (warning: I have never used TH before):

{-# OPTIONS -fth #-} module SerialiseTest where import Language.Haskell.TH

We have an application whose state is a function Int->Int. We want to be able to serialise this state so that, for example, we might transfer it to a remote location. To do so we preserve both the state and its Template Haskell representation:

type State = (Int -> Int,ExpQ)

This is the initial state:

initState :: State initState = (id,[|id|])

The state is modified by composition with the existing state:

modifyState (f,e) (nf,ne) = (nf . f , [| $(ne) . $(e) |] )

Some examples of state changing operations:

op1 state = modifyState state ((+4),[|(+4)|]) op2 state = modifyState state ((*2),[|(*2)|])

By the way, there must be a way of writing in TH a macro that avoids these repetitions of the same function so that we just write $(ser (*2)) rather then ((*2),[|(*2)|]).

main = do

Now a little test, we start with our initial state:

let st0 = initState

Apply a couple of operations:

let st1 = op1 st0 let st2 = op2 st1

Let's see what we got:

let (f2,e2) = st2 printCode e2 putStrLn . show $ f2 5 where printCode ast = runQ ast >>= putStrLn . pprint

This prints: (GHC.Num.* 2) GHC.Base.. ((GHC.Num.+ 4) GHC.Base.. GHC.Base.id) 18 So, the state is both applicable and serialisable (on the receiving side we should naturally have an interpreter for the TH representation). Not very efficient, but it kind of works :-) Best, titto On Thursday 21 June 2007 13:27:07 Bulat Ziganshin wrote:

Hello Pasqualino,

Thursday, June 21, 2007, 3:55:35 PM, you wrote:

...

I wonder: would it be possible to use the compile time reflection facilities of TH to write a 'serialise' function, keeping the TH AST so that it can be used at run-time?

yes. but you will need to find any functions used in definition - i.e. it should be either fixed set of hard-encoded functions or some sort of dynamic binding a-la hs-plugin

Hi Bulat, the receiving side has the option of either interpreting the TH representation or, as you suggested, to just dynamically compile its Haskell source equivalent (as produced by TH's pprint) using GHC API or hs-plugins. Probably not very efficient but quite easy to implement, Best, titto On Thursday 21 June 2007 16:39:58 Bulat Ziganshin wrote:

Hello Pasqualino,

Thursday, June 21, 2007, 7:35:47 PM, you wrote:

...

So, the state is both applicable and serialisable (on the receiving side we should naturally have an interpreter for the TH representation).

and this interpreter should have a way to find function definition by its name - it's the most complex part of job, as i have wrote in previous letter

Hi Bulat, On Thursday 21 June 2007 17:29:13 Bulat Ziganshin wrote:

how it can interpret call to "foo" without loading it? :)

I am not sure if I understand what you mean. It certainly does load it. Calling hspugins "eval" (or compiling with GHC API) will cause 'AModule.foo" to be loaded and returned (the compiled code will naturally need to import all the modules that are shared by the communicating parties). Or you mean: what if the receiving side does not have 'AModule.foo' in the first place? Well, a shared vocabulary is a pre-requisite for communication anyway. If we didn't share the knowledge of the term 'haskell', how would me and you be able to discuss this subject :-) ? What am I missing? Best, titto

Hello Tomasz, Thursday, June 21, 2007, 12:27:58 PM, you wrote:

I also find it hard to believe that most languages have reflection, especially those which are traditionally focused on efficiency and compilation to native code, like C, C++, Fortran, Pascal, etc.

for OOP languages, it is no problem to provide RTTI as part of Object interface (where Object is hierarchy base). to implement the same in Haskell, we need to make Typeable automatic father of any type class created Simon, can it be implemented in GHC? this should provide us advantages of dynamically-typed programming languages -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On Mon, 2007-18-06 at 15:01 +0100, Pasqualino 'Titto' Assini wrote:

LambdaBot, hpaste and hoogle as well as the Haskell Wiki and indeed even this mailing list can all be thought as elements of a cooperative, ubiquitous, customisable, zero-installation, distributed and integrated (that should be enough buzzwords for anybody) development system.

What we need is to connect the dots and [...]

Oh, good Lord! Are we going to write all of this in points-free style? :O -- Michael T. Richter (GoogleTalk: ttmrichter@gmail.com) I'm not schooled in the science of human factors, but I suspect surprise is not an element of a robust user interface. (Chip Rosenthal)

Thomas Schilling wrote:

* Structural (optionally Type-Directed) Editing

Structural editing means that your code is always (mostly) syntactically correct, and in case of haskell maybe also type-checked. This also implies that edit operations have syntactic awareness. paredit[1] emulates this quite nicely for lisp, Proxima does something like this in Haskell for Haskell and XML-based languages. This also needs some way of incremental parsing, for which good techniques already exist[3].

Paredit doesn't just work for lisp. It works for almost all emacs modes, including e.g., haskell and perl (incidentally, I believe it works well for XML/SGML type stuff too). I use it regularly for various programming languages. Of course, it isn't perfect in a few respects, but it's pretty customisable and the author is responsive to ideas and questions. It's not type-directed, of course. Merely structural. But that alone is very handy. (Incidentally I agree with most of Thomas' points which I snipped, too) Jules

peterv writes:

- How can I just compile and run in one go without having to type "ghc --make main.hs" as arguments for the "compile..." command and then typing "main.exe" for "shell command..."? This is what you do all the time when using Visual Studio, Eclipse, etc: just hit the F5 key which builds all the dependencies and runs one or more generated executables. Easy. Visual Haskell does this (in the cases it does not hang for some unknown reason). Of course I could dig into ELISP, but somehow I feel this must be supported somehow.

Use C-c C-l to load the file into GHCi. This is better than just compiling it: you get an interactive environment in which to play around with, test out functions, etc. You can still 'run' your program by typing 'main' in GHCi.

- There seems to be support for "Haskell Font Lock Symbols", which should convert \, -> and maybe other symbols to good looking Unicode fonts. I can't get the correct fonts working on Windows. Now as I understood this is not really supported because of indentation problems, as underneed the single Unicode arrow character is converted into ->? This is unfortunate, because that's one of the things I really like about Sun's Fortress: the usage of Unicode symbols makes the text look much more mathematically, versus the half-century-old ASCII text. Furthermore in almost all Haskell books the richer font symbols are used for clarity. Any hints on this topic?

I wouldn't be surprised if this code had bit-rotted, or if there never has been Windows support. It's a corner function, used by few and therefore not that polished. By all means, try it out, and if it doesn't work, feel free to submit patches, but I doubt it'll get changed any time soon by a haskell-mode developer :) -- -David House, dmhouse@gmail.com

peterv wrote:

And when I will me using HopenGL, I will want performance, as I will be doing experiments with particle systems, 3D rendering, etc. Basically the stuff I did for many years but now using Haskell :)

Having to do anything more than hitting a key to compile and run an application would simple be unacceptable from the point of view of an imp/OO developer (all imp/OO IDEs have that). Furthermore, when programming videogames or special effects, you have to run and test a lot, because what you see on screen usually determines your next actions.

I don't disagree that this should be possible. If you hit C-h f compile then you can read the documentation for the built-in compile command. By default this runs make, because many code projects especially in the unix world use make as their build system, but you can customise this. On the other hand a simple haskell project doesn't have any way of indicating which is the 'main' file (indeed most of my haskell projects with more than one file have more than one 'main' file with different purposes) so it's not immediately obvious which arguments to give to ghc --make. I suspect that this itch is just not sufficiently important to most haskell-mode users, since the alternatives (C-c C-l, C-x b M-p <RET>, or alt-tab <up> <ret>) [*] work so well. Incidentally I've developed using HOpenGL in ghci with no problem. The performance is not really an issue: only the 'current file' is interpreted, all other files are used compiled, and in any case most of the CPU usage is in the (compiled) GL libraries of your system. Jules * C-x b M-p <RET> being 'switch to your shell buffer, select previous command and re-run it', since your previous command is obviously ghc --make foo.hs && ./foo. Alt-tab <up> <ret> is the same thing except it switches to a non-emacs shell window using your window manager, if you don't like using emacs shells :)

peterv writes:

Yes, but I can only use GHCI for error checking, because I'm using GTK2HS/SOE which does not work well with GHCI under Windows, it only runs when using GHC.

Why is this? I'm not that familiar with Gtk2Hs, but I don't understand why it wouldn't work with GHCi if it works with GHC. They use the same code to compile it. On the other hand, you could always just set up a Makefile (which is pretty trivial) and use M-x compile (which you should bind to a key if you use it a lot). -- -David House, dmhouse@gmail.com

Yes this was also very very confusing for me because I had the same idea about that. I almost gave up on learning Haskell because of that (I wanted to practice stuff from the SOE book using the latest versions), until I suddenly found out that GHC *did* work. Here's the explanation: "On Fri, 2007-06-15 at 23:15 +0200, bf3@telenet.be wrote:

I'm learning Haskell using Paul Hudak's book SOE.

I'm using GHC 6.6 under Windows XP.

GHC on Windows does not seem to come with HGL (is this correct?), so I used Gtk2HS, which contains a SOE implementation.

I noticed that most programs hang when using GHCI, but they work fine with GHC.

It's not GHCi's fault as such. The reason it does not work well in GHCi at the moment is a bit technical. The Gtk2Hs SOE implementation currently uses Haskell threads. Like most GUI toolkits, Gtk+ is single threaded and requires special attention to use it from multiple OS threads. Currently, by default, GHC produces executables that use the single-threaded runtime system, and this works fine with multiple Haskell threads because they get multiplexed on the same OS thread. GHC can however produce executables that use the multi-threaded runtime system and ghci.exe itself is such a program. So when you use SOE with GHCi it's actually using multiple threads to access Gtk+ an not in a safe way, so it goes wrong in a myriad of ways. I'll take another look at trying to make the SOE stuff work with the threaded runtime system by using the primitives Gtk2Hs provides to use Gtk+ safely from multiple threads. Duncan" -----Original Message----- From: David House [mailto:dmhouse@gmail.com] Sent: Wednesday, June 20, 2007 6:42 PM To: peterv Cc: 'David House'; haskell-cafe@haskell.org Subject: RE: [Haskell-cafe] Haskell mode for emacs - some questions peterv writes:

Yes, but I can only use GHCI for error checking, because I'm using GTK2HS/SOE which does not work well with GHCI under Windows, it only runs when using GHC.

Why is this? I'm not that familiar with Gtk2Hs, but I don't understand why it wouldn't work with GHCi if it works with GHC. They use the same code to compile it. On the other hand, you could always just set up a Makefile (which is pretty trivial) and use M-x compile (which you should bind to a key if you use it a lot). -- -David House, dmhouse@gmail.com

...

I finally got emacs using Haskell-mode working. It seems that the latest

Sounds like it was difficult. Could you describe what you tried, what didn't work, and what did work in the end? Hopefully I can then improve the doc.

...

- How can I just compile and run in one go without having to type "ghc --make main.hs" as arguments for the "compile..." command and then typing "main.exe" for "shell command..."? This is what you do all the time when using Visual Studio, Eclipse, etc: just hit the F5 key which builds all the dependencies and runs one or more generated executables. Easy. Visual Haskell does this (in the cases it does not hang for some unknown reason). Of course I could dig into ELISP, but somehow I feel this must be supported somehow.

I never use GHC in this way, I always use GHCi instead. Furthermore, I tend to work on only parts of a program, so there isn't necessarily a "main" function. I'd be happy to add support for your usage pattern, but since I'm not familiar with it, I'm not sure what to add. Another problem is that unless your project is tiny, it'll have several files and currently Emacs doesn't know which file is the main one. I added very-preliminary support for Cabal in the CVS code of haskell-mode which should allow haskell-mode (at some point in the future) to figure out what's the main file an how to compile it. Currently all it does (other than font-lock the cabal file itself) is look for the Cabal file to figure out the root of the project, so that C-c C-l first does a "cd" to the root, which should allow dependencies in other directories to work more seemlessly. Patches (or precise feature requests) are very welcome. E.g. it should be fairly easy to add an "F5 binding" like you describe. The main issue is how to inform Emacs of what should be done. In VS or Eclipse, what do you have to do in order for F5 to work? Is opening some random source file enough, or do you have to select a Cabal file or what?

Use C-c C-l to load the file into GHCi. This is better than just compiling it: you get an interactive environment in which to play around with, test out functions, etc. You can still 'run' your program by typing 'main' in GHCi.

We could also add a binding which sends "main" to GHCi.

...

- There seems to be support for "Haskell Font Lock Symbols", which should convert \, -> and maybe other symbols to good looking Unicode fonts. I can't get the correct fonts working on Windows.

I never use Windows so I can't really help you there. Maybe ask on gnu.emacs.help how to get those chars displayed. I'm pretty sure Windows has the needed fonts, so all that's missing is some way to help Emacs make use of them. If you figure it out, please send me a note about what you had to do, so I can add it to the documentation.

...

Now as I understood this is not really supported because of indentation problems, as underneed the single Unicode arrow character is converted into ->?

You slightly misunderstood: this hack is fully supported. I just added some warnings to the docstring to make sure the user doesn't blame me when he gets bitten.

...

This is unfortunate, because that's one of the things I really like about Sun's Fortress: the usage of Unicode symbols makes the text look much more mathematically, versus the half-century-old ASCII text. Furthermore in almost all Haskell books the richer font symbols are used for clarity. Any hints on this topic?

I use this hack all the time and haven't been bitten yet. Some important aspects: - the λ should use a 1-char-wide lambda (i.e. the unicode one). There's also a JIS lambda used if the unicode one can't be found, but this one is 2-char wide, so it risks introducing mis-indentation. - the → and ← OTOH should use the 2-char-wide JIS arrow, rather than the unicode arrows for the same reasons. - ...

I wouldn't be surprised if this code had bit-rotted, or if there never has been Windows support.

I use it all the time, so it definitely hasn't bit-rotted (actually, it's one of the most often updated part of the code). There shouldn't need to be anything special for Windows, but indeed I've never tried it on Windows (and don't intend to either).

It's a corner function, used by few and therefore not that polished. By all means, try it out, and if it doesn't work, feel free to submit patches, but I doubt it'll get changed any time soon by a haskell-mode developer :)

You'd be surprised, Stefan