Ketil Malde wrote:

...

If a module M imports A.B.C, and I can see two such modules called A.B.C, then the meaning of the code is ambiguous and ill-defined.

Surely this depends on the relationship between the two A.B.C., and I think you make the underlying assumption that they implement different things.

No, I agree that they may be essentially the same module, but one of them has a bugfix, or a new API call, or something.

Now, to be able to test my version without rewriting all client code, I'd much rather do this substitution from the command line; ghc -hide foo -package myfoo ...

With the Simons' proposal, and with Brian's, you would need to edit the source code at all usage positions of "import A.B.C". I agree that this is tedious and undesirable. With my proposal, you could organise your source code such that only a single file might need to be edited, replacing say module Foo (namespace F) where import namespace "foo" as F with module Foo (namespace F) where import namespace "myfoo" as F

Another case that nobody seem to have mentioned, is what happens if I move a module from one package to another. Say somebody finds out that 'base' is getting crowded, and wants to move Data.Set to 'collections'. Annotated imports would then have to be updated,

Again, I believe my proposal would permit a single update, whilst the other proposals would require multiple updates. http://hackage.haskell.org/trac/ghc/wiki/GhcPackagesWithGrafting

PS: Talking about module A.B.C seems a bit abstract. Perhaps the wiki page could be updated with some examples of namespace collisions that people have experienced?

One thing I would very much like to be able to write is regression testing over libraries. Does the computation P using package HaXml-1.15 give the same result as running P with package HaXml-1.13? At the moment, one cannot write such a test directly in Haskell, one must escape to a separate test harness. Regards, Malcolm

Ketil Malde wrote:

This brings up an interesting point: if I need a bugfix from version X, later packages will probably also be okay. So it should be possible to do:

import A.B.C from "foo >= X"

Yes, I agree, some notion of version ranges might be desirable "1.3.2 <= foo <= 1.3.9" That is one good reason to leave the package name as a quoted string, so such questions can be resolved later in the design exercise, and even changed in the future, without impacting the basic syntax.

How about minor/major revisions?

Yes, you might want "foo-2" to indicate that any of "foo-2.0" up to "foo-2.99.9" are acceptable. If every library author sticks to the rule of no API changes in minor versions, then this might even cover the same territory as the version range idea.

...

With the Simons' proposal, and with Brian's, you would need to edit the source code at all usage positions of "import A.B.C".

I thought Simons' (and perhaps also Brian's) still allowed package juggling from the command line - is that incorrect? I.e. is "import from.." intended to be the *only* way of specifying packages?

Hmm. I am all in favour of removing the ability to name packages on the commandline, making the source code the definitive specification. (But I have the impression that the other proposals do in fact want to allow both ways of naming packages - source and environment.)

And another issue: presumably the source code is prioritized, so if the cabal file (or command line) specifies -package foo > v2, but the sources specify "foo-2.0", the compilation will fail if only foo-2.1 is installed?

Just one example of why it is better to have only one way of naming the package dependencies. The whole question of conflicts between source and command-line is nasty.

Cabal lets you specify package dependencies, at least for the program as a whole - doesn't that mean it would have to rewrite the source before compilation?

If the packages are named in the source, then an automated tool can gather all the dependencies and write the Cabal file for you. The Cabal file is descriptive, not prescriptive. I see it essentially as a cache of information (packages needed, source file names, language extensions) that is otherwise already available directly from the source in a scattered fashion.

...

With my proposal, you could organise your source code such that only a single file might need to be edited, replacing say module Foo (namespace F) where import namespace "foo" as F with module Foo (namespace F) where import namespace "myfoo" as F

I think you can do this, as long as symbols are annotated with package names. Even if only command line options are available, you can do something like:

{-# OPTIONS -package foo #-} -- or replace with myfoo module F where import Foo

No, I think you missed the point. Let's say there are 30 files that import modules from the package "foo". At the moment, and with the current proposal, when you decide to replace "foo" with "myfoo", you must edit all 30 files to change the OPTIONS line (or the "import from" specs). The key new concept in my proposal, by contrast, is that you can manage the visibility of namespaces separately from the particular usages of the namespace (i.e. module imports). This makes it possible to locate the specification of package "foo" in a single file, and when you change to "myfoo", only the one file needs to be edited.

PS: Is it correct that -- except for the new functionality in symbols being annotated with package as well as module name -- all of this is just a matter of convenience/syntactic sugar? (Which would explain the volume of the discussion :-)

For the current proposal, yes. For my extended proposal, no, it introduces a new language concept that was not previously expressible. Regards, Malcolm

Malcolm Wallace writes: Just one more thing:

...

PS: Is it correct that -- except for the new functionality in symbols being annotated with package as well as module name -- all of this is just a matter of convenience/syntactic sugar? (Which would explain the volume of the discussion :-)

For the current proposal, yes.

Then I'm not convinced it is worth introducing new import syntax. I think 90% of the cases of ambiguity should be resolved globally for the project, i.e. by compiler switches (typically in .cabal, Makefile, or similar). These cases include using newer versions of standard packages, regression testing for library versions, or alternative implementations. For the remaining (and bear in mind that nobody have put forward anything but hypothetical examples yet), I think a proxy module is sufficient as a solution. This is necessary where one program needs to simultaneously use modules with the same hierarchical name from different packages. And the advantage of making this hurt a bit, is that it puts pressure on different-functionality modules to populate different locations in the hierarchical namespace - the pain here is a feature, not a bug. (Please enlighten me if I'm missing some important cases here.) -k -- If I haven't seen further, it is by standing in the footprints of giants

Aaron Denney wrote:

...

...

Package names should never appear in source files IMHO.

I tend to the opposite view.

Then, as John points out, how is package Foo module A.B.C and package Bar module A.B.C any different than modules Foo.A.B.C and Bar.A.B.C?

I have great sympathy with this view - that packages are little different from a top-level name in the hierarchy. But Simon PJ's comment (on the wiki page) about the difference between specifying the _purpose_ of a module in its name, and the _provenance_ of a module in its package identifier, was very convincing. I have added (yet another) alternative proposal, to the wiki here: http://hackage.haskell.org/trac/ghc/wiki/GhcPackagesWithGrafting The details overlap significantly with the current proposals, but the main contribution I am trying to bring to the table is the (old but never implemented) idea of grafting a sub-hierarchy at an arbitrary location. This idea has a close relationship with specifying what package a module should come from. So, I have tried to combine the two. Regards, Malcolm

Simon Marlow wrote:

...

http://hackage.haskell.org/trac/ghc/wiki/GhcPackagesWithGrafting

Please correct me if I get anything wrong:

- the proposal is to let you specify grafting in the source code

This is not the main focus of the proposal. Grafting is one oft-requested feature that is not currently supported, but yet it seems very close to the idea of specifying a package dependency in the source code. So my intention is merely to take it into account when designing the latter facility. If one mechanism (with a little tweaking) can achieve both aims, so much the better. The central plank of my proposal is to introduce an explicit concept of "namespace". Namespaces can be managed separately from the imports that range over them. That's it.

- you graft a *sub-hierarchy* of a package anywhere in the global module namespace (the sub-hiearchy bit is new, I haven't seen this proposed before).

The ability to graft hierarchies is just a side-effect. I would phrase this point as - a "namespace" is a sub-hierarchy of the modules contained in a package. (The sub-hierarchy bit is indeed new.)

- you can also graft a sub-hierarchy of a package onto the *current module*, so that it becomes available when importing this module. This is new too.

Again, I would rephrase this: - "namespaces" can be imported and exported, separately from the modules they contain. (Yes, this is new too.) Regards, Malcolm

Simon Marlow wrote:

So I don't like this idea of re-exporting namespaces. Here's why.

When I see 'import M', I have to search not only the filesystem, but also for possible namespaces re-exported by *all* the other import declarations in the file, which means I have to understand all those other import declarations, and the process is recursive.

True. But I don't see how it is much different from ordinary imports: if an entity e is used in the source text, but not defined there, you must search the modules that are imported to see where e was defined. In general, e might not be explicitly defined in /any/ of the directly-imported modules, so you must recursively check their imports/re-exports.

If this is hard for the compiler, it's hard for the user too. Someone understanding your source code has a lot of places to look to find the target of an import.

I think it is rather easy for the compiler actually. Agreed that it provides a slight extra burden on the human reader. But I don't think it is significantly larger than the current burden of knowing the definition site of a function.

I believe there should be a clean separation between - constructing the global module namespace, and - interpreting the source code of a module

OK, but that is in direct conflict with the original proposal (to specify a package alongside a module import), which abandons the idea of a "global" module namespace! Indeed, the whole motivation was that we _don't_ want a global module namespace, because that would require all modules in all packages to have unique names.

that is, mapping an import declaration to a module should require nothing more than searching the filesystem and package database. This is true now, and I think we should ensure it remains true, for the benefit of tool and language implementors.

At the moment, the compiler must search the filesystem, package database, and *interface files* (for imports/re-exports). With explicit namespaces, nothing changes there (except that namespaces must also be recorded in the interface files). Regards, Malcolm

Simon Marlow wrote:

Consider that at the moment, an import declaration can be considered completely independently of the rest of the module. Imports do not affect each other. Under your proposal, all imports have to be considered together to determine whether any are ambiguous. We can't do Hugs-style one-at-a-time import chasing, for example.

Ah, now I see what you are getting at. Program P imports modules M and N. How do the compiler know where M is located? It must presume the default namespace. But then later, module N might re-export a different namespace which means that M is now ambiguous. Conversely, let's say M is not available in the default namespace. So we must wait until module N has been imported, to see if it brings a namespace into scope that does contain a module M. Very nasty.

I accept that the GhcPackages proposal has a serious shortcoming that your proposal addresses, namely that there's no way to name a package in just one place if you're using package-qualified imports. I don't yet know of a good way to fix this, but I'm fairly sure that generalising the module system with namespaces is not a good power/weight tradeoff.

OK, I accept that my proposal for first-class namespaces is rather too heavy a solution. (But I still think the problem is a real one, and we should keep thinking about other ways to solve it.) Regards, Malcolm

Simon Marlow wrote:

...

Surely the programmer knew which import was intended. Is it so difficult to communicate that information somewhere close to the import itself?

I think this is an oversimplification. Usually you want your source code to be abstract with respect to certain properties of the environment: eg. I don't care to include the path to the file containing module M in the source code, so that I can move M around if I need to. Similarly I don't want to specify the exact version of any module I depend on, since the same code probably works with a range of versions.

I agree with all that. The mapping of the package name to its location in the filespace should not be in the source code. That is the job of a package management tool.

I certainly want to specify that dependency somewhere (but only in one place, eg. foo.cabal).

The question for me is whether the .cabal file therefore counts as source code. Certainly the .cabal file no more specifies where packages are located in the filesystem than do any of the proposals on the table. But if the .cabal file is needed in order to disambiguate the code, what happens if someone wants to use a Makefile instead of Cabal? You can see where this leads - any number of build systems could potentially have an influence on the meaning of the code. So I think one good thing about the current proposals is that they move the package dependencies into the source code itself, where they cannot get lost or forgotten or overlooked. The *dependency* is noted, but the details of how to locate a package are rightly left to tools. But the biggest problem is really your phrase "(but only in one place)". I agree that specifying the dependencies close together is good in principle. But occasionally, you need to have different package dependencies for different modules of a single project - this is a finer granularity of dependency than Cabal is currently good at expressing. Regards, Malcolm

Simon Marlow wrote:

...

this is a finer granularity of dependency than Cabal is currently good at expressing.

By all means have package names scattered through the source code, but the dependencies still need to be specified in a single place for each project. The main reason is so that tools can grab the dependencies of a package without too much fuss: tools that turn Haskell packages into OS packages, for example.

Indeed. No problem with that. Any individual module might have a small and specific set of package dependencies. The project as a whole will have the union of all of those. Regards, Malcolm