RE: Module Holism (was RE: exposed packages and cabal depends)
On 11 April 2005 18:58, S. Alexander Jacobson wrote:
You said that no two modules in a program can share the same name.
At the same time, we want a library system that allows us to use any combination of modules in any program.
The problem is, we don't want to import two modules, only to discover that somewhere in their dependencies they each use the same module name to refer to conflicting module implementations.
This is the problem that the overlap restriction leads to, yes. It is mitigated by the fact that the granularity of dependencies is made coarser by the packaging system, so we believe it will rarely be a problem in practice. Remember, when you're importing library modules, your dependencies are on *packages*, not modules.
Therefore, we really want to say that no two modules we might want to import into our programs (either directly or indirectly) should share the same name. And, in particular, we don't want a packaging or versioning system that encourages it!
No, you've drawn a bogus conclusion again. We most definitely want the ability to choose between multiple instances of a particular module in programs. For example, if I have two versions of a package installed, say P-1 and P-2, I want to be able to compile my old code that depends on P-1 while still being able to write new code against P-2. And I want to be able to use other packages that still depend, for the time being, on P-1. When P-3 comes out, I don't want to be forced to uninstall P-1 and P-2 before I can use it. This restriction on module overlaps is mainly for engineering reasons, as I've explained before. Someday it might be lifted, and we want the design of the system to extend gracefully when that happens, which the current design will.
Build-depends assumes it is useful to talk about dependencies with respect to a given package, when in fact, all modules have to share the same namespace. -hide-all-packages assumes it is useful to talk about dependencies with respect to a given system when, in practice, all users should be in as similar a namespace as possible. In effect -hide-all-packages encourages untracked dependencies on some very local namespace.
Sorry, I don't understand that paragraph at all.
What we want is a common system for mapping module names to module implementations.
* a protocol for resolving module-names to implementations at various haskell module name servers (hackage?)
Go ahead, invent a protocol!
* support for addition and overrides by specific communities (proxies?)
* support for creation of new versions by individuals (-i override?)
And we need to constraint the versioning system to require that every new version of a module must fulfill the contracts of the prior versions so no existing dependency is broken by any changes.
That's way too restrictive. We'd never be able to remove anything. Furthermore, for GHC, you can't actually upgrade a library in place. Libraries that were built against the previous version will break, because in GHC the "contract" between compiled modules is extremely intimate (in includes a lot more than the exported interface of the modules). When we start using shared libraries, even binaries will break if you upgrade shared libraries in place. That's why we must have versioning, and the ability to have multiple versions of a package installed. Cheers, Simon
On Mon, 11 Apr 2005, Simon Marlow wrote:
The problem is, we don't want to import two modules, only to discover that somewhere in their dependencies they each use the same module name to refer to conflicting module implementations.
This is the problem that the overlap restriction leads to, yes.
No, you have this problem even with atomic modules. It is a result simply of not allowing two modules to share the same name in the same program and has nothing to do with package overlap. e.g. Suppose module Foo imports module Goo' and module Bar imports module Goo'' but that modules Goo' and Goo'' both claim to be named "Module Goo". Foo and Bar both work on their own but you can't use both in the same program. That's a big problem!
Remember, when you're importing library modules, your dependencies are on *packages*, not modules.
Notice that in my example, it doesn't matter whether these modules are all in the same package or whether they are all in different packages or anything in between.
It is mitigated by the fact that the granularity of dependencies is made coarser by the packaging system, so we believe it will rarely be a problem in practice.
In practice, it is all too easy for two diffferent packages or modules to assume two different versions of e.g. Network.HTTP.
Therefore, we really want to say that no two modules we might want to import into our programs (either directly or indirectly) should share the same name. And, in particular, we don't want a packaging or versioning system that encourages it!
No, you've drawn a bogus conclusion again. We most definitely want the ability to choose between multiple instances of a particular module in programs.
My point is that the choice of instance should be made at compile/build/run times and not at packaging time. Allowing packages/modules to choose implementations increases the risk of the sort of conflict described above.
For example, if I have two versions of a package installed, say P-1 and P-2, I want to be able to compile my old code that depends on P-1 while still being able to write new code against P-2. And I want to be able to use other packages that still depend, for the time being, on P-1. When P-3 comes out, I don't want to be forced to uninstall P-1 and P-2 before I can use it.
Now what happens when you want to use one package that depends on P-1 and another that depends on P-2 at the same time? My claim is that you can solve this only if a package that depends on P-1 can also work with P-2 and one that depends on P-2 can also work with P-3. The only reason a build should fail is that you don't have a sufficiently recent version of some module. [clarifying explanation of build-depends problems from prior mail] When you say that package 'A' "build-depends" on packages 'B' and 'C', you are implicitly selecting a specific set of implementations for the module names used in package 'A'. However if both 'B' and 'C' define and use module M then you are going to run into trouble. But now lets suppose that 'B' uses module M and 'C' doesn't. But that package 'C' build-depends on package 'D' and package 'D' build-depends on package 'E'. Now suppose that a new version of 'E' is released that defines and uses module M. Now you have real trouble! Either way, you need some way to specify which module is authoritative implementation of a particular module name for your entire build and not per package! In other words, the mapping of module names to implementations is not specific to any particular package, it is explicitly global to the build of the resulting program and implicitly global to the community of module/package authors. For the same reason, you don't want "-hide-all-packages." You want "-hide-local-variance" from the consensus mapping of module names to implementations.
What we want is a common system for mapping module names to module implementations.
* a protocol for resolving module-names to implementations at various haskell module name servers (hackage?)
Go ahead, invent a protocol!
I offered two strawman versions of this protocol in my last mail in the "stop untracked dependencies" thread: Strawman protocol 1: Define a new DNS record, URL, and define some translation of module names into domain names. Strawman protocol 2: Use HTTP/HTTPS and define a query syntax such as "GET ResolverURL/moduleName HTTP/1.0" and use 30x headers for redirection to the appropriate server. I like the later because we can then reuse WebDAV versioning semantics, but think that there is little point in fleshing it out further unless we agree on the need.
And we need to constraint the versioning system to require that every new version of a module must fulfill the contracts of the prior versions so no existing dependency is broken by any changes.
That's way too restrictive. We'd never be able to remove anything.
Its not too restrictive. It is correct. You said that module names are intended to describe functionality. If you want to change the contract, you need either to define a new module name or accept that programs that depend on a sufficiently old version of the current module will simply break. What does 'deprecated' mean?
When we start using shared libraries, even binaries will break if you upgrade shared libraries in place. That's why we must have versioning, and the ability to have multiple versions of a package installed.
Again, I am not opposed to having multiple versions of a package available for programs to use. My point is that the choice of version must occur at build/run time and not at package time. -Alex- ______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com
Short summary of my big point from my prior mail We want late binding of module names to implementations. Effectively, module names should be treated as URLS. Argument 1. We want loose coupling of packages/modules. 2. "Build-depends" tightly couples packages/modules 3. Binding of module names to implementations should be at run time not package time See here http://www.manageability.org/blog/stuff/recombinant-long-tail/view http://www.markbaker.ca/2002/09/Blog/2003/10/03#2003-09-mnot-late-binding There is a natural connection between late binding of module names and REST architectural style. -Alex- ______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com On Mon, 11 Apr 2005, Simon Marlow wrote:
On 11 April 2005 18:58, S. Alexander Jacobson wrote:
You said that no two modules in a program can share the same name.
At the same time, we want a library system that allows us to use any combination of modules in any program.
The problem is, we don't want to import two modules, only to discover that somewhere in their dependencies they each use the same module name to refer to conflicting module implementations.
This is the problem that the overlap restriction leads to, yes. It is mitigated by the fact that the granularity of dependencies is made coarser by the packaging system, so we believe it will rarely be a problem in practice.
Remember, when you're importing library modules, your dependencies are on *packages*, not modules.
Therefore, we really want to say that no two modules we might want to import into our programs (either directly or indirectly) should share the same name. And, in particular, we don't want a packaging or versioning system that encourages it!
No, you've drawn a bogus conclusion again. We most definitely want the ability to choose between multiple instances of a particular module in programs. For example, if I have two versions of a package installed, say P-1 and P-2, I want to be able to compile my old code that depends on P-1 while still being able to write new code against P-2. And I want to be able to use other packages that still depend, for the time being, on P-1. When P-3 comes out, I don't want to be forced to uninstall P-1 and P-2 before I can use it.
This restriction on module overlaps is mainly for engineering reasons, as I've explained before. Someday it might be lifted, and we want the design of the system to extend gracefully when that happens, which the current design will.
Build-depends assumes it is useful to talk about dependencies with respect to a given package, when in fact, all modules have to share the same namespace. -hide-all-packages assumes it is useful to talk about dependencies with respect to a given system when, in practice, all users should be in as similar a namespace as possible. In effect -hide-all-packages encourages untracked dependencies on some very local namespace.
Sorry, I don't understand that paragraph at all.
What we want is a common system for mapping module names to module implementations.
* a protocol for resolving module-names to implementations at various haskell module name servers (hackage?)
Go ahead, invent a protocol!
* support for addition and overrides by specific communities (proxies?)
* support for creation of new versions by individuals (-i override?)
And we need to constraint the versioning system to require that every new version of a module must fulfill the contracts of the prior versions so no existing dependency is broken by any changes.
That's way too restrictive. We'd never be able to remove anything.
Furthermore, for GHC, you can't actually upgrade a library in place. Libraries that were built against the previous version will break, because in GHC the "contract" between compiled modules is extremely intimate (in includes a lot more than the exported interface of the modules).
When we start using shared libraries, even binaries will break if you upgrade shared libraries in place. That's why we must have versioning, and the ability to have multiple versions of a package installed.
Cheers, Simon
participants (2)
-
S. Alexander Jacobson -
Simon Marlow