April 2005 - Libraries

RE: Module Holism (was RE: exposed packages and cabal depends)
by Simon Marlow 13 Apr '05

13 Apr '05

On 12 April 2005 01:28, S. Alexander Jacobson wrote: > On Mon, 11 Apr 2005, Simon Marlow wrote: >>> The problem is, we don't want to import two modules, only to >>> discover that somewhere in their dependencies they each use the >>> same module name to refer to conflicting module implementations. >> >> This is the problem that the overlap restriction leads to, yes. > > No, you have this problem even with atomic modules. It is a result > simply of not allowing two modules to share the same name in the same > program and has nothing to do with package overlap. When I say "overlap restriction" I mean the restriction that prevents having two modules with the same name in the same program. >>> Therefore, we really want to say that no two modules we might want >>> to import into our programs (either directly or indirectly) should >>> share the same name. And, in particular, we don't want a packaging >>> or versioning system that encourages it! >> >> No, you've drawn a bogus conclusion again. We most definitely want >> the ability to choose between multiple instances of a particular >> module in programs. > > My point is that the choice of instance should be made at > compile/build/run times and not at packaging time. Dependencies must be expressed at package time, otherwise they are untracked dependencies. Right? A package must state that it needs implementation P of module M rather than implementation Q, otherwise there's a chance that the guy building the package will get the wrong one. Build-depends lets you select an implementation from a range (currently it's just a version range, but we could make the language more expressive and include alternatives, I doubt it would be that useful in practice, though). So dependencies are selected at compile time. Selecting dependencies at run-time (I assume this is what you mean by late binding) is another kettle of fish: at least, it would mean that GHC couldn't do its aggressive cross-module optimisation against the library that you're late-binding to. But I think it's worthwhile investigating to what extent this is possible, so that we can have upgradable shared libraries. >> For example, if I have two versions of a package installed, >> say P-1 and P-2, I want to be able to compile my old code that >> depends on P-1 while still being able to write new code against P-2. >> And I want to be able to use other packages that still depend, for >> the time being, on P-1. When P-3 comes out, I don't want to be >> forced to uninstall P-1 and P-2 before I can use it. > > Now what happens when you want to use one package that depends on P-1 > and another that depends on P-2 at the same time? Of course, you can't do that. That's what the "overlap restriction" prevents. As I've explained. [ the next three paragraphs, which I deleted, all complain about scenarios which we can't handle because of the overlap restriction. I don't think I need to comment any further. ] [ more stuff deleted, this thread is too long already ] Cheers, Simon

3 2

[ANN] SDL 0.2
by Lemmih 12 Apr '05

12 Apr '05

Greetings, In the spirit of release early, release often, I hereby announce SDL[1] 0.2, a Haskell binding to libSDL[2] and various addons[3]. It is a cabal[4] package and has been tested with GHC 6.4 and Hugs 20050308. -- Friendly, Lemmih [1] http://www.scannedinavian.org/~lemmih/SDLstable/dist/SDL-0.2.tar.gz [2] http://www.libsdl.org/ [3] http://www.ferzkopp.net/~aschiffler/Software/SDL_gfx-2.0/, http://www.libsdl.org/projects/SDL_image/, http://www.libsdl.org/projects/SDL_mixer/ and http://www.libsdl.org/projects/SDL_ttf/. [4] http://www.haskell.org/cabal/

1 0

RE: Module Holism (was RE: exposed packages and cabal depends)
by Simon Marlow 12 Apr '05

12 Apr '05

On 11 April 2005 18:58, S. Alexander Jacobson wrote: > You said that no two modules in a program can share the same name. > > At the same time, we want a library system that allows us to use any > combination of modules in any program. > > The problem is, we don't want to import two modules, only to > discover that somewhere in their dependencies they each use the > same module name to refer to conflicting module implementations. This is the problem that the overlap restriction leads to, yes. It is mitigated by the fact that the granularity of dependencies is made coarser by the packaging system, so we believe it will rarely be a problem in practice. Remember, when you're importing library modules, your dependencies are on *packages*, not modules. > Therefore, we really want to say that no two modules we might want to > import into our programs (either directly or indirectly) should share > the same name. And, in particular, we don't want a packaging or > versioning system that encourages it! No, you've drawn a bogus conclusion again. We most definitely want the ability to choose between multiple instances of a particular module in programs. For example, if I have two versions of a package installed, say P-1 and P-2, I want to be able to compile my old code that depends on P-1 while still being able to write new code against P-2. And I want to be able to use other packages that still depend, for the time being, on P-1. When P-3 comes out, I don't want to be forced to uninstall P-1 and P-2 before I can use it. This restriction on module overlaps is mainly for engineering reasons, as I've explained before. Someday it might be lifted, and we want the design of the system to extend gracefully when that happens, which the current design will. > Build-depends assumes it is useful to talk about dependencies with > respect to a given package, when in fact, all modules have to share > the same namespace. -hide-all-packages assumes it is useful to talk > about dependencies with respect to a given system when, in practice, > all users should be in as similar a namespace as possible. In effect > -hide-all-packages encourages untracked dependencies on some very > local namespace. Sorry, I don't understand that paragraph at all. > What we want is a common system for mapping module names to > module implementations. > > * a protocol for resolving module-names to implementations at > various haskell module name servers (hackage?) Go ahead, invent a protocol! > * support for addition and overrides by specific communities > (proxies?) > > * support for creation of new versions by individuals (-i override?) > > And we need to constraint the versioning system to require that > every new version of a module must fulfill the contracts of the > prior versions so no existing dependency is broken by any changes. That's way too restrictive. We'd never be able to remove anything. Furthermore, for GHC, you can't actually upgrade a library in place. Libraries that were built against the previous version will break, because in GHC the "contract" between compiled modules is extremely intimate (in includes a lot more than the exported interface of the modules). When we start using shared libraries, even binaries will break if you upgrade shared libraries in place. That's why we must have versioning, and the ability to have multiple versions of a package installed. Cheers, Simon

2 2

RE: exposed packages and cabal depends
by Simon Marlow 12 Apr '05

12 Apr '05

On 08 April 2005 22:24, S. Alexander Jacobson wrote: > The -hide-all-packages option would make sense only if there were real > flexibility in selecting which packages to use in implementing the > modules imported into your program. However, in our discussions, you > you've defined two rules that eliminate this flexibility: > > * you cannot use two different modules with the same module ID in the > same program > > * you cannot use two different packages with the same module ID in the > same program > > Together these constraints imply that packages should contain only > minimal sets of mutually interdependent modules; every additional > module included in a package restricts its usability. So, if no two > packages should contain the same module name, one must therefore > conclude: > > Package Dependency Rule: The module names used in a program > completely specify the packages required to interpret/compile it. No, that doesn't follow. You can most certainly have lots of packages with the same module, you just can't use them in the same program. How do you think we plan to have multiple versions of packages? [ stuff about the -i overriding problem deleted... I'll think about it and post to a new thread.] > To be clear, my problem with -hide-all-packages is that it creates > much greater risk of getting out of sync with the global table. > Build-depends tags are actively bad because you cannot know the nth > degree dependencies of the packages you are using. Correctness is > only with respect to the global table. Yes, the user has to know about dependencies because of the overlap restriction. But dependencies are completely visible: you *can* know the dependencies of the packages you're using. I'm not sure what you mean by "global table". One day we'll lift the overlap restriction and things will get easier, but not right now (unless you have a patch!). > Yes, I am begging the question of how we construct and manage global > tables in a way that preserves free-packaging/decentralization. > (Perhaps it is with user specifiable hackage tables) But, there is > no point in talking about solutions until we agree on the problems > that need to be solved. At this point I'm a bit hazy on what problem you still think needs to be solved :-) We've eliminated untracked dependencies from Cabal packages. At some point, we'll generalise package names to URLs (assuming community agreement and a concrete proposal). Hackage will be providing a centralised package repository for those that want it. Cheers, Simon

2 1

Cabal bug?
by Niklas Broberg 12 Apr '05

12 Apr '05

The following works fine: hspr.cabal: -------------------------------------------- Name: hspr Version: 0.2 License: PublicDomain Author: Niklas Broberg Maintainer: nibro(a)cs.chalmers.se Build-Depends: base, haskell98, haskelldb, hsp Executable: hspr Main-Is: HSPR.hs -------------------------------------------- This does not: hspr.cabal -------------------------------------------- Name: hspr Version: 0.2 License: PublicDomain Author: Niklas Broberg Maintainer: nibro(a)cs.chalmers.se Build-Depends: base, haskell98, haskelldb, hsp GHC-Options: -F -pgmFtrhsx Executable: hspr Main-Is: HSPR.hs -------------------------------------------- > runhaskell Setup.hs configure Error: Non-empty library, but empty exposed modules list. Cabal may not build this library correctly The difference is the added GHC-Options, same thing happens if I try to add an Extensions rule. Is this an intended behavior? /Niklas

2 3

RE: exposed packages and cabal depends
by Simon Marlow 09 Apr '05

09 Apr '05

Hi Alex, On 08 April 2005 16:09, S. Alexander Jacobson wrote: > This choice adds complexity without adding value. It adds a lot of value: it will prevent untracked dependencies from creeping into Cabal packages. That's gotta be a good thing, right? With -hide-all-packages, there are no exposed packages by defualt, which is what you were arguing for! > The simplest approach would be for -i to take precedence over > default packages. That only partly solves the problem - you can still write source code that depends on environmental settings by importing modules from exposed packages. Even so, it seems a reasonable suggestion. We thought about it, but discovered that it leads to more problems: you can allow a local module M to override a module M in a package P only as long as nothing from P is required by any module in the current program. To know that, you have to examine all the modules in the current program, which is difficult when doing separate compilation. You could end up with some modules compiled in a context that thinks M is a local module, and others compiled thinking M comes from package P. Bad things probably happen, and it's not clear that we can always tell that this has happened before delivering the executable. At the least, the error turns up at unpredictable points (eg. when you turn optimisation on). > The -hide-all-packages approach creates much higher risk of testing > in a package context out of sync with the world at large. Hmm, I don't think you've understood what -hide-all-packages does. It gives you a *consistent* view of packages across all installations: it eliminates differences between what packages are exposed by default at different sites. That is a Good Thing. You'd probably be more comfortable if -hide-all-packages was the default. Well, I'm not going to argue with that, except to say that it probably will be the default when using Cabal. > What we really need is a community consensus system (wiki or dns) > for mapping module names to package locations and for > implementations to interpret modules in this context. Hackage will effectively provide a function from Module -> [Package] that's what you want, right? But we don't want a single centralised registry for module names, so it will also be possible to get packages from source other than Hackage. > Package names that are not URLs add complexity without value. Just think of all our existing packages as having an implicit prefix of http://haskell.org/packages/ (or something). Does that help? We'll get full URLs as package names in the future (at least, that's what I'd like to see). Cheers, Simon

3 2

RE: exposed packages and cabal depends
by Simon Marlow 09 Apr '05

09 Apr '05

On 31 March 2005 23:15, Isaac Jones wrote: > Ian Lynagh mentioned this problem to me a few weeks ago and Alex's > thread reminded me of it. This doesn't really relate to Alex's > proposal, so I'm starting a new thread. > > Let's say that from GHC's perspective, package A is exposed and > package B is hidden. Angela Author creates package C which > build-depends on both A and B. She incorrectly lists the > build-depends on B, but not on A. While building, she doesn't notice, > since A is an exposed package. > > This kinda sucks, since it would be nice to check build-dependencies > at build time. This is very similar to the situation in Debian where > a packager doesn't realize that there's a build-dependency on A, since > she's already got it installed, but when the auto-builders get to it, > the build fails since they don't have A installed. > > I don't know how to solve this; is there a way to require explicit > -package flags for all packages in ghc 6.4? If there were, say > -fhide-all-packages, then cabal could use that while building and > Angela would catch the error of her ways much earlier. I've now added -hide-all-packages. Cabal can start using this from GHC 6.4.1. Cheers, Simon

2 1

Data.Map, Data.Set working together
by Cale Gibbard 08 Apr '05

08 Apr '05

It just crossed my mind that due to the apparent similarity in implementation it might be possible to implement some extra functions on Maps which take Sets as parameters and which are more efficient than first going through lists. For instance: restrict :: Set a -> Map a b -> Map a b which would restrict the map's domain to the given set -- basically set intersection, but one of the objects is a map. fromFunction :: Set a -> (a -> b) -> Map a b Build a finite map in the obvious way from a set and a function. (basically, restrict the function on the potentially infinite domain a to the finite one given by the set, and record the result as a finite map) Depending on the way things work internally, perhaps the constructed Map could inherit the tree structure of the Set directly. Thoughts? I haven't really looked at the code, but it seems plausible that these sorts of operations where a Set and Map are involved could be done in a better fashion than going through association lists. - Cale

5 8

dead link on libraries page
by Benjamin Franksen 08 Apr '05

08 Apr '05

Hello, the libraries page (http://www.haskell.org/libraries/) lists "Sven's Haskell Libaries for interfacing to DaVinci, HTML/Cgi, OpenGL; a module for option handling, a Haskell parser, a binary library." The link is dead. Cheers, Ben

1 0

New function candidate for Network.URI
by Graham Klyne 08 Apr '05

08 Apr '05

Using Network.URI, I find I quite often want to do relative URI calculations on URI strings without explicitly handling URI values. I have a candidate function that might be added to Network.URI, with test cases: [[ -- Create a new URI string that is a supplied local URI relative -- to a given base URI, both supplied as strings. relativeToStr :: String -> String -> Maybe String relativeToStr rel base = do -- in Maybe monad: { b <- parseURIReference base ; r <- parseURIReference rel ; u <- r `relativeTo` b ; return $ (uriToString id u) "" } test_rs1 = ("foo" `relativeToStr ` "zz://auth/base/") == Just "zz://auth/base/foo" test_rs2 = ("abs://a/u" `relativeToStr ` "zz://auth/base/") == Just "abs://a/u" test_rs3 = ("[][]" `relativeToStr ` "zz://auth/base/") == Nothing test_rs4 = ("foo" `relativeToStr ` "[][]") == Nothing test_rs5 = ("foo" `relativeToStr ` "bar") == Just "foo" test_rs = and [test_rs1,test_rs2,test_rs3,test_rs4,test_rs5] ]] (I'm not sure the name is the best possible choice.) Is this kind of helper function useful to add to Network.URI? Maybe also a corresponding: relativeFromStr :: String -> String -> Maybe String ? #g ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0