November 2004 - Libraries

RE: Cabal interface changes proposal
by Simon Marlow 23 Nov '04

23 Nov '04

On 23 November 2004 09:04, Krasimir Angelov wrote: > I am strongly advocating to have one to one > correspondence between Cabal packages and Haskell > packages. It would be even simpler if in each Cabal > package we have only one library or only one > executable. I like the Simon's proposal of shipments. > If we need a package with one library and multiple > executables we can use shipment too. Currently we have > Setup.description for each package. I propose to > change the name to Package.description and to > introduce Shipment.description file. The latter will > contain the name of shipment and the list of > Package.description files in it. The Cabal package > should export defaultMain and defaultShipmentMain > functions, where the former builds a single package > while the latter builds a shipment. Sounds good. I like the fact that the existing package description can be simplified. Presumably the Setup.lhs in the shipment works by invoking Setup.lhs from each of the packages, in which case we'll need to expand Cabal with a notion of a local package database. Perhaps: ./setup configure --pkg-db=PATH satisifies dependencies using packages from PATH, layered on top of the global & user packages as normal. ./setup register --pkg-db=PATH registers in the package database at PATH. The default is still to register in the global db, or the user db if --user is given. We need this to support the "directory of packages" idea too (PATH can be either a directory or the name of a file containing the database). Cheers, Simon

1 0

Cabal interface changes proposal
by Isaac Jones 23 Nov '04

23 Nov '04

Hi Gang, I would like to make a few interface changes to Cabal, but I want to get some feedback first, and to give a heads-up. By "target" I mean "place where the package will end up getting installed" rather than the build machine, which may be different. -= New commands =- ./setup copy (similar to ./setup install, with or without a prefix. Never runs register or postinst) ./setup test (runs a user-supplied IO () function) ./setup postinst (guaranteed to run on the target after register, runs a user-supplied IO () function) -= Changes to existing commands =- ./setup install now will not have the --install-prefix flag. Use "copy" instead. Install will always run register and postinst. ./setup register may or may not run on the target machine -= Changes to package description file =- I can imagine a cabal package (wxhaskell) which has more than one "haskell package" / library in a single source tree. We might like to have more than one "haskell package" per "cabal package". That is, installing a single cabal package, Foo, might make "-package bar -package bang" available to the compilers. So I propose that we add the ability to have multiple libraries, just as we can currently do multiple executables. Right now, there's no great way to do this, besides perhaps having multiple directories with multiple Setup.lhs files. The downside to this is that it might make the description file a little more complex, though in the simple case, it won't change much. What do folks think? -= Notes =- Copy function: I'm not crazy about the way that "install" has different behavior depending on the --install-prefix flag. In particular, without the flag, it doesn't register, with the flag, it does. This is convenient, but inconsistent. I hope that this command will clean it up. It was Ian Lynagh's idea. Postinst: This is related to the discussion about ghc-pkg on the libraries list. The idea is to have a command that's guaranteed to run on the target. This frees up the register command to be more flexible. By default, postinst will do nothing, just like "./setup test". peace, isaac

2 1

RE: Cabal interface changes proposal
by Simon Marlow 22 Nov '04

22 Nov '04

On 21 November 2004 01:15, Isaac Jones wrote: > I would like to make a few interface changes to Cabal, but I want to > get some feedback first, and to give a heads-up. > > By "target" I mean "place where the package will end up getting > installed" rather than the build machine, which may be different. > > -= New commands =- > > ./setup copy (similar to ./setup install, with or without a > prefix. Never runs register or postinst) > ./setup test (runs a user-supplied IO () function) > ./setup postinst (guaranteed to run on the target after register, runs > a user-supplied IO () function) > > -= Changes to existing commands =- > > ./setup install now will not have the --install-prefix flag. Use > "copy" instead. Install will always run register and postinst. > > ./setup register may or may not run on the target machine This all looks good to me. > -= Changes to package description file =- > > I can imagine a cabal package (wxhaskell) which has more than one > "haskell package" / library in a single source tree. > > We might like to have more than one "haskell package" per "cabal > package". That is, installing a single cabal package, Foo, might make > "-package bar -package bang" available to the compilers. > > So I propose that we add the ability to have multiple libraries, just > as we can currently do multiple executables. > > Right now, there's no great way to do this, besides perhaps having > multiple directories with multiple Setup.lhs files. The downside to > this is that it might make the description file a little more complex, > though in the simple case, it won't change much. What do folks think? First of all, I should say that I doing see an urgent need for this. Packaging systems such as Debian, *BSD & Gentoo manage dependencies and can build multiple packages in the right order just fine, and have support for "collection" packages (eg. installing "kde" causes a bunch of sub-packages to be installed). Introducing this functionality in Cabal would be redundant on these systems. My feeling has always been that we shouldn't duplicate functinoality that is available in a good package system. Given that, if you really want to do this, then here are my thoughts: basically it would mean introducing another "layer" of packaging, albeit one that lives only at the Cabal level, which Haskell compilers know nothing of. Here are some issues that initially spring to mind: - Packages in this layer should be called something other than packages, to avoid confusion (I'll call them shipments, for now). - Should the shipment have a name of its own? - Should it have a version? - Presumably the packages in a shipment depend on each other. That means we need to "temporarily" install packages so we can build against them. This isn't something that the current package proposal really supports, but I think it would be useful. Cheers, Simon

2 1

RE: proposal for ghc-pkg to use a directory of .conf files
by Simon Marlow 22 Nov '04

22 Nov '04

On 21 November 2004 00:56, Isaac Jones wrote: > The systems that would want to do this kind of thing, such as Debian, > have other mechanisms for deciding whether packages conflict, etc. IIRC, this is the argument I just used against adding support for multiple libraries in Cabal, so I guess I agree :-D > Over-all I'm kinda neutral about whether HC-pkg needs to be an opaque > interface to the packaging system. What are the advantages to this? Well, for one thing it allows us flexibility in how we store the package database. In GHC, I'm using the show/read form of [InstalledPackageInfo] to store the database, but it'd be nice if I could use binary serialisation in the future. To support a directory of config files, we don't have to expose the complete format, though. As long as hc-pkg can process the InstalledPackageInfo to produce the native format into a file, then we just ship that file with the distribution. So I'm fine with this, as long as we're not specifying the contents of the *.conf file. Cheers, Simon

1 0

RE: proposal for ghc-pkg to use a directory of .conf files
by Simon Marlow 21 Nov '04

21 Nov '04

On 09 November 2004 17:36, Isaac Jones wrote: > "Simon Marlow" <simonmar(a)microsoft.com> writes: > >> On 08 November 2004 18:47, Duncan Coutts wrote: >> >>> We can use ghc-pkg at the build / install-into-temp phase to create >>> the $(package).conf files under >>> $TMP_INSTALL_ROOT/usr/lib/ghc-$VER/package.conf.d/ and then final >>> installation is jsut merging files without any post-install calls to >>> ghc-pkg to modify installed files (ie the global ghc package.conf >>> file) >>> >>> So we can still keep the abstraction of $HC-pkg and gain simpler >>> packaging stuff. >> >> Ok, sounds reasonable. >> >> I'm going to be working on the package support in ghc and ghc-pkg to >> improve support for Cabal, so let's do this at the same time. > > As a Debian packager, I like the idea of changing the way HC-PKG > handles individual packages. > > The question in my mind is whether we want to execute any code on the > install target. Previously, I have thought of "./setup register" as > being a step that happens on the target, no matter what. So if Marcus > Makefile wants to do something specifically for the target at install > time, this is where he could do it. > > If we go this route and have the package registration happen at > install-in-temp time, then we don't have any standard way to run a > post-install script. Some people may prefer that we never execute > anything from Cabal on the target, but I would prefer to leave that > ability. > > One solution would be to move the registration step into > install-into-temp time, as above, but to add another standard command > to Cabal like "./setup postinstall" and maybe some others "preinst, > prerm, postrm" as in Debian. > > This would solve both problems; haskell packages installed with a > packaging system like Debian would usually just be moving files into > place, but if Marcus or Angela really needed to run something on the > target, this is how they'd do it. If ./setup register isn't going to run at install time, then I agree we might want ./setup postinst too. There's another thing that bothers me though: when you install a package using hc-pkg, a number of checks are made: 1. there isn't already a package with that name/version 2. If the package is to be exposed, then the modules provided by the package don't overlap with another exposed package. 3. if an older version of the package is already exposed, then the older one is supposed to be hidden in favour of the new one Since with the proposed change hc-pkg isn't running on the target system, it can't make any of these tests. GHC can detect at run-time that you have overlapping packages, but then it might not be possible to make changes to the package database (you might need to 'su' in order to do it). Cheers, Simon

2 1

Cabal - C sources in a subdirectory
by Einar Karttunen 17 Nov '04

17 Nov '04

Hello I have problems with Cabal failing to build simple packages if C-Sources are inside a subdirectory. With a Setup.description like: Name: Foo Version: 0.1 License: LGPL Modules: Foo Exposed-Modules: Foo Extensions: ForeignFunctionInterface C-Sources: c_src/foo.c Build creates c_src/foo.o => dist/build/foo.o (from the C-Sources build instruction in Distribution.Simple.Build). In the later stage it tries to look for the object file in dist/build/c_src/foo.o which fails. A complete minimal example is at http://www.cs.helsinki.fi/u/ekarttun/haskell/cfail.tar.gz - Einar Karttunen

2 4

Data.FiniteMap proposed addition, bug fix
by Graham Klyne 15 Nov '04

15 Nov '04

At 13:06 09/11/04 +0000, Simon Marlow wrote: >On 09 November 2004 12:45, Graham Klyne wrote: > > > I'd like to propose an addition to the FiniteMap module in the form > > of a monadic version of plusFM_C. The proposed implementation is > > pretty much a copy of the existing implementation within a do block. > >Data.FiniteMap will shortly be deprecated in favour of DData.Map (which >will be renamed to Data.Map when it is imported). Perhaps you'd like to >reformulate the proposal using Data.Map instead? I added tests for the monad threading order, and realized it was a complete red herring. (I got it in my head that when reversing the parameters to optimize the hedge union, I needed to reverse the monad ordering. Duh!) So the two sets of functions for hedgeUnionWithKeyM* were unnecessary, unless it really is desired to have right-to-left monad threading #g -- Proposed addition: [[ {-------------------------------------------------------------------- Monadic union --------------------------------------------------------------------} -- | /O(n+m)/. Monadic version of union with a combining function. -- The implementation uses the efficient /hedge-union/ algorithm. -- -- The combining function returns a monadic value, which is threaded though -- the combined elements in key order, yielding a Map that is bound -- to the same monadic type. The intended use for this is with a Maybe -- monad, allowing result returned to be Nothing if any of the attempted -- combinations of key values return Nothing. Could also be usefully used -- with an error or state monad. -- unionWithM :: (Ord k, Monad m) => (a -> a -> m a) -> Map k a -> Map k a -> m (Map k a) unionWithM f m1 m2 = unionWithKeyM (\k x y -> f x y) m1 m2 -- | /O(n+m)/. Monadic version of @unionWithKey@. -- The implementation uses the efficient /hedge-union/ algorithm. -- -- See @unionWithM@ for further details. -- unionWithKeyM :: (Ord k, Monad m) => (k -> a -> a -> m a) -> Map k a -> Map k a -> m (Map k a) unionWithKeyM f Tip t2 = return t2 unionWithKeyM f t1 Tip = return t1 unionWithKeyM f t1 t2 -- hedge-union is more efficient on (bigset `union` smallset) | size t1 >= size t2 = hedgeUnionWithKeyML f (const LT) (const GT) t1 t2 | otherwise = hedgeUnionWithKeyML flipf (const LT) (const GT) t2 t1 where flipf k x y = f k y x -- Left version of monadic hedgeUnionWithKey. -- (Monad is threaded left-to-right in tree) -- hedgeUnionWithKeyML :: (Ord k, Monad m) => (k -> a -> a -> m a) -> (k -> Ordering) -> (k -> Ordering) -> Map k a -> Map k a -> m (Map k a) hedgeUnionWithKeyML f cmplo cmphi t1 Tip = return t1 hedgeUnionWithKeyML f cmplo cmphi Tip (Bin _ kx x l r) = return $ join kx x (filterGt cmplo l) (filterLt cmphi r) hedgeUnionWithKeyML f cmplo cmphi (Bin _ kx x l r) t2 = do { newl <- hedgeUnionWithKeyML f cmplo cmpkx l lt ; newx <- case found of Nothing -> return x Just y -> f kx x y ; newr <- hedgeUnionWithKeyML f cmpkx cmphi r gt ; return $ join kx newx newl newr } where cmpkx k = compare kx k lt = trim cmplo cmpkx t2 (found,gt) = trimLookupLo kx cmphi t2 -- And some test cases: fm1 = fromList [(1,["a"]),(2,["b","c"]),(3,["d"])] fm2 = fromList [(1,["b","c"]),(4,["e"])] fm3 = fromList [(1,["d","e"]),(2,["c","d"]),(4,["f"])] -- Test function returns Nothing if list values have a member in common: comb ovs nvs | null (List.intersect ovs nvs) = Just (ovs++nvs) | otherwise = Nothing fm12 = Just $ fromList [(1,["a","b","c"]),(2,["b","c"]),(3,["d"]),(4,["e"])] fm13 = Nothing fm23 = Just $ fromList [(1,["b","c","d","e"]),(2,["c","d"]),(4,["e","f"])] fmt1 = unionWithM comb fm1 fm2 == fm12 fmt2 = unionWithM comb fm1 fm3 == fm13 fmt3 = unionWithM comb fm2 fm3 == fm23 -- Test function uses state to accumulate combined keys combk :: k -> [a] -> [a] -> State.State [k] [a] combk k ov nv = State.State (\s -> (ov++nv,s++[k])) fmk12 = ( fromList [(1,["a","b","c"]),(2,["b","c"]),(3,["d"]),(4,["e"])] , [1] ) fmk13 = (fromList [(1,["a","d","e"]),(2,["b","c","c","d"]),(3,["d"]),(4,["f"])] ,[1,2] ) fmk23 = (fromList [(1,["b","c","d","e"]),(2,["c","d"]),(4,["e","f"])] ,[1,4]) fmt4 = State.runState (unionWithKeyM combk fm1 fm2) [] == fmk12 fmt5 = State.runState (unionWithKeyM combk fm1 fm3) [] == fmk13 fmt6 = State.runState (unionWithKeyM combk fm2 fm3) [] == fmk23 fmtall = and [fmt1,fmt2,fmt3,fmt4,fmt5,fmt6] ]] ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact _______________________________________________ Libraries mailing list Libraries(a)haskell.org http://www.haskell.org/mailman/listinfo/libraries ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

3 4

GHC.Unicode
by Ross Paterson 11 Nov '04

11 Nov '04

This module diverges slightly from Haskell 98, which has isAlpha c = isUpper c || isLower c I'd also suggest using the H98 definitions of isDigit and isControl, even though glibc does the same thing.

1 0

RE: proposal for ghc-pkg to use a directory of .conf files
by Simon Marlow 09 Nov '04

09 Nov '04

On 08 November 2004 18:47, Duncan Coutts wrote: > We can use ghc-pkg at the build / install-into-temp phase to create > the $(package).conf files under > $TMP_INSTALL_ROOT/usr/lib/ghc-$VER/package.conf.d/ and then final > installation is jsut merging files without any post-install calls to > ghc-pkg to modify installed files (ie the global ghc package.conf > file) > > So we can still keep the abstraction of $HC-pkg and gain simpler > packaging stuff. Ok, sounds reasonable. I'm going to be working on the package support in ghc and ghc-pkg to improve support for Cabal, so let's do this at the same time. Cheers, Simon

2 1

RE: Proposed addition to Data.FiniteMap
by Graham Klyne 09 Nov '04

09 Nov '04

At 13:06 09/11/04 +0000, Simon Marlow wrote: >On 09 November 2004 12:45, Graham Klyne wrote: > > > I'd like to propose an addition to the FiniteMap module in the form > > of a monadic version of plusFM_C. The proposed implementation is > > pretty much a copy of the existing implementation within a do block. > >Data.FiniteMap will shortly be deprecated in favour of DData.Map (which >will be renamed to Data.Map when it is imported). Perhaps you'd like to >reformulate the proposal using Data.Map instead? OK, the proposed addition is below. I haven't fully tested the monad threading order. (Hmm.. using a state monad to collect a list of keys might be a way to do that.) Deprecation notwithstanding, given that FiniteMap is the current library module, and that it would be nice if my code works out-of-the-box with the next public release of the libraries, would there be any objection to also adding my original proposal to the current FiniteMap module? #g -- Proposed addition: [[ {-------------------------------------------------------------------- Monadic union --------------------------------------------------------------------} -- | Monadic version of union with a combining function. -- -- The combiner function returns a monadic value, which is threaded though -- the combined elements in key order, yielding a Map that is bound -- to the same monadic type. The intended use for this is with a Maybe -- monad, allowing result returned to be Nothing if any of the attempted -- combinations of key values return Nothing. Could also be usefully used -- with an error or state monad. -- unionWithM :: (Ord k, Monad m) => (a -> a -> m a) -> Map k a -> Map k a -> m (Map k a) unionWithM f m1 m2 = unionWithKeyM (\k x y -> f x y) m1 m2 -- | Monadic version of unionWithKey. -- -- (See unionWithM) -- unionWithKeyM :: (Ord k, Monad m) => (k -> a -> a -> m a) -> Map k a -> Map k a -> m (Map k a) unionWithKeyM f Tip t2 = return t2 unionWithKeyM f t1 Tip = return t1 unionWithKeyM f t1 t2 -- hedge-union is more efficient on (bigset `union` smallset) | size t1 >= size t2 = hedgeUnionWithKeyML f (const LT) (const GT) t1 t2 | otherwise = hedgeUnionWithKeyMR f (const LT) (const GT) t2 t1 -- Left version of monadic hedgeUnionWithKey -- (Monad is threaded left-to-right in tree) hedgeUnionWithKeyML :: (Ord k, Monad m) => (k -> a -> a -> m a) -> (k -> Ordering) -> (k -> Ordering) -> Map k a -> Map k a -> m (Map k a) hedgeUnionWithKeyML f cmplo cmphi t1 Tip = return t1 hedgeUnionWithKeyML f cmplo cmphi Tip (Bin _ kx x l r) = return $ join kx x (filterGt cmplo l) (filterLt cmphi r) hedgeUnionWithKeyML f cmplo cmphi (Bin _ kx x l r) t2 = do { newl <- hedgeUnionWithKeyML f cmplo cmpkx l lt ; newx <- case found of Nothing -> return x Just y -> f kx x y ; newr <- hedgeUnionWithKeyML f cmpkx cmphi r gt ; return $ join kx newx newl newr } where cmpkx k = compare kx k lt = trim cmplo cmpkx t2 (found,gt) = trimLookupLo kx cmphi t2 -- Right version of monadic hedgeUnionWithKey -- (Monad is threaded Right-to-Left in tree) hedgeUnionWithKeyMR :: (Ord k, Monad m) => (k -> a -> a -> m a) -> (k -> Ordering) -> (k -> Ordering) -> Map k a -> Map k a -> m (Map k a) hedgeUnionWithKeyMR f cmplo cmphi t1 Tip = return t1 hedgeUnionWithKeyMR f cmplo cmphi Tip (Bin _ kx x l r) = return $ join kx x (filterGt cmplo l) (filterLt cmphi r) hedgeUnionWithKeyMR f cmplo cmphi (Bin _ kx x l r) t2 = do { newr <- hedgeUnionWithKeyMR f cmpkx cmphi r gt ; newx <- case found of Nothing -> return x Just y -> f kx y x ; newl <- hedgeUnionWithKeyMR f cmplo cmpkx l lt ; return $ join kx newx newl newr } where cmpkx k = compare kx k lt = trim cmplo cmpkx t2 (found,gt) = trimLookupLo kx cmphi t2 -- And some test cases: fm1 = fromList [(1,["a"]),(2,["b","c"]),(3,["d"])] fm2 = fromList [(1,["b","c"]),(4,["e"])] fm3 = fromList [(1,["d","e"]),(2,["c","d"]),(4,["f"])] fm12 = Just $ fromList [(1,["a","b","c"]),(2,["b","c"]),(3,["d"]),(4,["e"])] fm13 = Nothing fm23 = Just $ fromList [(1,["b","c","d","e"]),(2,["c","d"]),(4,["e","f"])] -- Test function returns Nothing if list values have a member in common: comb ovs nvs | null (List.intersect ovs nvs) = Just (ovs++nvs) | otherwise = Nothing fmt1 = unionWithM comb fm1 fm2 == fm12 fmt2 = unionWithM comb fm1 fm3 == fm13 fmt3 = unionWithM comb fm2 fm3 == fm23 fmtall = and [fmt1,fmt2,fmt3] ]] ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0