April 2005 - Libraries

argument order of functions in Data.Bits
by Henning Thielemann 07 Apr '05

07 Apr '05

The version of Data.Bits which is shipped with GHC-6.2 is marked as experimental. So changes in the API are still possible? :-] My experiments suggest that the functions should have a different order of arguments. Many of the functions of this module alter some bits in a machine word, thus they can be considered as update functions and their type signature should end with a -> a. Then we could easily combine several operations this way shiftL 2 . clearBit 7 . setBit 4 . setBit 1 instead of flip shiftL 2 . flip clearBit 7 . flip setBit 4 . flip setBit 1 or (`shiftL` 2) . (`clearBit` 7) . (`setBit` 4) . (`setBit` 1) . I don't see the benefit of urging the programmer to use infix notation instead of prefix notation here, e.g. (w `shift` 2) instead of (shift 2 w).

2 1

RE: argument order of functions in Data.Bits
by Simon Marlow 07 Apr '05

07 Apr '05

On 07 April 2005 07:54, Henning Thielemann wrote: > The version of Data.Bits which is shipped with GHC-6.2 is marked as > experimental. So changes in the API are still possible? :-] > > My experiments suggest that the functions should have a different > order of arguments. Many of the functions of this module alter some > bits in a machine word, thus they can be considered as update > functions and their type signature should end with a -> a. Then we > could easily combine several operations this way > > shiftL 2 . clearBit 7 . setBit 4 . setBit 1 > > instead of > flip shiftL 2 . flip clearBit 7 . flip setBit 4 . flip setBit 1 > > or > (`shiftL` 2) . (`clearBit` 7) . (`setBit` 4) . (`setBit` 1) > . > > I don't see the benefit of urging the programmer to use infix notation > instead of prefix notation here, e.g. (w `shift` 2) instead of > (shift 2 w). On the whole I agree, but I'm inclined against changing this because it would break so much code gratuitously. I vote for just putting this down to a small mistake in the original design, and leaving it. If there's overwhelming support for the change of course we'll make it, but I doubt there will be. Also the order of the arguments to shift matches C's shift operators (although they are real infix operators, of course). Cheers, Simon

3 2

RE: Testing library modules
by Graham Klyne 07 Apr '05

07 Apr '05

At 11:59 07/04/05 +0100, Simon Marlow wrote: >In GHC you can say '-hide-package network' (or, perhaps better >'-ignore-package network') to remove the network package from the module >namespace. It will then use the local version instead. So it does! I *thought* I had tried that (per warning message from the compiler), but somehow it was still picking up the installed files. But on a renewed attempt, it now works fine despite having the installed files reinstated. Thanks. The proposed change to URI is now in CVS, the correct version having been tested (I think ;-). There's also an updated test harness. #g ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

RE: Testing library modules
by Simon Marlow 07 Apr '05

07 Apr '05

On 07 April 2005 11:18, Graham Klyne wrote: > I've just discovered that my recent checkins of Network.URI were not > safe. I thought that I had run the changes against my test suite, > but just discovered that the test suite was actually being compiled > against the version of Network.URI shipped with GHC 6.4. > > I had expected the "-i" option on the GHC command line (see below) to > override the shipped version of Network.URI. > > As it happens, I've now checked the new module is OK by forcibly > renaming the Network.URI files that were installed with GHC 6.4. But > this raises a concern for me: how can I safely and reliably test a > replacement version of a GHC-shipped library module? In GHC you can say '-hide-package network' (or, perhaps better '-ignore-package network') to remove the network package from the module namespace. It will then use the local version instead. Cheers, Simon

1 0

Testing library modules
by Graham Klyne 07 Apr '05

07 Apr '05

I've just discovered that my recent checkins of Network.URI were not safe. I thought that I had run the changes against my test suite, but just discovered that the test suite was actually being compiled against the version of Network.URI shipped with GHC 6.4. I had expected the "-i" option on the GHC command line (see below) to override the shipped version of Network.URI. As it happens, I've now checked the new module is OK by forcibly renaming the Network.URI files that were installed with GHC 6.4. But this raises a concern for me: how can I safely and reliably test a replacement version of a GHC-shipped library module? #g -- Batch file used to compile test suite on Windows: [[ set INCL=E:\HaskellCVS\fptools\libraries\network;C:\Dev\Haskell\Lib\HUnit;C:\Dev\Haskell\Lib\Parsec set GHCC=C:\DEV\ghc\ghc-6.4\bin\ghc %GHCC% --make -fglasgow-exts -ignore-package network-1.0. -i%INCL% -o %1.exe %1 -main-is %1.main ]] ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

RE: Network.URI -- typo in new exported function name
by Simon Marlow 07 Apr '05

07 Apr '05

On 07 April 2005 10:48, Graham Klyne wrote: > While cleaning up some comments in the new Network.URI module, I > noticed that there is a typo in the name of one of the new exported > functions, viz 'parseabsoluteURI', which was meant to be > 'parseAbsoluteURI'. > > I am guessing that this is not widely used; is it acceptable to fix > this in CVS now that the library has been publicly released (keeping > the current name as a deprecated synonym)? I guess we could bend the rules for something this small, and it's better to fix it now rather than wait until there's lots of code dependent on the old name. Please go ahead. Cheers, Simon

1 0

Network.URI -- typo in new exported function name
by Graham Klyne 07 Apr '05

07 Apr '05

While cleaning up some comments in the new Network.URI module, I noticed that there is a typo in the name of one of the new exported functions, viz 'parseabsoluteURI', which was meant to be 'parseAbsoluteURI'. I am guessing that this is not widely used; is it acceptable to fix this in CVS now that the library has been publicly released (keeping the current name as a deprecated synonym)? #g ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

Cabal Bug
by Dominic Steinitz 07 Apr '05

07 Apr '05

When I try to build using ./Setup build I get the following error: Compiling Codec.Encryption.DESAux ( ./Codec/Encryption/DESAux.hs, dist/build/./Codec/Encryption/DESAux.o ) ./Codec/Encryption/DESAux.hs:19:0: Illegal instance declaration for `Num [Bool]' (The instance type must be of form (T a b c) where T is not a synonym, and a,b,c are distinct type variables) In the instance declaration for `Num [Bool]' ./Codec/Encryption/DESAux.hs:21:0: Illegal instance declaration for `Bits [Bool]' (The instance type must be of form (T a b c) where T is not a synonym, and a,b,c are distinct type variables) In the instance declaration for `Bits [Bool]' So I added Ghc-options: -fglasgow-exts to the cabal file. Now this module compiles but I still get the same error on a different module and I know -fglasgow-exts allows this to compile when I do it by hand. ./Codec/ASN1/TLV.hs:140:0: Illegal instance declaration for `Binary (State ([Octet], Offset))' (The instance type must be of form (T a b c) where T is not a synonym, and a,b,c are distinct type variables) In the instance declaration for `Binary (State ([Octet], Offset))' In fact, this seems to be the second time this module has been compiled and the first it completed successfully. Compiling Codec.ASN1.TLV ( ./Codec/ASN1/TLV.hs, dist/build/./Codec/ASN1/TLV.o ) Compiling Codec.ASN1.X509 ( ./Codec/ASN1/X509.hs, dist/build/./Codec/ASN1/X509.o ) Why is cabal recompiling sources and why am I getting errors on the second compilation? The sources and the cabal file are available via: darcs get http://www.haskell.org/Codec Dominic.

3 5

RE: Stop untracked dependencies!
by Simon Marlow 05 Apr '05

05 Apr '05

On 01 April 2005 20:31, S. Alexander Jacobson wrote: > If we are in agreement about the overall goal, then I'll move on to > the specific issues that need addressing: > > * We Need Free Packaging > > Anyone should be able to create a package at any time for use in their > projects without having having to negotiate with any particular > centralized authority or community. Absolutely! I suggested that URLs should be valid package names before, and I still think that's a reasonable solution. Using Hackage will be completely optional. No-one is about to tell you how and where you can distribute your own packages. > * Free Packaging Means We To Handle Collisions in Module Namespace > > Anyone should be able to use any combination of packages they want in > any of their programs at any time without worrying about whether those > packages export the same module name. > > Simon says: >> We don't allow programs to contain two modules with the same name, >> for good reasons. > > I'm not asking for that. I am totally ok with each module name > mapping to one and only one implementation per program. I just want > to be able choose that implementation even when my program uses two > packages that both happen to export the same module name. I'm afraid it's the same thing, because packages are indivisible. Packages are indivisible for two reasons: - modules within a package depend on each other. If you need one, you need all its dependencies too. Packages don't tell you what their inter-module dependencies are. That's part of the reason for having packages: to decrease the granularity of dependencies. - technical reasons: a package may be shipped as a shared library, which is indvisible and populates the global symbol table with all of its symbols. You can't pick and choose which ones you want. (static libraries allow a kind of shadowing, but it would also lead to problems I think). So to do what you're asking for, we'd have to support overlapping modules. I agree with your reasoning for wanting to be able to do this: we need as much freedom to be able to combine packages from third parties as possible. Unfortunately, it requires a non-trivial technological leap. > * We Need to Associate Module Names with Specific Packages You could specify the connection between modules and packages if you want. Given that you can't use two packages containing the same module, the information would be redundant (apart from documentation) right now. > * Default Exposed Packages Are Untenable > > Although, I agree that fussing with a -packages command line is > annoying, the need to associate module names with packages ids > makes the existing default exposed package compromise untenable. > > But, all is not lost! If you choose (c) above, nothing stops you from > having a default global "Packages" file. Then your marginal work is > just to supply a package name in your import statements e.g.: > > import HaXML HaXML.XML.Parse Your global "Packages" file is a source of untracked dependencies, isn't it? Don't I have to provide that file with every scrap of source code I publish? How do I merge someone else's Packages file with mine? In what way is it better than specifying dependencies in a .cabal file which is already shipped with a package? Also, the Haskell module hierarchy is supposed to reflect functionality, whereas package names are purely administrative. This is a reason for not including package names in source code. There are some other administrative details which amount to dependencies, and which are tracked by .cabal files but not by your proposal: - language extensions - include paths - library paths - C library dependencies etc. some of these you can put in {-# GHC_OPTIONS #-}, but not all. And in any case you probably want to specify them once per package, not once per module. > * In Any Case, Default Exposed Packages Are Also a Poor Compromise. > > I don't want to have to worry about accumulating untracked dependcies > when I am doing quickie work with GHCi. I want my dependencies > checked every step of the way and should not have to round trip > through a Cabal packaging step just to verify them. Your dependencies are still untracked. But anyway, I see the point. If you want package dependencies checked as you write source code, here are a couple of possibilities: - GHC could have a checked pragma on each import, eg: import {-# PACKAGE base >= 1.0 #-} Data.List I don't want to write these in my source code personally, but it would be easy enough to implement. It could be made mandatory, optionally. Or: - GHCi could have a Cabal front end. i.e. you give GHCi a .cabal file instead of specifying source files directly, and it checks dependencies. This is easy enough to implement too. In both cases you would have to restart GHCi if you made use of a non-exposed package, thoughl. > * Conclusion: Freedom is good. No argument from me on that one! Cheers, Simon

2 1

Data.Map: monadic combining functions
by Graham Klyne 05 Apr '05

05 Apr '05

While discussing Data.Map and Data.Set, I'd like to raise again a point I mentioned previously with respect to Data.FiniteMap: http://www.haskell.org/pipermail/libraries/2004-November/002670.html I since created a private copy of Data.Map with the functionality I desire [1]. At the time, concern was expressed that consistency would require a lot of new monadic functions to be defined. Is there any easy way the module interface could be revised to allow this kind of added functionality to be created externally to the module? #g -- [1] Monadic union for Data.Map: [[ {-------------------------------------------------------------------- Monadic union --------------------------------------------------------------------} -- | /O(n+m)/. Monadic version of union with a combining function. -- The implementation uses the efficient /hedge-union/ algorithm. -- -- The combining function returns a monadic value, which is threaded though -- the combined elements in key order, yielding a Map that is bound -- to the same monadic type. The intended use for this is with a Maybe -- monad, allowing result returned to be Nothing if any of the attempted -- combinations of key values return Nothing. Could also be usefully used -- with an error or state monad. -- unionWithM :: (Ord k, Monad m) => (a -> a -> m a) -> Map k a -> Map k a -> m (Map k a) unionWithM f m1 m2 = unionWithKeyM (\k x y -> f x y) m1 m2 -- | /O(n+m)/. Monadic version of @unionWithKey@. -- The implementation uses the efficient /hedge-union/ algorithm. -- -- See @unionWithM@ for further details. -- unionWithKeyM :: (Ord k, Monad m) => (k -> a -> a -> m a) -> Map k a -> Map k a -> m (Map k a) unionWithKeyM f Tip t2 = return t2 unionWithKeyM f t1 Tip = return t1 unionWithKeyM f t1 t2 -- hedge-union is more efficient on (bigset `union` smallset) | size t1 >= size t2 = hedgeUnionWithKeyML f (const LT) (const GT) t1 t2 | otherwise = hedgeUnionWithKeyML flipf (const LT) (const GT) t2 t1 where flipf k x y = f k y x -- Left version of monadic hedgeUnionWithKey. -- (Monad is threaded left-to-right in tree) -- hedgeUnionWithKeyML :: (Ord k, Monad m) => (k -> a -> a -> m a) -> (k -> Ordering) -> (k -> Ordering) -> Map k a -> Map k a -> m (Map k a) hedgeUnionWithKeyML f cmplo cmphi t1 Tip = return t1 hedgeUnionWithKeyML f cmplo cmphi Tip (Bin _ kx x l r) = return $ join kx x (filterGt cmplo l) (filterLt cmphi r) hedgeUnionWithKeyML f cmplo cmphi (Bin _ kx x l r) t2 = do { newl <- hedgeUnionWithKeyML f cmplo cmpkx l lt ; newx <- case found of Nothing -> return x Just y -> f kx x y ; newr <- hedgeUnionWithKeyML f cmpkx cmphi r gt ; return $ join kx newx newl newr } where cmpkx k = compare kx k lt = trim cmplo cmpkx t2 (found,gt) = trimLookupLo kx cmphi t2 {- -- Right version of monadic hedgeUnionWithKey -- (Monad is threaded right-to-left in tree) hedgeUnionWithKeyMR :: (Ord k, Monad m) => (k -> a -> a -> m a) -> (k -> Ordering) -> (k -> Ordering) -> Map k a -> Map k a -> m (Map k a) hedgeUnionWithKeyMR f cmplo cmphi t1 Tip = return t1 hedgeUnionWithKeyMR f cmplo cmphi Tip (Bin _ kx x l r) = return $ join kx x (filterGt cmplo l) (filterLt cmphi r) hedgeUnionWithKeyMR f cmplo cmphi (Bin _ kx x l r) t2 = do { newr <- hedgeUnionWithKeyMR f cmpkx cmphi r gt ; newx <- case found of Nothing -> return x Just y -> f kx y x ; newl <- hedgeUnionWithKeyMR f cmplo cmpkx l lt ; return $ join kx newx newl newr } where cmpkx k = compare kx k lt = trim cmplo cmpkx t2 (found,gt) = trimLookupLo kx cmphi t2 -} -- And some test cases: fm1 = fromList [(1,["a"]),(2,["b","c"]),(3,["d"])] fm2 = fromList [(1,["b","c"]),(4,["e"])] fm3 = fromList [(1,["d","e"]),(2,["c","d"]),(4,["f"])] -- Test function returns Nothing if list values have a member in common: comb ovs nvs | null (List.intersect ovs nvs) = Just (ovs++nvs) | otherwise = Nothing fm12 = Just $ fromList [(1,["a","b","c"]),(2,["b","c"]),(3,["d"]),(4,["e"])] fm13 = Nothing fm23 = Just $ fromList [(1,["b","c","d","e"]),(2,["c","d"]),(4,["e","f"])] fmt1 = unionWithM comb fm1 fm2 == fm12 fmt2 = unionWithM comb fm1 fm3 == fm13 fmt3 = unionWithM comb fm2 fm3 == fm23 -- Test function uses state to accumulate combined keys combk :: k -> [a] -> [a] -> State.State [k] [a] combk k ov nv = State.State (\s -> (ov++nv,s++[k])) fmk12 = ( fromList [(1,["a","b","c"]),(2,["b","c"]),(3,["d"]),(4,["e"])] , [1] ) fmk13 = (fromList [(1,["a","d","e"]),(2,["b","c","c","d"]),(3,["d"]),(4,["f"])] ,[1,2] ) fmk23 = (fromList [(1,["b","c","d","e"]),(2,["c","d"]),(4,["e","f"])] ,[1,4]) fmt4 = State.runState (unionWithKeyM combk fm1 fm2) [] == fmk12 fmt5 = State.runState (unionWithKeyM combk fm1 fm3) [] == fmk13 fmt6 = State.runState (unionWithKeyM combk fm2 fm3) [] == fmk23 fmtall = and [fmt1,fmt2,fmt3,fmt4,fmt5,fmt6] ]] ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0