new release of LIP prototype API
Greetings, I just uploaded a new release of the Library Infrastructure prototype API for Distribution.Package and the command-line utility "haskell-config". The API for package configuration is included below, FYI. I'm going to be focusing more on the Build stuff (rather than the packaging stuff) for a little while, but with luck, this can be near to finalization soon. The web page is here: http://www.haskell.org/libraryInfrastructure/ Feedback and volunteers for coding are always welcome :) peace, isaac ps. I'm using arch (tla) for version control, so if you've been wanting to check it out, now is a good time :) ------------------------------------------------------------ lookupPackage :: PackageDB -> PkgIdentifier -> Maybe PackageConfig addPackage :: PackageDB -> PackageConfig -> PackageDB delPackage :: PackageDB -> PkgIdentifier -> (PackageDB, Bool) allPackages :: PackageDB -> [PackageConfig] createPackageDB :: FilePath -> (PackageDB -> IO PackageDB) -> IO () withPackageDB :: FilePath -> (PackageDB -> IO (PackageDB, a)) -> IO a userPkgConfigLocation :: String -> FilePath userPkgConfigLocation home = home ++ "/.haskell/packages.conf" systemPkgConfigLocation :: FilePath systemPkgConfigLocation = "/etc/haskell/packages.conf" PackageConfig(..), PkgIdentifier(..), License(..), Dependency(..), PackageDB
I'm reading (belatedly) through your proposal doc, and have some questions/comments: ... Section 3: [[ As mentioned above, the foremost user interface for this system will be a Haskell program to be executed by the default Haskell Implementation. This Haskell program, Setup.lhs, will perform the tasks of building the package (where necessary), installing the package, and making the package available to the system. ]] I'm guessing this setup program to be part of each library "package" distribution? ... Section 3.3 (comment): Depending on which Haskell implementation has been installed, and how, running setup.lhs directly may not be an option under Windows, especially if it needs to access non-standard support libraries. So there may be some operating-system dependent variations in the command line structure. I don't think this is a serious problem, but I expect one might end up with a command line structure with two parts: [run setup program] [command line to setup program] where the first part may vary between operating systems, or even installations. The important part (the second part) should be commoin across systems. ... Section 3.4: [[ main = defaultMain toolInfo id id -- Those last to parameters might be pre-install and post-install functions ]] s/to/two/? ... Section 4.2: [[ ... but Haskell should offer a more robust set of file operations in order to encourage the use of Haskell for common scripting tasks. ]] I do agree. This an area where I find, say, Python is very strong. The other area that I think needs to be abstracted uniformly across systems is running and interacting with external programs; e.g. as POpen does for Unix systems. ... Section 4.3: [[ To let the Haskell Implementations know how to use a package, whether its available by default (or whether it requires a -package flag), and where the root of its hierarchy is. ]] This comment about the -package flag intrigues me. Is it not the goal that this library infrastructure will eventually displace (what I understand to be) the GHC-specific packaging framework? I guess it may still be needed as a transition measure. ... Section 4.3: [[ All information will be made available through the Distribution.Package module. The information can be made available to non-haskell tools by way of a command-line tool, haskell-config (Section 4.4) with easily parsable output (similar to package-config) though a different solution may be necessary for windows. ]] I'm not sure I fully understand, so this may be irrelavant... A colleague recently told me that the current Windows "badging" requirements are for applications to *not* keep configuration information in the registry, but to use an XML file in the same location as the executable program. Hooray! Maybe we'll start to see the end of the abominable registry monster? Anyway, the idea of keeping and/or presenting configuration information as an easily parsed file format seems to me to be quite appropriate for Windows. ... Section 4.3.1: [[ The package data structure might look something like this (based on GHC's Package class) data PkgIdentifier = PkgIdentifier {pkgName::String, pkgVersion::Version} ... ]] Glancing at this, I wonder if this isn't getting us rather locked into a fixed structure of information? If future requirements should emerge, is there any easy way to accommodate them without breaking existing config data? If the datatype is purely internal and transitory, then I suppose there's no real problem as long as a strategy for extensibility is in place w.r.t. the stored config format. ... Section 5: [[ This may include writing functionality so the compiler can be asked questions about itself. ]] This reminds me that I think it would be good if all Haskell implementations had a (simple) library module that would return key information about the compiler environment (operating system, hardware architecture, install directory, library base directory, file system characteristics, etc.) I have been surprised to find none of this in the standard System module. ... Section 7: I think the first 4 bullets would represent a big win. Beyond that, I think the 6th bullet (external tool/library support) would be the next most useful. ... General. On a quick read through, this looks very good to me. I felt that I didn't fully understand all the intricacies of what is discussed, but that will no doubt become clearer when I start to use and/or create packages that use this framework. I'll try and find time to look at the code again, and see if I can figure out some details from there. (Is there a package example in there?) Which raises an idea I don't remember seeing: does a package configuration include information about installation test modules? I'm thinking that it would be good practice for packages to include an HUnit-like test module that can be used to instill confidence that the installation is working as intended, and that there should be a common way to find it and run it. How do you overcome issues of system dependencies in advance of having a common method across implementations to access key information? Maybe a simple configuration file and an environment variable to point at it could be used in advance of widespread language implementation support? That way, a simple script and/or editing process might be used to bootstrap the system. #g -- At 22:58 17/02/04 -0500, Isaac Jones wrote:
Greetings,
I just uploaded a new release of the Library Infrastructure prototype API for Distribution.Package and the command-line utility "haskell-config". The API for package configuration is included below, FYI.
I'm going to be focusing more on the Build stuff (rather than the packaging stuff) for a little while, but with luck, this can be near to finalization soon.
The web page is here: http://www.haskell.org/libraryInfrastructure/
Feedback and volunteers for coding are always welcome :)
peace,
isaac
ps. I'm using arch (tla) for version control, so if you've been wanting to check it out, now is a good time :)
------------------------------------------------------------
lookupPackage :: PackageDB -> PkgIdentifier -> Maybe PackageConfig addPackage :: PackageDB -> PackageConfig -> PackageDB
delPackage :: PackageDB -> PkgIdentifier -> (PackageDB, Bool) allPackages :: PackageDB -> [PackageConfig] createPackageDB :: FilePath -> (PackageDB -> IO PackageDB) -> IO ()
withPackageDB :: FilePath -> (PackageDB -> IO (PackageDB, a)) -> IO a
userPkgConfigLocation :: String -> FilePath userPkgConfigLocation home = home ++ "/.haskell/packages.conf"
systemPkgConfigLocation :: FilePath systemPkgConfigLocation = "/etc/haskell/packages.conf"
PackageConfig(..), PkgIdentifier(..), License(..), Dependency(..), PackageDB _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries
------------ Graham Klyne For email: http://www.ninebynine.org/#Contact
At 22:58 17/02/04 -0500, Isaac Jones wrote:
Greetings,
I just uploaded a new release of the Library Infrastructure prototype API for Distribution.Package and the command-line utility "haskell-config". The API for package configuration is included below, FYI.
I've downloaded the latest, but I'm having a hard time getting my head around the overall structure. It feels to me a bit like a tangled ball of string, whose free end I cannot find. (This a comment on my understanding, not your code/design!) Looking at the LIP from the point of view of installing some LIP package in a target environment, it seems to me the following details need to be addressed, somehow: (1) Library files are copied to some place on the target filesystem. Is it intended that they can be installed anywhere, or does the LIP software choose a location? How is that location determined? (2) It may be necessary that the library files need to be compiled on the target system. I think LIP must know about the compilation and related tools available for this purposes. How does LIP learn about these, and where is the information kept? (3) When a package is installed, its presence must be made known in some way, so that other programs that use the facility can find the appropriate files. Where is such information kept, and how is it used when building a program that uses a library installed using LIP? (4) There are many more possible questions about things like dependencies between installed packages, system requirements, compiler requirements, and other secondary configuration information. I think these issues would be easy to track down once the description basic relationship between compiler/library/programmer is exposed. Generally, then, LIP must know things about the installed Haskell compilers, libraries, host system type, other tools, etc. Where is this information kept, or how does LIP decide where it is kept? I'm fishing here for clues to help me locate the free end of the string, so I can start to unravel it. ... Maybe a clue I'm looking for is this: [[ systemPkgConfigLocation :: FilePath systemPkgConfigLocation = "/etc/haskell/packages.conf" ]] But I don't see how the base config location can be hardwired into the code and be portable across systems. Do you envisage a fixed location for each kind of system? For Windows, it's common to keep additional data and /etc/ stuff in the same directory as the executable program; IIRC the Windows API provides means to locate that directory. Also, under Windows, its possible to define environment variables that are defined for all users; I'm not sure if Unix/Linux provides an equivalent. ... Also, looking at: [[ userPkgConfigLocation :: String -> FilePath userPkgConfigLocation home = home ++ "/.haskell/packages.conf" ]] I note that the Unix convention of using a name starting with '.' for "hidden" config files doesn't really work for Windows. ... Looking at the API outline, and all those functions that operate on a PackageDB value, a question that occurs to me is: where does the packageDB come from? My best guess is that it's all bound up with: [[ withPackageDB :: FilePath -> (PackageDB -> IO (PackageDB, a)) -> IO a ]] which creates a PackageDB value, threads it through the I/O monad created by the second argument, puts the updated DB back to bed and returns some value derived from the process. Am I close here? ... I just spotted this as a possible way that LIP locates its config file ... In the file Main.hs, I see: [[ home <- getEnv "HOME" let getPathH = getPath home ]] which value is used later in contexts that seem to name the config file. This does seem to be rather Unix-centric (isn't HOME a Unix/Linux environment variable for the user's login directory?). And in any case, is this really something that ought to be configured on a per-user basis? My thought is either: (1) to apply a ladder something like this: (a) look for a specific environment variable, say 'HaskellLibConfig', which can be configured to point to the config file (b) look for environment variable HOME (c) look for environment variables HOMEDRIVE and HOMEPATH (Windows equivalent of HOME) and use a fixed default filename ... this will work as a per-user default if the explicit environment variable is not set. (d) [try some system-dependent convention for MACs and other computer systems] or (2), when installing the LIP software on a machine, select a system-dependent module that performs an appropriate series of tests, so that we might have: Unix: (a) Env:HaskellLibConfig, (b) Env:HOME, (c) /etc/HaskellLib.conf Windows: (a) Env:HaskellLibConfig, (b) Env:HOMEDRIVE+HOMEPATH, (c) <LibSoftware>\HaskellLib.conf (where <libsoftware> is the executable program directory reported by Windows Other: (etc...) Generally, my feeling is that there should be a fairly easy means to specify exactly where the config file is kept, with a fallback to a reasonable default if that means is not employed. ... I hope my fiddling with this is more help than hindrance ... I think the project is an important undertaking for the Haskell world. (Time to go off and learn about hmake, I think.) #g ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact
Graham Klyne
I've downloaded the latest, but I'm having a hard time getting my head around the overall structure. It feels to me a bit like a tangled ball of string, whose free end I cannot find. (This a comment on my understanding, not your code/design!)
:)
(1) Library files are copied to some place on the target filesystem. Is it intended that they can be installed anywhere, or does the LIP software choose a location? How is that location determined?
For now, this isn't addressed. This is going to be in the middle layer, the installation (Distribution.Install) layer, which is not yet implemented or specified. The only thing implemented right now is the packaging component, which is an API for helping the Implementations to locate and use packages, as well as a command-line tool for registering packages, Distribution.Package.
(2) It may be necessary that the library files need to be compiled on the target system. I think LIP must know about the compilation and related tools available for this purposes. How does LIP learn about these, and where is the information kept?
This information must be made available to the Distribution.Build package. I had originally intended it to be made available through the Distribution.Package interface, where the compilers, etc are just more packages. I'm not so convinced anymore that this is the way to go, but the idea is similar. The compilers and preprocessors, etc, might register themselves in a central location. I hope that Distribution.Build (which I'm sorta working on now, and which may end up being based on hmake) will provide an API which glosses over some of the differences between the compilers, at least for simple tools. Right now, what hmake does is close to what we need. Each installed compiler is registered with hmake somehow, and it keeps track of what features they have.
(3) When a package is installed, its presence must be made known in some way, so that other programs that use the facility can find the appropriate files. Where is such information kept, and how is it used when building a program that uses a library installed using LIP?
This is the part that is solved by Distribution.Package. The information is made available through that API.
Generally, then, LIP must know things about the installed Haskell compilers, libraries, host system type, other tools, etc. Where is this information kept, or how does LIP decide where it is kept?
It should all be available, I hope, through Haskell APIs like Distribution.Package. How will LIP _decide_ where to keep it should be answered below.
Maybe a clue I'm looking for is this: [[ systemPkgConfigLocation :: FilePath systemPkgConfigLocation = "/etc/haskell/packages.conf" ]]
But I don't see how the base config location can be hardwired into the code and be portable across systems.
It is not intended to be. Right above that you should have seen this comment: -- |Hopefully this can be grabbed from a more intelligent Directory -- module. This refers to a discussion on this list some months back: http://haskell.org/pipermail/libraries/2003-October/001462.html Well, lots of things are talked about in that thread, but the basic idea is that I think we'd win a lot by coming up with some way to populate something like those configuration file locations, but that this is a specific case of a more general problem of getting system configuration information like path separators, os versions, etc. Several solutions were discussed, but no one has raised their hand to implement one of them :) I'm hoping that this will magically happen while I'm working on the other stuff.
Also, looking at: [[ userPkgConfigLocation :: String -> FilePath userPkgConfigLocation home = home ++ "/.haskell/packages.conf" ]] I note that the Unix convention of using a name starting with '.' for "hidden" config files doesn't really work for Windows.
Yeah. With a better system, I would expect to build that path like this: userPkgConfigLocation = Directory.userConfigDirectory ++ Platform.pathSeparator ++ packages.conf Well, that's not perfect, but you get the idea. The over-arching point is that if the standard APIs provided more _infrastructure_ type stuff, like Distribution.Package and Platform.pathSeparator, then most libraries could be taken out of the ghc/nhc/hugs distributions and be more effectively distributed on their own, or in nice collections.
Looking at the API outline, and all those functions that operate on a PackageDB value, a question that occurs to me is: where does the packageDB come from? My best guess is that it's all bound up with: [[ withPackageDB :: FilePath -> (PackageDB -> IO (PackageDB, a)) -> IO a ]] which creates a PackageDB value, threads it through the I/O monad created by the second argument, puts the updated DB back to bed and returns some value derived from the process. Am I close here?
Yep. withPackageDB is the only way to get to the database contents. Look at haskell-config/Main.hs for how it's used.
I just spotted this as a possible way that LIP locates its config file ...
In the file Main.hs, I see: [[ home <- getEnv "HOME" let getPathH = getPath home ]] which value is used later in contexts that seem to name the config file.
This does seem to be rather Unix-centric (isn't HOME a Unix/Linux environment variable for the user's login directory?). And in any case, is this really something that ought to be configured on a per-user basis?
Yep. It's unixy. See above. If you have any #defines I can add to the code for windows, please pass them along. The somewhat good news is that the config file locations are only used for the --user and --system flags in haskell-config. You can also use the --config flag to pass the location of the package configuration. (snip some good ideas)
I hope my fiddling with this is more help than hindrance ... I think the project is an important undertaking for the Haskell world.
Always, and thanks :) Let me know if I answered your questions well enough. They are all on-point and relevant. peace, isaac
participants (2)
-
Graham Klyne -
Isaac Jones