March 2013 - cabal-devel

darcs patch: Add a new checkout Command (and 1 more)
by Joachim Breitner 20 Jul '13

20 Jul '13

Hi, the small feature implemented by the attached patches could make it more convenient for people to quickly and easily contribute to hackage packages. Do you want me to open a ticket for this or can you appl yit right away? Thanks, Joachim 2 patches for repository http://darcs.haskell.org/cabal-install: Sun Nov 7 12:35:32 CET 2010 Joachim Breitner <mail(a)joachim-breitner.de> * Add a new checkout Command Given a package name, this uses the SourceRepo field of the newest versionf of that pacakge to figure out the repository location and calls the right VCS invocation. Sun Nov 7 12:43:35 CET 2010 Joachim Breitner <mail(a)joachim-breitner.de> * Add a --dry-run flag to cabal checkout

2 3

hackage 2: alpha testing, testers wanted!
by Duncan Coutts 15 Apr '13

15 Apr '13

Hi folks, I believe we're ready for alpha testing for hackage 2. The server is up at: http://new-hackage.haskell.org/ Key points: * All the old data is imported (including docs) * Uploading is enabled * Live mirroring is also enabled * Per-package maintainer groups are pre-populated * Existing users should be able to log in (after a one-time upgrade) I would like to invite everyone on this list to help out with the alpha testing and to report issues here on this list. Patches are even better. The server is up now so you'll get "instant patch gratification"tm if you chip in and fix things. In addition to any bugs you may discover, the server main page gives an overview of the features that are partially complete and could do with some love. When we're satisfied we'll move into beta testing and advertise more widely within the community (reddit etc). The doc builder is not set up yet, but the builder client code exists, just need to get that set up. In general the switch over plan remains that we run the two servers in parallel for long enough to sort out remaining issues (with help from the kind folks who volunteered as admins). Code and wiki ============= The code is all still in darcs. darcs get http://code.haskell.org/hackage-server/ The plan is to move it to github at some stage. The wiki content is still useful though starting to get a bit out of sync: http://hackage.haskell.org/trac/hackage/wiki/HackageDB/2.0 We had to turn off wiki editing for most users because of spammers. If you'd like to work on the wiki, let me know and I'll give your account permission to edit it. Note also that I think Ben Millwood is planning to move the content to github at some stage, though we can still make improvements to the content in the meantime. Hackage Admins ============== I believe the following people volunteered to be admins, at least for the alpha/beta testing period: * Erik Hesselink * Ben Millwood * Johan Tibell If you folks (or anyone else who wants to volunteer) would like to confirm that then I'll add you to the admins group. User accounts ============= Existing users should be fine. That process should now be fairly smooth (testing welcome!). For new users, they can sign up with email confirmation. An administrator is needed to add a user into the uploaders group. Currently there's not a lot you can do with an account if you're not in the uploaders group. That may change in future if we implement "social" features. Hackage Trustees ================ In addition to admins we have the "trustees" group. This should eventually be a rather useful job. The idea is that the trustees don't upload packages but look after the health of the package collection as a whole (or some subset they're interested in), a bit like distro maintainers. Currently they can only edit package tags and upload docs (e.g. if the doc builder fails) but the plan is for them (as well as maintainers) to be able to edit package metadata, in particular the constraints on dependencies. Once we get there, that'll be a rather important job. So I'm also looking for volunteers there, both to be trustees and to help finish implementing the metadata-editing feature. -- Duncan Coutts, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

3 4

hackage trac
by Ian Lynagh 05 Apr '13

05 Apr '13

Hi all, A few IPs were hammering http://hackage.haskell.org/trac/hackage/wiki/HackageDB today which (is probably what) caused abbot to become inaccessible. /srv/trac/hackage/db/trac.db is 613MB, which seems rather suspicious: /srv/trac/ghc/db/trac.db is only 359MB. I wonder if it is full of spam? I'm also not able to load http://hackage.haskell.org/trac/hackage/wiki/HackageDB?action=history (it just times out). Could someone take a look please? Or perhaps the Cabal wiki should be moved elsewhere (github?) and the trac instance disabled? Thanks Ian

6 11

patch applied (hackage-server): "Add link to cabal file editing to maintainers page"
by devnull＠community.haskell.org 27 Mar '13

27 Mar '13

Wed Mar 27 12:50:51 GMT 2013 Duncan Coutts <duncan(a)well-typed.com> * Add link to cabal file editing to maintainers page Ignore-this: 99bf7ccf84a4cfeab89406c672fa1d7f M ./datafiles/templates/Html/maintain.html.st +5

1 0

patch applied (hackage-server): "Remove the google search from the package page" and 5 others
by devnull＠community.haskell.org 26 Mar '13

26 Mar '13

Thu Mar 14 17:37:47 GMT 2013 Duncan Coutts <duncan(a)well-typed.com> * Remove the google search from the package page Ignore-this: 7c0346431aec1bc7f195cdd2037991c6 M ./Distribution/Server/Pages/Index.hs -8 Tue Mar 26 14:43:11 GMT 2013 Duncan Coutts <duncan(a)well-typed.com> * Tweak page css based on feedback Ignore-this: 6b91314fa8fc506a02b42d30ac390161 Center content, bigger font, wider page body, no justification of text M ./datafiles/static/hackage.css -3 +2 Tue Mar 26 14:44:38 GMT 2013 Duncan Coutts <duncan(a)well-typed.com> * Fix last few occurrences of HackageDB for simply Hackage Ignore-this: b4bf3d65b92523f8d3d5a4cc2a1392f5 M ./datafiles/templates/UserSignupReset/ResetConfirm.st -1 +1 M ./datafiles/templates/UserSignupReset/ResetEmailSent.st -1 +1 M ./datafiles/templates/UserSignupReset/ResetRequest.st -1 +1 M ./datafiles/templates/UserSignupReset/SignupConfirm.st -1 +1 M ./datafiles/templates/UserSignupReset/SignupEmailSent.st -1 +1 M ./datafiles/templates/accounts.html.st -1 +1 Tue Mar 26 18:24:00 GMT 2013 Duncan Coutts <duncan(a)well-typed.com> * Redo the maintainer links and page based on feedback Ignore-this: 2805cd1d99d8f91721ca4671a3241ead The feedback was that the links for maintainers were confusing for ordinary users because it is not clear if they are allowed to use them or not, and "maintain" is not very descriptive as a link name. This patch removes the tags edit link from the package page. It moves the maintainer edit link to the bottom of the page and make it clearer that it is only for maintainers and trustees. The new maintainer page is now a lot clearer (and uses a template so easier to edit). M ./Distribution/Server/Features/Html.hs -29 +34 M ./Distribution/Server/Pages/Package.hs +11 A ./datafiles/templates/Html/ A ./datafiles/templates/Html/maintain.html.st Tue Mar 26 23:34:47 GMT 2013 Duncan Coutts <duncan(a)well-typed.com> * Add feature for editing cabal files Ignore-this: 2d62a3ae7d5be1cf0b26045cdb623584 Maintainers and truestees can make new revisions without uploading a new tarball. Only certain changes are allowed. In particular adjusting dependency version constraints is allowed. M ./Distribution/Server/Features.hs +7 A ./Distribution/Server/Features/EditCabalFiles.hs M ./Distribution/Server/Framework/Resource.hs -2 +14 A ./datafiles/templates/EditCabalFile/ A ./datafiles/templates/EditCabalFile/cabalFileEditPage.html.st A ./datafiles/templates/EditCabalFile/cabalFilePublished.html.st M ./hackage-server.cabal +1 Tue Mar 26 23:37:15 GMT 2013 Duncan Coutts <duncan(a)well-typed.com> * Update the favicon to the haskell.org one Ignore-this: 349208e2a7bc61452918590bd87e1b9e M ./datafiles/static/favicon.ico

1 0

Code style guide?
by Ben Millwood 23 Mar '13

23 Mar '13

Hi everyone, Do we have a general notion of what we consider best practices for contributed code to Cabal? Perhaps we should formalise something? One of the reasons I wanted to get the wiki back online was in the hope that it had some pointers, but it doesn't seem to. (I've started migrating stuff anyway: have a look at https://github.com/haskell/cabal/wiki/Migration-from-trac for status). For example, many of the Cabal modules have explicit, and therefore quite long, import lists, with no clear organisation, and hence when adding new imports I have to read the entire list to make sure they are not already there, and then wonder where makes sense to insert them. Usually I just make something up with no pattern to it, and hence perpetuate the problem. I am happy to start things off with a proposal if no-one else wants to do the writing. Regards, Ben Millwood (benmachine)

6 12

Re: Advance notice that I'd like to make Cabal depend on parsec
by Duncan Coutts 22 Mar '13

22 Mar '13

On Mon, 2013-03-18 at 09:32 -0700, Iavor Diatchki wrote: > Hello, > > To me it seems that the dependency here is incorrect---as far as I > understand, GHC does not need to parse Cabal files, so it should not depend > on the code and the library to do so. Yes, GHC does not parse .cabal files. > Furthermore, what is the overall architecture of the whole thing? My > understanding has been that each implementation should have its own notion > of a "package", and cabal simply has support for working with the package > formats for each implementation. Thus, it seems that the package types and > code for (de)serializing them should be in the implementation (i.e., GHC), > not Cabal. I can see that it might make sense to have a common > representation about package meta-data (e.g., names, versions, license, > etc.), so perhaps these should all go in a separate package. This looks a > bit like the `cabal-types` that Duncan suggested, but I'd imagine that > Cabal would need more types than just package meta-data so this is not an > ideal name. The Cabal spec defines a few things that all Haskell implementations are supposed to support/provide. This covers the notion of an installed package (not .cabal source packages). It defines what it is, the meta-data that is stored and the format in which the implementation should accept it (ie the input format for ghc-pkg). So it's a compiler independent notion and the natural place to put the code to support it was the Cabal lib, so that's what happened. We can split that off into another package but it still makes sense for that package to provide a parser and pretty printer (because implementations have to accept them in the external format). So the natural way to partition things doesn't help us avoid ghc depending on a parser lib. > Finally, I agree that a "real" parser is good, but do you really want to > write it using Parsec? A sensible alternative would be to write a Happy > grammar. Having an actual grammar would both benefit users of the system, > and it would avoid the dependency on all those package. My experience of > having to maintain some largish Parsec (and in general, combinator based) > parsers, is that over the years the parsers get more and more complex, and > are quite hard to maintain. I'm somewhat repeating other parts of the thread at this point (see the discussion on "why not happy"), but I'd like to point out again that it is not simply the outline parser for cabal-style files that we're talking about. We also need parsers/pretty printers for all the various little types that make up the info about packages, like versions, package names, package ids, version constraints, module names, licenses etc etc. In the Cabal lib we have a type class with parser and pretty printer and all these various little types are instances. That stuff pretty much has to use a combinator lib because it needs to be compositional. You need to be able to reuse the version number parser in other parsers, or programs like cabal-install or the hackage-server need to reuse them as part of other parsers in their command line UI or config files, urls etc. So we have to use a combinator lib anyway. We currently also use this for parsing the fields of .cabal files and the InstalledPackageInfo. Currently we use ReadP from the base library which produces no error messages and has pretty bad performance in some cases (exploding memory use). It's that part particularly that should be using something like parsec. Happy simply isn't an option there. I could possibly use happy for the outline parser but it wouldn't buy us much. Duncan

2 2

RE: Advance notice that I'd like to make Cabal depend on parsec
by Simon Peyton-Jones 21 Mar '13

21 Mar '13

Is it essential, or even sensical, that the serialization format GHC needs for storing package info bear any relation to the human authored form? If not, the split out of the package types could be accomplished in a way where GHC uses simple show/read(P) style serialization for storage of package info, where as cabal-lib would use a lovely parsec parser for humans. I'd like this approach. Good idea -- esp if it makes the packaging story simpler. GHC already uses a binary format for interface files, so there’s no good reason to use a human-readable format for package data base stuff. For interface files you can read them with ghc --show-iface, and as Ian remarks something similar is already true for the package data base. Simon From: ghc-devs-bounces(a)haskell.org [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Mark Lentczner Sent: 17 March 2013 16:57 To: dag.odenhall(a)gmail.com Cc: Haskell Libraries; cabal-devel; Duncan Coutts; ghc-devs(a)haskell.org; Antoine Latter Subject: Re: Advance notice that I'd like to make Cabal depend on parsec This thread is raising all sorts of questions for me: Is it essential, or even sensical, that the serialization format GHC needs for storing package info bear any relation to the human authored form? If not, the split out of the package types could be accomplished in a way where GHC uses simple show/read(P) style serialization for storage of package info, where as cabal-lib would use a lovely parsec parser for humans. I'd like this approach. The issue of putting the yet one more HP package into GHC's core packages is increasing the exposure of the difficulty of the current GHC/HP relationship. See also threads in HP's mailing list for why can't we bump some packages in GHC's core set for the next HP release. The split arrangement is strange because we have two groups making up what is in the HP, but they have different processes and aims. The complex technical relationship between the moving parts only heightens the difficulty. Perhaps the major cause is that because GHC is shipped as a library itself, it exposes all it's package dependencies. And as it is a large, and growing, piece of software, the list only wants to grow. But I wonder how often GHC is used as a library itself? If not often, then perhaps GHC should be shipped as two parts: Just a compiler (plus the small number of packages that the compiler forces), and ghc-lib as an optional, even separate, package - perhaps one with even a traditional way of depending on other packages. In otherwords, users that wanted to incorporate the ghc-lib into their programs would depend, and download, and configure, and build, ghc-lib indpenendant of the GHC binaries installed on their system. Perhaps then, GHC, the compiler, built from ghc-lib, would be bootstrapped not from the past compiler, but from the past HP..... Okay, perhaps that is all just fantasy. But, no other programming system operates the way we do. They all fall into one of two camps: * The dominant implementation is maintained, built, and shipped along with a large collection of "common packages". Examples: Python, Ruby, PHP, Java. * The dominant implementation is shipped as a bare tool, and large common libraries are maintained and shipped independently. Examples: C++ (think g++ and boost), JavaScript (think browsers, and jQuery). We are in the middle and, I think, experiencing growing pains because of it. - Mark On Sat, Mar 16, 2013 at 3:42 PM, dag.odenhall(a)gmail.com<mailto:dag.odenhall@gmail.com> <dag.odenhall(a)gmail.com<mailto:dag.odenhall@gmail.com>> wrote: I'd love to have a proper parser and source-location-aware AST for sake of editor/IDE tools, so +1 from me. If you don't end up doing this after all, I'd still like to see your parser in a separate package, although I understand if you don't feel like maintaining two parsers especially given the tedious process for verifying they work similarly. I guess it could still be useful in the same way we find haskell-src-exts useful despite some incompatibilities with GHC. On Thu, Mar 14, 2013 at 3:53 PM, Duncan Coutts <duncan.coutts(a)googlemail.com<mailto:duncan.coutts@googlemail.com>> wrote: Hi folks, I want to give you advance notice that I would like to make Cabal depend on parsec. The implication is that GHC would therefore depend on parsec and thus it would become a core package, rather than just a HP package. So this would affect both GHC and the HP, though I hope not too much. The rationale is that Cabal needs to parse things, like .cabal files and currently we do not have a decent parser in the core libraries. By decent I mean one that can produce error messages with source locations and that doesn't have unpredictable memory use. The only parser in the core libraries at the moment is Text.ParserCombinators.ReadP from the base package and that fails my "decent" criteria on both counts. Its idea of an error message is (), and on some largish .cabal files we take 100s of MB to parse (I realise that the ReadP in the base package is a cutdown version so I don't mean to malign all ReadP-style libs out there). Partly due to the performance problem, the terrible .cabal file error messages, and partly because Doaitse Swierstra keeps asking me if .cabal files have a grammar, I've been writing a new .cabal parser. It uses an alex lexer and a parsec parser. It's fast and the error messages are pretty good. I have reverse engineered a grammar that closely matches the existing parser and .cabal files in the wild, though I'm not sure Doaitse will be satisfied with the approach I've taken to handling layout. Why did I choose parsec? Practicality dictates that I can only use things in the core libraries, and the nearest thing we have to that is the parser lib that is in the HP. I tried to use happy but I could not construct a grammar/lexer combo to handle the layout (also, happy is not exactly known for its great error messages). I've been doing regression testing against hackage and I'm satisfied that the new parser matches close enough. I've uncovered all kinds of horrors with .cabal files in the wild relying on quirks of the old parser. I've made adjustments for most of them but I will be breaking a half dozen old packages (most of those don't actually build correctly because though their syntax errors are not picked up by the parser, they do cause failure eventually). So far I've just done the outline parser, not the individual field parsers. I'll be doing those next and then integrate. So this change is still a bit of a ways off, but I thought it'd be useful to warn people now. Duncan _______________________________________________ cabal-devel mailing list cabal-devel(a)haskell.org<mailto:cabal-devel@haskell.org> http://www.haskell.org/mailman/listinfo/cabal-devel

2 1

Re: Advance notice that I'd like to make Cabal depend on parsec
by Duncan Coutts 18 Mar '13

18 Mar '13

On Sun, 2013-03-17 at 09:57 -0700, Mark Lentczner wrote: > This thread is raising all sorts of questions for me: [..] > The issue of putting the yet one more HP package into GHC's core packages > is increasing the exposure of the difficulty of the current GHC/HP > relationship. See also threads in HP's mailing list for why can't we bump > some packages in GHC's core set for the next HP release. The split > arrangement is strange because we have two groups making up what is in the > HP, but they have different processes and aims. The complex technical > relationship between the moving parts only heightens the difficulty. This is certainly worth thinking about. Perhaps parsec is the straw that broke the camel's back. It's not qualitatively different from the other core libs: for all of them we have the issue about version pinning and the effects on the release cycle. Duncan

1 0

Re: Advance notice that I'd like to make Cabal depend on parsec
by Ian Lynagh 18 Mar '13

18 Mar '13

On Sun, Mar 17, 2013 at 09:57:25AM -0700, Mark Lentczner wrote: > > Is it essential, or even sensical, that the serialization format GHC needs > for storing package info bear any relation to the human authored form? If > not, the split out of the package types could be accomplished in a way > where GHC uses simple show/read(P) style serialization for storage of > package info, where as cabal-lib would use a lovely parsec parser for > humans. I'd like this approach. I think it would be feasible to stop GHC itself from using the human readable format. The only place I can think of it being used is in the package database, but we could use either Read/Show for that, or just exclusively use the binary format. It would be a little less user-friendly, but maybe worth it to remove the ghc library dependencies on most-of-Cabal, mtl and parsec. > Perhaps the major cause is that because GHC is shipped as a library itself, > it exposes all it's package dependencies. Yes. > In otherwords, users that wanted to > incorporate the ghc-lib into their programs would depend, and download, and > configure, and build, ghc-lib indpenendant of the GHC binaries I think this would create more problems than it solves. > Okay, perhaps that is all just fantasy. But, no other programming system > operates the way we do. They all fall into one of two camps: > > - The dominant implementation is maintained, built, and shipped along > with a large collection of "common packages". Examples: Python, Ruby, PHP, > Java. > - The dominant implementation is shipped as a bare tool, and large > common libraries are maintained and shipped independently. Examples: C++ > (think g++ and boost), JavaScript (think browsers, and jQuery). > > We are in the middle and, I think, experiencing growing pains because of it. I would say that we are doing the first option, in the form of the HP. It's just that the core gets frozen (i.e., ghc + libs gets released) earlier than the higher level libraries. I don't think that moving (back) to trying to freeze/release everything all at once would be an improvement. You just need to remain strong, and keep saying "no" :-) (you're doing a great job, BTW!) Thanks Ian

3 7