January 2014 - cabal-devel

Cabal hack night suggestions
by David Laing 09 Jan '14

09 Jan '14

Hi all, There are a few people in my local FP meetup group looking into doing some semi-regular Haskell hack nights, and we're hoping to target various tools and libraries in the Haskell ecosystem so that we can give back a little while having fun and honing our skills. Cabal is pretty high on our list of things to hack on, and we're hoping to start mid next week. I'm sure we'll be able to click through github issues and submit pull requests on our own, but I thought I'd ask if anyone has any thoughts on areas that would be good to look at that might sit in a sweet spot of being both beneficial to Cabal and accessible to newcomers to the code. Does anyone have any thoughts? Cheers, Dave

2 3

Broken documentation on Hackage.
by Mateusz Kowalczyk 06 Jan '14

06 Jan '14

Hi all, It seems that we are having a rather big issue with Hackage in recent months and I'm sure many of you have noticed: a lot of packages aren't getting their docs built. As far as I can tell, there can be multiple reasonable causes: * Dependencies fail to build so your package does * Your package fails to build directly * Your package requires non-cabal libraries which aren't installed * Your package requires different version of install libraries While all of these are understandable, there also seem to be problems with some packages which are seemingly perfectly fine otherwise. This problem is not new and has been reported[1]. There is even a system in place with Hackage 2 that would grant package owners to manually upload documentation and a system where some people (trustees and package maintainers) have the ability to do things like deleting the broken docs to have the builder try again[3] but it seems that this isn't actually used[4]. Over night I hacked up a quick program to parse some Hackage parts to see just how much stuff was broken. I have only considered the most recent package versions. Out of the 7761 packages, 811 came up with missing documentations. While it is a big number (~9.56%), it includes every package on Hackage. Next thing that follows is to restrict the search a bit. Only considering packages from 2013 and the still very young 2014, 210 are missing documentation. 140 of those were uploaded since August 2013 (around the Hackage 2 move). Remember that this is only for most recent versions on Hackage. Assuming that we didn't have a large spike in package uploads in second half of 2013 (I don't know, are there any charts or something?), that's unreasonably more than in the first half of 2013. It should be fairly clear that something is broken and I'd love to know what. I snooped around in #hackage today for a bit and there doesn't seem to be much sense of the urgency. Granted, people are busy but isn't this a pretty important issue? It's not like it's recent either. What can we do? Why isn't it fixed? Are there any suspects? Why isn't the trustee system being used to mitigate this problem a bit while it's being fixed? Is there any way to improve the amount of things that can be built by trying to work out any of the points at the top of this e-mail? Honestly, scrolling through the build logs of packages, all failures seem reasonable (Haddock parse failure, no dependencies installed…) and yet somehow I find myself having to click on older version of packages to get docs very often recently. What changed? Is there even anything to fix? Maybe I'm just getting unlucky with the packages I click on recently. I have uploaded the list of packages with missing documentation that have been uploaded on Hackage in 2013 and 2014 to [5]. The format is: (packageUrl, MissingDocs (Maybe reason) packageVersion dateUploaded). If your package is on there, you might want to consider uploading the docs by hand for now! Packages with ‘Nothing’ for a reason have no build log. See [6] for a reason for those. See [7] for how to view build logs yourself. PS: I'm unsure if these are the right lists to e-mail. I've been told that Cabal folk maintain Hackage issues and considering so many broken docs, libraries might also be interested in it. Feel free to point me elsewhere. PPS: There also seems to be a lot of 404 errors while trying to click on some docs on some older packages that probably existed in the past. Perhaps they were lost in the Hackage move or are a different issue all together. [1]: https://github.com/haskell/hackage-server/issues/145 [2]: https://github.com/haskell/hackage-server/issues/145#issuecomment-29468613 [3]: https://github.com/haskell/hackage-server/issues/145#issuecomment-29422332 [4]: http://hackage.haskell.org/packages/trustees/ [5]: http://fuuzetsu.co.uk/misc/sorted.txt [6]: https://github.com/haskell/hackage-server/issues/145#issuecomment-30129142 [7]: https://github.com/haskell/hackage-server/issues/145#issuecomment-29472445 -- Mateusz K.

7 13

Re: Broken documentation on Hackage.
by Malcolm Wallace 06 Jan '14

06 Jan '14

On 6 Jan 2014, at 12:32, Henning Thielemann wrote: > Am 06.01.2014 12:59, schrieb Mateusz Kowalczyk: >> On 06/01/14 10:27, Malcolm Wallace wrote: >>> >>> Haddock also was once stand-alone. I think it might be time to wind the clock backwards and retrieve this desirable property. >> >> Was Haddock ever stand alone? AFAIK it used to be part of GHC and then >> David Waern separated it into a separate package. > > The versions with leading zero by Simon Marlow used a custom parser. Yes, early versions of Haddock assumed that the developer would always write a type signature for any top-level value that is exported. No need to rely on a compiler to infer a type. Although inference is hugely useful, I think the discipline of actually writing down what the compiler infers for you is an absolutely essential software engineering practice. It is one good step towards future maintainability of the library. At my workplace, for instance, our git repo infrastructure is configured to reject any code that exports a value without an explicit type signature. Regards, Malcolm

1 0

Re: Broken documentation on Hackage.
by Mateusz Kowalczyk 06 Jan '14

06 Jan '14

Oops, seems I sent to libraries. Let's keep the thread on cabal-devel only as here's where the replies happened. On 06/01/14 10:27, Malcolm Wallace wrote: > > On 5 Jan 2014, at 10:15, Mateusz Kowalczyk wrote: > >> It seems that we are having a rather big issue with Hackage in recent >> months and I'm sure many of you have noticed: a lot of packages aren't >> getting their docs built. As far as I can tell, there can be multiple >> reasonable causes: >> >> * Dependencies fail to build so your package does >> * Your package fails to build directly >> * Your package requires non-cabal libraries which aren't installed >> * Your package requires different version of install libraries > > I think the fundamental problem is that Haddock is now built on top of ghc. So if a package cannot be built by ghc (for whatever reason, e.g. missing C library dependency), then it cannot be documented either. This is a good deal less than useful. A documentation generator ought to do a reasonable job, even if the code it is looking at is technically not-compilable. It is because it uses GHC to type check the modules and generate the signatures. I agree that there should be some kind of fall back but it would be a great deal less useful of an output. > At work, we have a stand-alone documentation generator for Haskell, which requires no compiler. Haddock also was once stand-alone. I think it might be time to wind the clock backwards and retrieve this desirable property. Was Haddock ever stand alone? AFAIK it used to be part of GHC and then David Waern separated it into a separate package. I honestly can't think of how it would ever have been stand-alone (that is not relying on GHC to do part of the work). > > Regards, -- Mateusz K.

1 0

Re: Broken documentation on Hackage.
by Mateusz Kowalczyk 05 Jan '14

05 Jan '14

On 05/01/14 14:14, Sven Panne wrote: > My ALUT package is among the ones without documentation, and I have a > theory what's wrong (probably a missing installed C library/package > libalut-dev, see > https://github.com/haskell-openal/ALUT/blob/master/.travis.yml) Two > questions: > > * As a package maintainer, how can I find out why the documentation > is not built/published on Hackage? Logs, etc.? I mention a link to a GitHub post in my opening post which outlines how to access the log. In case of ALUT, the build log is at [1] http://hackage.haskell.org/package/ALUT-2.3.0.1/reports/1/log and indeed a C library is missing. > * Who should be contacted to polish the server a bit (i.e. install > libalut-dev)? No idea, but I'd love to find out myself! I'm just a disgruntled Hackage user myself. > Currently things are very opaque from the outside, both from a > technical and from an organizational point of view, and manually > uploading documentation which is effectively guaranteed to be > wrong/outdated/etc. sooner or later is not a solution IMHO. You would be uploading documentation for a specific package version. So for example, if you upload something for ALUT 2.3.0.1 and later upload upload ALUT 2.4, either the docs will be built fine (which would be ideal) or you will have to upload them yourself again for that version. I agree that this is subpar but I think it's better than nothing. There should really be a ‘upload documentation’ button on the site or maybe even a Cabal flag or maybe even ability to bundle documentation in the package tarball when uploading it, to act as a fallback if package fails to build. I suppose this is something that has to be discussed. Ideally, it should just work. > > Cheers, > S. > [1]: http://hackage.haskell.org/package/ALUT-2.3.0.1/reports/1/log -- Mateusz K.

1 0

Warning about missing modules in 'other-modules'
by Daniel Trstenjak 05 Jan '14

05 Jan '14

Hi, I now at least twice ran into the issue, that I forgot to add a module to the 'other-modules' section of the cabal file and uploaded a non buildable package to hackage. It would be nice if 'cabal sdist' or 'cabal upload' would warn the user about missing modules. I just tried 'cabal upload -c', but I'm getting: $ cabal upload -c dist/hsimport-0.2.6.2.tar.gz Checking dist/hsimport-0.2.6.2.tar.gz... Error: dist/hsimport-0.2.6.2.tar.gz: 404 Not Found Error: Page not found Sorry, it's just not here. $ cabal --version cabal-install version 1.18.0.2 using version 1.18.1 of the Cabal library It seems that currently the best way to guarantee a buildable package is to build the tarbar with 'cabal sdist' and then install this tarball with 'cabal install', right? Greetings, Daniel

2 1

hackage helpers!!
by Carter Schonwald 04 Jan '14

04 Jan '14

hey all, isn't there supposed to be an ACLs level for hackage helpers? (ie folks who can force docbuilders to rebuild docs for packager, edit pkg info stuff, etc?) Am I correct in thinking no ones in that group yet? (i know duncan asked for volunteers, but I don't think anyone's been put in that group yet!) cheers! -Carter

1 0