is it normal for docs to be pending for 24 hours?

This is only my second time uploading a package to Hackage, so I don't yet have a feel for how it's supposed to go. I uploaded normalization-insensitive-2.0.0.1 about 24 hours ago: https://hackage.haskell.org/package/normalization-insensitive-2.0.0.1 Under "Status", it says "Docs pending". (And the module names are all non-clickable.) Is it normal to take this long to build the docs? Is there some way to find out where in the queue my job is? Is this an indication that something has gone wrong? How do I fix it? Thanks, --Patrick

On Tue, Nov 08, 2016 at 05:37:07PM -0800, Patrick Pelletier wrote:
This is only my second time uploading a package to Hackage, so I don't yet have a feel for how it's supposed to go. I uploaded normalization-insensitive-2.0.0.1 about 24 hours ago:
https://hackage.haskell.org/package/normalization-insensitive-2.0.0.1
Under "Status", it says "Docs pending". [..]
Sometimes hackage fails to build documentation (maybe there is a hint on why on the logs). In those cases, I take advantage of cabal: cabal upload --documentation (more info with `cabal upload --help`).

On Tue, Nov 08, 2016 at 07:08:31PM -0800, Patrick Pelletier wrote:
On 11/8/16 5:59 PM, Francesco Ariis wrote:
Sometimes hackage fails to build documentation (maybe there is a hint on why on the logs).
Where can I find those logs?
https://hackage.haskell.org/package/normalization-insensitive-2.0/reports/1 They are not yet available for 2.1 though!

On 11/8/16 9:04 PM, Francesco Ariis wrote:
On Tue, Nov 08, 2016 at 07:08:31PM -0800, Patrick Pelletier wrote:
On 11/8/16 5:59 PM, Francesco Ariis wrote:
Sometimes hackage fails to build documentation (maybe there is a hint on why on the logs). Where can I find those logs? https://hackage.haskell.org/package/normalization-insensitive-2.0/reports/1
They are not yet available for 2.1 though!
So then 2.0.0.1 is still queued up somewhere? Is there a way to check on the queue? --Patrick

On Tue, Nov 08, 2016 at 11:41:54PM -0800, Patrick Pelletier wrote:
on 11/8/16 9:04 pm, francesco ariis wrote:
on tue, nov 08, 2016 at 07:08:31pm -0800, patrick pelletier wrote:
on 11/8/16 5:59 pm, francesco ariis wrote:
sometimes hackage fails to build documentation (maybe there is a hint on why on the logs). Where can I find those logs? https://hackage.haskell.org/package/normalization-insensitive-2.0/reports/1
They are not yet available for 2.1 though!
So then 2.0.0.1 is still queued up somewhere? Is there a way to check on the queue?
Not sure really how you can check the queue (maybe someone more experienced can answer), I see that if a package gets 'skipped' for wichever reason, a build is not retriggered.

it is possible to upload docs manually: Maintainer's Corner edit package information Manage documentation select version Upload documentation or see this post: https://begriffs.com/posts/2014-10-25-creating-package-hackage.html there is a script at the bottom of the page. It works well. New docs appear on Hackage within minutes after running this script.

On 11/8/16 5:59 PM, Francesco Ariis wrote:
Sometimes hackage fails to build documentation (maybe there is a hint on why on the logs). In those cases, I take advantage of cabal:
cabal upload --documentation
(more info with `cabal upload --help`).
Thanks! This worked like a charm, although I first had to upgrade my cabal, and I also had to fix an incorrect lower bound for "text" in the "unicode-transforms" package (which is a dependency of normalization-insensitive). I wonder if the incorrect lower bound is why the docs failed to build automatically on Hackage? Although if the docs failed, it really ought to say "docs failed", not "docs pending". --Patrick

On Tue, Nov 08, 2016 at 05:37:07PM -0800, Patrick Pelletier wrote:
Under "Status", it says "Docs pending". (And the module names are all non-clickable.) Is it normal to take this long to build the docs?
A related point: If you get your package into Stackage then the docs will be built there too, and it's probably more reliable. I uploaded a new version of Opaleye yesterday and the docs haven't been built by Hackage yet but they have been built by Stackage. Tom

On 11/9/16 5:06 AM, Tom Ellis wrote:
If you get your package into Stackage then the docs will be built there too, and it's probably more reliable. I uploaded a new version of Opaleye yesterday and the docs haven't been built by Hackage yet but they have been built by Stackage.
I get your point, but I don't really like seeing the suggestion that Hackage should be abandoned because Stackage is better. (I especially dislike packages like "resourcet" which have entirely forgone giving a description of the package in the cabal file, and instead have only a note saying you should go to Stackage instead.) Hackage and Stackage serve different purposes. Hackage is all packages, while Stackage is a curated set of packages. (Therefore, Hackage is essentially the "upstream" of Stackage.) We shouldn't give up on Hackage. If FP Complete has a better doc builder, then it would be nice to see FP Complete and haskell.org work together on getting Hackage to use the FP Complete doc builder. Although a bit of competition can be healthy, it would also be nice to see cooperation in places where there is not a difference in philosophy. (And hopefully, "should have docs" is something everyone can agree on.) --Patrick

On Wed, Nov 09, 2016 at 01:11:14PM -0800, Patrick Pelletier wrote:
On 11/9/16 5:06 AM, Tom Ellis wrote:
If you get your package into Stackage then the docs will be built there too, and it's probably more reliable. I uploaded a new version of Opaleye yesterday and the docs haven't been built by Hackage yet but they have been built by Stackage.
I get your point, but I don't really like seeing the suggestion that Hackage should be abandoned because Stackage is better.
I never suggested that!

On 11/9/16 2:35 PM, Tom Ellis wrote:
On Wed, Nov 09, 2016 at 01:11:14PM -0800, Patrick Pelletier wrote:
On 11/9/16 5:06 AM, Tom Ellis wrote:
If you get your package into Stackage then the docs will be built there too, and it's probably more reliable. I uploaded a new version of Opaleye yesterday and the docs haven't been built by Hackage yet but they have been built by Stackage. I get your point, but I don't really like seeing the suggestion that Hackage should be abandoned because Stackage is better. I never suggested that!
I apologize; I know that's not exactly what you said, and I shouldn't have put words in your mouth. I think I was just expressing frustration with a general feeling I get sometimes that stack / Stackage / haskell-lang.org are trying to replace cabal / Hackage / haskell.org instead of complement them. But I misspoke in attributing that to you. Sorry. --Patrick

Stackage has a much easier time since it only builds against one version of
dependencies and one version of GHC. Adding a package to stackage also
requires that someone makes sure that any external build dependencies are
provided. Hackage on the other hand does not know any of these things so
there is a lot that can go wrong.
But new library versions can also be temporarily blocked from stackage, so
it can take months before a new version has docs on stackage.org.
For library authors stackage and hackage, cabal-install and stack are to a
large degree complimentary to each other, you will likely want to use both.
Application authors have the freedom to choose.
- Adam
On Fri, Nov 11, 2016 at 12:33 AM Patrick Pelletier
wrote:
On Wed, Nov 09, 2016 at 01:11:14PM -0800, Patrick Pelletier wrote:
On 11/9/16 5:06 AM, Tom Ellis wrote:
If you get your package into Stackage then the docs will be built
On 11/9/16 2:35 PM, Tom Ellis wrote: there too,
and it's probably more reliable. I uploaded a new version of Opaleye yesterday and the docs haven't been built by Hackage yet but they have been built by Stackage. I get your point, but I don't really like seeing the suggestion that Hackage should be abandoned because Stackage is better. I never suggested that!
I apologize; I know that's not exactly what you said, and I shouldn't have put words in your mouth. I think I was just expressing frustration with a general feeling I get sometimes that stack / Stackage / haskell-lang.org are trying to replace cabal / Hackage / haskell.org instead of complement them. But I misspoke in attributing that to you. Sorry.
--Patrick
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Hackage and Stackage serve different purposes. Hackage is all packages, while Stackage is a curated set of packages. (Therefore, Hackage is essentially the "upstream" of Stackage.) We shouldn't give up on Hackage.
I know I'm jumping into a very, very touchy topic, but isn't the following possible: * Stackage & Hackage combine -- even the .cabal & .stack file formats (if possible) * The combined entity supports curated packages (via LTS) AND a global free-for-all package list. I, as a user, get to decide how I want my packages to be resolved -- via a curated LTS or via the global package index. * The combined entity focuses on building a kickass, unified, piece of community infra, instead of dividing effort+resources and solving similar problems twice. -- Saurabh.

On Sun, Nov 13, 2016 at 3:35 PM, Saurabh Nanda
Hackage and Stackage serve different purposes. Hackage is all packages,
while Stackage is a curated set of packages. (Therefore, Hackage is essentially the "upstream" of Stackage.) We shouldn't give up on Hackage.
I know I'm jumping into a very, very touchy topic, but isn't the following possible:
* Stackage & Hackage combine -- even the .cabal & .stack file formats (if possible)
Let's not conflate two things. I assume you're talking about stack.yaml as the .stack file format. This should be a completely separate discussion for multiple reasons: * That's about Stack vs cabal-install instead of Stackage vs Hackage * It's completely necessary to have package-level vs project-level configuration (even cabal-install has a separate project config format separate from the .cabal format)
* The combined entity supports curated packages (via LTS) AND a global free-for-all package list. I, as a user, get to decide how I want my packages to be resolved -- via a curated LTS or via the global package index.
This was discussed in ernest at ICFP in 2014, and the resulting proposal was GPS Haskell. The idea was that Hackage would add support for curated package sets. Personally, I didn't think this was necessary, and cabal-install should have just learnt logic to get information from stackage.org so that adding the functionality to Hackage wasn't a blocker for getting curated package sets available to users. In reality, the curated package set feature never got added to Hackage, cabal-install never added curated package set support, GPS Haskell was abandoned, and Stack and LTS Haskell were created instead.
* The combined entity focuses on building a kickass, unified, piece of community infra, instead of dividing effort+resources and solving similar problems twice.
It's a wonderful idea. Michael

* Stackage & Hackage combine -- even the .cabal & .stack file formats (if possible)
Let's not conflate two things. I assume you're talking about stack.yaml as the .stack file format. This should be a completely separate discussion for multiple reasons:
* That's about Stack vs cabal-install instead of Stackage vs Hackage * It's completely necessary to have package-level vs project-level configuration (even cabal-install has a separate project config format separate from the .cabal format)
I didn't realise that. Let me read more about the problems that stack is trying to solve vs those that cabal is trying to solve. To my untrained eye, they're solving very similar problems to exist as two separate projects. Isn't a package a kind of a project? Or vice-versa. As I said, I need to read more on this topic.
This was discussed in ernest at ICFP in 2014, and the resulting proposal was GPS Haskell. The idea was that Hackage would add support for curated package sets. Personally, I didn't think this was necessary, and cabal-install should have just learnt logic to get information from stackage.org so that adding the functionality to Hackage wasn't a blocker for getting curated package sets available to users.
Was the only reason to suggest the fetching of curated lists from Stackage in interest of faster go-to-market? Wouldn't it require the community to maintain 2 sets of highly-available infra? Also, wondering aloud, is the curation process purely algorithmic or human assisted?
In reality, the curated package set feature never got added to Hackage, cabal-install never added curated package set support, GPS Haskell was abandoned, and Stack and LTS Haskell were created instead.
Where can I read more about GPS Haskell? I managed to get to http://www.ozonehouse.com/mark/platform/GPS-Haskell-HIW2014.pdf from your blog post at https://www.fpcomplete.com/blog/2014/12/backporting-bug-fixes, but it's a 404 right now. -- Saurabh.

On Sun, Nov 13, 2016 at 4:05 PM, Saurabh Nanda
* Stackage & Hackage combine -- even the .cabal & .stack file formats (if possible)
Let's not conflate two things. I assume you're talking about stack.yaml as the .stack file format. This should be a completely separate discussion for multiple reasons:
* That's about Stack vs cabal-install instead of Stackage vs Hackage * It's completely necessary to have package-level vs project-level configuration (even cabal-install has a separate project config format separate from the .cabal format)
I didn't realise that. Let me read more about the problems that stack is trying to solve vs those that cabal is trying to solve. To my untrained eye, they're solving very similar problems to exist as two separate projects. Isn't a package a kind of a project? Or vice-versa.
As I said, I need to read more on this topic.
This has been discussed a few times on this mailing list and Reddit (I think), but I don't have any links handy. The basic idea is: * A package is a single .cabal file and everything it references * A project is a set of 1 or more packages, plus some project settings * Project settings include things like "use this version of GHC," "use these dependencies," etc. * Projects allow us to work on multiple packages at once in a coherent way, without hard-coding things like a specific GHC version into a package configuration.
This was discussed in ernest at ICFP in 2014, and the resulting proposal
was GPS Haskell. The idea was that Hackage would add support for curated package sets. Personally, I didn't think this was necessary, and cabal-install should have just learnt logic to get information from stackage.org so that adding the functionality to Hackage wasn't a blocker for getting curated package sets available to users.
Was the only reason to suggest the fetching of curated lists from Stackage in interest of faster go-to-market? Wouldn't it require the community to maintain 2 sets of highly-available infra? Also, wondering aloud, is the curation process purely algorithmic or human assisted?
One aspect was "faster go-to-market." But I also think there's no reason why one website needs to be the central location for all things. Some people disagree with me, which is fine. In practice though, Hackage has had issues with adding new features, whereas adding new functionality elsewhere is much simpler. As to "maintain[ing] 2 sets of highly-available infra," ignoring questions of whether we actually have 2 such infras right now: all of the Stackage snapshot configurations are actually stored in Git repositories on Github, so: 1. It's trivial to mirror them to as many places as desired, and 2. We're leveraging pre-existing infrastructure from others In other words, when I said above "point at Stackage," that's not really too relevant anymore. It would really be "point at relevant Github repos," which is precisely what Stack does. You probably will want to read the stackage README[1] and linked locations, it include the relevant repos, maintainer agreement, and curator guides, which explains what pieces of the puzzle are algorithmic and which parts human assisted. [1] https://github.com/fpco/stackage#stackage
In reality, the curated package set feature never got added to Hackage, cabal-install never added curated package set support, GPS Haskell was abandoned, and Stack and LTS Haskell were created instead.
Where can I read more about GPS Haskell? I managed to get to http://www.ozonehouse.com/mark/platform/GPS-Haskell-HIW2014.pdf from your blog post at https://www.fpcomplete.com/blog/2014/12/backporting-bug-fixes, but it's a 404 right now.
-- Saurabh.
I don't have a copy of the presentation linked from the blog post, but here's the email I sent after ICFP 2014 describing the work items to make GPS Haskell a reality: https://gist.github.com/snoyberg/b53672e04a432ecbedb106d63725b5a4 Michael

On 11/13/16 6:05 AM, Saurabh Nanda wrote:
* Stackage & Hackage combine -- even the .cabal & .stack file formats (if possible)
Let's not conflate two things. I assume you're talking about stack.yaml as the ..stack file format. This should be a completely separate discussion for multiple reasons:
* That's about Stack vs cabal-install instead of Stackage vs Hackage * It's completely necessary to have package-level vs project-level configuration (even cabal-install has a separate project config format separate from the .cabal format)
I didn't realise that. Let me read more about the problems that stack is trying to solve vs those that cabal is trying to solve. To my untrained eye, they're solving very similar problems to exist as two separate projects. Isn't a package a kind of a project? Or vice-versa.
Here's what I wrote last time this was discussed: https://mail.haskell.org/pipermail/haskell-cafe/2016-September/124896.html which is pretty much what Snoyman said in his follow-up here. To me, stack is trying to solve the same problems as hsenv, as well as the same problems as cabal-install, as well as some problems that neither solves. And although there's overlap between cabal-install and stack, that's not the same as saying there's overlap between stack.yaml and .cabal files. --Patrick

Here's what I wrote last time this was discussed:
https://mail.haskell.org/pipermail/haskell-cafe/2016-September/124896.html
which is pretty much what Snoyman said in his follow-up here. To me, stack is trying to solve the same problems as hsenv, as well as the same problems as cabal-install, as well as some problems that neither solves. And although there's overlap between cabal-install and stack, that's not the same as saying there's overlap between stack.yaml and .cabal files.
I read that and https://www.fpcomplete.com/blog/2015/06/why-is-stack-not-cabal as well. Comparing to the Ruby world, would it be fair to say stack is analogous to rvm (or rbenv) and cabal is analogous to rubygems? One of the things that makes stack different from a Gemfile.lock is probably the following (from https://www.fpcomplete.com/blog/2015/06/why-is-stack-not-cabal): It's also a black box which depends on three pieces of global, mutable,
implicit state: the compiler and versions of system libraries on your system, the Cabal packages installed in GHC’s package database, and the package metadata du jour downloaded from Hackage (via cabal update). Running cabal install at different times can lead to wildly different install plans, without giving any good reason to the user.
Basically the LTS. If you're using the same LTS resolver+OS, you'll get the same set of dependent packages every single time, right? Still pressing this further, is there really no way to wrap dependency management (cabal) and environment management (stack) in a single UI? Although that's lesser of a problem than the hackage/stackage divide. Really hope to see a unified tool that everyone gets behind, instead of diving effort and resources. -- Saurabh.

Saurabh Nanda
Really hope to see a unified tool that everyone gets behind, instead of diving effort and resources.
I disagree that this is always the best approach. Duplication of effort is not the worst thing that can happen to Haskell. To the contrary, many of us *love* working with Haskell. At the point we're at right now, it's more important to let people work on stuff they care about without being told they are doing it wrong. Let's be aware that efforts to dictate unification in this area have led to a lot of social wounds, that just need to be given time to heal before they are stressed again.

On Mon, Nov 14, 2016 at 12:43 AM, Chris Smith
Saurabh Nanda
wrote: Really hope to see a unified tool that everyone gets behind, instead of diving effort and resources.
I disagree that this is always the best approach. Duplication of effort is not the worst thing that can happen to Haskell. To the contrary, many of us *love* working with Haskell. At the point we're at right now, it's more important to let people work on stuff they care about without being told they are doing it wrong. Let's be aware that efforts to dictate unification in this area have led to a lot of social wounds, that just need to be given time to heal before they are stressed again.
I know this is a sensitive topic. I won't press this further. -- Saurabh.

Sorry — the builder for docs has been in rough shape and we’re working on it. In this case it got stuck due to disk space issues and monitoring didn’t catch it. Its now running again but doing some catching up. (And lots of stuff in the queue it failed to build due to other problems needs to be replaced there, which is a manual process at the moment if we don’t want the queue as a whole to just drown out any new things at all with backlog). It would be much better to expose more queues and have better monitoring (and real prioritization), and another volunteer to help ben out on this would be very welcome (please contact me if you’re interested — the code lives at https://github.com/haskell/hackage-server/blob/master/BuildClient.hs and there’s some uncommitted work on queuing as well). —gerhsom On November 8, 2016 at 8:37:19 PM, Patrick Pelletier (code@funwithsoftware.org) wrote:
This is only my second time uploading a package to Hackage, so I don't yet have a feel for how it's supposed to go. I uploaded normalization-insensitive-2.0.0.1 about 24 hours ago:
https://hackage.haskell.org/package/normalization-insensitive-2.0.0.1
Under "Status", it says "Docs pending". (And the module names are all non-clickable.) Is it normal to take this long to build the docs? Is there some way to find out where in the queue my job is? Is this an indication that something has gone wrong? How do I fix it?
Thanks,
--Patrick
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

I shared this thought privately before, but worth sharing publicly if
someone wants to try following up on it: all of the docs build for the
Stackage server are available on S3, and could by leveraged by Hackage as
well. When I brought this up earlier, there was a large-ish list of
requested changes to the generated HTML, which I didn't want to implement,
but perhaps someone can find a way to bridge the gap.
On Fri, Nov 11, 2016 at 6:43 PM, Gershom B
Sorry — the builder for docs has been in rough shape and we’re working on it. In this case it got stuck due to disk space issues and monitoring didn’t catch it. Its now running again but doing some catching up. (And lots of stuff in the queue it failed to build due to other problems needs to be replaced there, which is a manual process at the moment if we don’t want the queue as a whole to just drown out any new things at all with backlog). It would be much better to expose more queues and have better monitoring (and real prioritization), and another volunteer to help ben out on this would be very welcome (please contact me if you’re interested — the code lives at https://github.com/haskell/hackage-server/blob/master/ BuildClient.hs and there’s some uncommitted work on queuing as well).
—gerhsom
On November 8, 2016 at 8:37:19 PM, Patrick Pelletier ( code@funwithsoftware.org) wrote:
This is only my second time uploading a package to Hackage, so I don't yet have a feel for how it's supposed to go. I uploaded normalization-insensitive-2.0.0.1 about 24 hours ago:
https://hackage.haskell.org/package/normalization-insensitive-2.0.0.1
Under "Status", it says "Docs pending". (And the module names are all non-clickable.) Is it normal to take this long to build the docs? Is there some way to find out where in the queue my job is? Is this an indication that something has gone wrong? How do I fix it?
Thanks,
--Patrick
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Can you share the list of requested changes? And what is the cause of the difference? Is Stackage running Haddock with different options than Hackage does? Or is it doing something other than just running Haddock? Is the code that builds the Stackage docs available? --Patrick On 11/12/16 10:46 PM, Michael Snoyman wrote:
I shared this thought privately before, but worth sharing publicly if someone wants to try following up on it: all of the docs build for the Stackage server are available on S3, and could by leveraged by Hackage as well. When I brought this up earlier, there was a large-ish list of requested changes to the generated HTML, which I didn't want to implement, but perhaps someone can find a way to bridge the gap.
On Fri, Nov 11, 2016 at 6:43 PM, Gershom B
mailto:gershomb@gmail.com> wrote: Sorry — the builder for docs has been in rough shape and we’re working on it. In this case it got stuck due to disk space issues and monitoring didn’t catch it. Its now running again but doing some catching up. (And lots of stuff in the queue it failed to build due to other problems needs to be replaced there, which is a manual process at the moment if we don’t want the queue as a whole to just drown out any new things at all with backlog). It would be much better to expose more queues and have better monitoring (and real prioritization), and another volunteer to help ben out on this would be very welcome (please contact me if you’re interested — the code lives at https://github.com/haskell/hackage-server/blob/master/BuildClient.hs https://github.com/haskell/hackage-server/blob/master/BuildClient.hs and there’s some uncommitted work on queuing as well).
—gerhsom
On November 8, 2016 at 8:37:19 PM, Patrick Pelletier (code@funwithsoftware.org mailto:code@funwithsoftware.org) wrote: > This is only my second time uploading a package to Hackage, so I don't > yet have a feel for how it's supposed to go. I uploaded > normalization-insensitive-2.0.0.1 about 24 hours ago: > > https://hackage.haskell.org/package/normalization-insensitive-2.0.0.1 https://hackage.haskell.org/package/normalization-insensitive-2.0.0.1 > > Under "Status", it says "Docs pending". (And the module names are all > non-clickable.) Is it normal to take this long to build the docs? Is > there some way to find out where in the queue my job is? Is this an > indication that something has gone wrong? How do I fix it? > > Thanks, > > --Patrick > > _______________________________________________ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post.
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Here were Duncan's comments at the time:
* the docs need to match the URL scheme on hackage, in terms of
linking back to other packages docs on hackage. This needs to be
done using sufficiently-relative urls so that http & https
works.
* it's generally not desirable to overwrite/replace existing docs.
You can check which packages are missing docs by consulting this
resource: http://hackage.haskell.org/packages/docs.json
* we generate the docs so that only one style is listed, so that
haddock does not include it's style switcher menu.
* we do include all the files haddock generates, including the
minus.gif, synopsis.png.
* hackage currently does not use a shared haddock css and js file
(though doing so is a good idea), so these are also included in
the doc bundle, just as haddock produces them.
This presumes having some kind of an upload script running from the
Stackage side, whereas my recommendation had been that Hackage be modified
to use the Stackage S3 docs as a backup for when its doc building fails.
You can see an example of generated docs at:
https://s3.amazonaws.com/haddock.stackage.org/lts-7.0/
tagstream-conduit-0.5.5.3/Text-HTML-TagStream-ByteString.html
The code which calls Haddock is available at:
https://github.com/fpco/stackage-curator/blob/master/
Stackage/PerformBuild.hs
By the way, regarding your comments about resourcet and FP Complete: I'm
honestly offended at this tone. I've clearly attempted quite often and
quite sincerely to get such cooperation to happen, and have been rebuffed.
Even if you're unaware of that, the implied accusation assumes a lot which
isn't true. I honestly considered ignoring this thread entirely based on
this tone, I'm tired of dealing with it.
On Sun, Nov 13, 2016 at 8:59 AM, Patrick Pelletier wrote: Can you share the list of requested changes? And what is the cause of the
difference? Is Stackage running Haddock with different options than
Hackage does? Or is it doing something other than just running Haddock?
Is the code that builds the Stackage docs available? --Patrick On 11/12/16 10:46 PM, Michael Snoyman wrote: I shared this thought privately before, but worth sharing publicly if
someone wants to try following up on it: all of the docs build for the
Stackage server are available on S3, and could by leveraged by Hackage as
well. When I brought this up earlier, there was a large-ish list of
requested changes to the generated HTML, which I didn't want to implement,
but perhaps someone can find a way to bridge the gap. On Fri, Nov 11, 2016 at 6:43 PM, Gershom B Sorry — the builder for docs has been in rough shape and we’re working on
it. In this case it got stuck due to disk space issues and monitoring
didn’t catch it. Its now running again but doing some catching up. (And
lots of stuff in the queue it failed to build due to other problems needs
to be replaced there, which is a manual process at the moment if we don’t
want the queue as a whole to just drown out any new things at all with
backlog). It would be much better to expose more queues and have better
monitoring (and real prioritization), and another volunteer to help ben out
on this would be very welcome (please contact me if you’re interested — the
code lives at https://github.com/haskell/hackage-server/blob/master/Bui
ldClient.hs and there’s some uncommitted work on queuing as well). —gerhsom On November 8, 2016 at 8:37:19 PM, Patrick Pelletier (
code@funwithsoftware.org) wrote: This is only my second time uploading a package to Hackage, so I don't
yet have a feel for how it's supposed to go. I uploaded
normalization-insensitive-2.0.0.1 about 24 hours ago: https://hackage.haskell.org/package/normalization-insensitive-2.0.0.1 Under "Status", it says "Docs pending". (And the module names are all
non-clickable.) Is it normal to take this long to build the docs? Is
there some way to find out where in the queue my job is? Is this an
indication that something has gone wrong? How do I fix it? Thanks, --Patrick _______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post. _______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

On 11/13/16 1:29 AM, Michael Snoyman wrote:
By the way, regarding your comments about resourcet and FP Complete: I'm honestly offended at this tone. I've clearly attempted quite often and quite sincerely to get such cooperation to happen, and have been rebuffed. Even if you're unaware of that, the implied accusation assumes a lot which isn't true. I honestly considered ignoring this thread entirely based on this tone, I'm tired of dealing with it.
I'm very sorry! I did not mean to offend. I take Simon PJ seriously in his call to have a productive, respectful discourse in the Haskell community. To be clear, I did not object to resourcet linking to Stackage; what I objected to is that the link to stackage is in place of having any sort of description of resourcet, rather than in addition to it. To me, that feels like it is making Hackage less useful (by denying it the package description) rather than making the most of Hackage, even though it isn't perfect. This makes me a little grouchy, and perhaps this grouchiness seeped into my comments, for which I apologize. Also, Hackage does have the ability to upload docs very conveniently, via "cabal upload --documentation", so this can be used to work around the fact that "Hackage documentation generation is not reliable." We need to all work together in good faith to make the Haskell ecosystem better. I am doing my best to do that, even if I am not perfect. I have already volunteered privately to Gershom B to work on the Hackage documentation generation code. I would like to make Hackage docs more reliable. This may or may not involve integrating with the S3 docs. That is a technical decision, and no slight should be implied, whatever the outcome. Once again, I'm sorry if I have not succeeded in upholding Simon PJ's standards for the Haskell community. I genuinely appreciate the contributions you and FP Complete have made to the Haskell ecosystem. --Patrick

On Sun, Nov 13, 2016 at 6:43 PM, Patrick Pelletier wrote: On 11/13/16 1:29 AM, Michael Snoyman wrote: By the way, regarding your comments about resourcet and FP Complete: I'm
honestly offended at this tone. I've clearly attempted quite often and
quite sincerely to get such cooperation to happen, and have been rebuffed.
Even if you're unaware of that, the implied accusation assumes a lot which
isn't true. I honestly considered ignoring this thread entirely based on
this tone, I'm tired of dealing with it. I'm very sorry! I did not mean to offend. I take Simon PJ seriously in
his call to have a productive, respectful discourse in the Haskell
community. To be clear, I did not object to resourcet linking to Stackage; what I
objected to is that the link to stackage is in place of having any sort of
description of resourcet, rather than in addition to it. To me, that feels
like it is making Hackage less useful (by denying it the package
description) rather than making the most of Hackage, even though it isn't
perfect. This makes me a little grouchy, and perhaps this grouchiness
seeped into my comments, for which I apologize. Also, Hackage does have the ability to upload docs very conveniently, via
"cabal upload --documentation", so this can be used to work around the fact
that "Hackage documentation generation is not reliable." We need to all work together in good faith to make the Haskell ecosystem
better. I am doing my best to do that, even if I am not perfect. I have
already volunteered privately to Gershom B to work on the Hackage
documentation generation code. I would like to make Hackage docs more
reliable. This may or may not involve integrating with the S3 docs. That
is a technical decision, and no slight should be implied, whatever the
outcome. Once again, I'm sorry if I have not succeeded in upholding Simon PJ's
standards for the Haskell community. I genuinely appreciate the
contributions you and FP Complete have made to the Haskell ecosystem. --Patrick Thank you for that, offense _un_taken :). Just to let you (and others) know
where I'm coming from on this:
I regularly get complaints of "why don't you do X," where X is a
significant amount of extra work. Writing up dual descriptions in both
README.md and cabal description fields is a prime example, and something I
argued very hard for on Hackage. I'm disappointed with the way the
description is displayed; I'd have much preferred that the README.md files
I write would have been used. If you look on resourcet on Hackage, for
example, there's a much more thorough description of the package at the
bottom. I think this was a major mistake, but there's not much I can do
about it.
So my general request: instead of saying "I dislike X" and imply that the
author (me, in this case) should take on some unspecified work to change
what they're doing, try assuming some good faith. I put a lot of effort
into getting display of my packages to be better on Hackage, and eventually
gave up. Each comment about how I didn't do enough work is offensive, and
pretty much puts another nail in the coffin of me wanting to be involved in
Hackage ever again.
Michael

On 2016-11-13 at 18:04:51 +0100, Michael Snoyman wrote: [...]
I regularly get complaints of "why don't you do X," where X is a significant amount of extra work. Writing up dual descriptions in both README.md and cabal description fields is a prime example, and something I argued very hard for on Hackage.
I'm disappointed with the way the description is displayed; I'd have much preferred that the README.md files I write would have been used.
Maybe the reasoning behind this wasn't explained well enough, so let me try again. The three seemingly overlapping parts a) synopsis b) description c) README serve three different purposes and have different technical properties (and you find a very similar 3-part division in e.g. Debian's packaging and others): a) the synopsis is a single-line short title which `cabal list` or search results show as a single line; it should give a first rough idea what the package is about in order to decide whether you want to look closer at b) b) this is like a paper abstract, giving you the gist of what a package is about, and wether to give it a try; it's displayed by e.g. `cabal info somepkg` or at the top of Hackage packages; this should contain enough information to know the feature scope of the package in order to decide whether to continue looking at c) and/or the Haddocks. The description field is typically just a few paragraphs, and ideally fits on a terminal screen; you don't want it to be too long, as it's supposed to be an elevator-pitch for your package. ---- c) Finally, the README is for additional information which isn't suitable for the description-field (like e.g. more in-depth explainations, history about package (NB: not the changelog), code examples, installation instructions, information about contributing etc...); also, since the README is quite easily rather large is *not* included in the 01-index.tar So while there may appear to be some overlap between b) and c), it's really not that much; and there's precedent for doing it this way. Also note there's a technical boundary between b) & c) in that a) & b) are indexed; i.e. by being inside the `.cabal` meta-data, they are contained in 01-index.tar, and thus are available to external tooling for text-indexing; Moreover, the synopsis/description fields are part of the meta-data that gets registered with ghc-pkg, so that information is available via `ghc-pkg` as well. Whereas in order to access/search c) you'd currently have to download the source-tarball (or bypass hackage-security, and access hackage.h.o's web-services directly but that would come with its own technical problems by doing so). So READMEs are rather out-of-band information, and should be treaated as such.
If you look on resourcet on Hackage, for example, there's a much more thorough description of the package at the bottom. I think this was a major mistake, but there's not much I can do about it.
Btw, you can modify the current `description` (as well as the `synopsis`) field via .cabal editing on Hackage if you want to set a more meaningful description for the benefit of e.g. `cabal info resourcet` or also Hackage users (since the README is *not* indexed bythe Hackage search service).

As an update -- the builder had a few other issues besides disk-space
in terms of getting a better retry policy in place, and properly
prioritizing new over older packages, etc. I think as of this morning
they're finally all worked through and it seems to be slowly chugging
through the backlog in a reasonable order, prioritizing new
uploads.I'll be checking in on the logs daily for the time being to
make sure it does't seem to be getting stuck.
If your docs haven't built and you're impatient (understandably so!),
please don't bump your package version to kick the priority -- that'll
just put more metadata on the server we don't need. Instead, you can
build and upload your own docs as in this sample shell script:
https://github.com/ekmett/lens/blob/master/scripts/hackage-docs.sh
Best,
Gershom
On Fri, Nov 11, 2016 at 11:43 AM, Gershom B
Sorry — the builder for docs has been in rough shape and we’re working on it. In this case it got stuck due to disk space issues and monitoring didn’t catch it. Its now running again but doing some catching up. (And lots of stuff in the queue it failed to build due to other problems needs to be replaced there, which is a manual process at the moment if we don’t want the queue as a whole to just drown out any new things at all with backlog). It would be much better to expose more queues and have better monitoring (and real prioritization), and another volunteer to help ben out on this would be very welcome (please contact me if you’re interested — the code lives at https://github.com/haskell/hackage-server/blob/master/BuildClient.hs and there’s some uncommitted work on queuing as well).
—gerhsom
On November 8, 2016 at 8:37:19 PM, Patrick Pelletier (code@funwithsoftware.org) wrote:
This is only my second time uploading a package to Hackage, so I don't yet have a feel for how it's supposed to go. I uploaded normalization-insensitive-2.0.0.1 about 24 hours ago:
https://hackage.haskell.org/package/normalization-insensitive-2.0.0.1
Under "Status", it says "Docs pending". (And the module names are all non-clickable.) Is it normal to take this long to build the docs? Is there some way to find out where in the queue my job is? Is this an indication that something has gone wrong? How do I fix it?
Thanks,
--Patrick
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

I am not sure if the problem has been resolved or if it has resurfaced
again after getting resolved. I uploaded a package recently and the docs
have not been generated even after a day. Here is the link to the package
candidate that I uploaded:
https://hackage.haskell.org/package/unicode-transforms-0.2.1/candidate
Is there a place where I can see any debug logs or diagnostic messages to
figure out what went wrong?
-harendra
On 16 November 2016 at 23:31, Gershom B
As an update -- the builder had a few other issues besides disk-space in terms of getting a better retry policy in place, and properly prioritizing new over older packages, etc. I think as of this morning they're finally all worked through and it seems to be slowly chugging through the backlog in a reasonable order, prioritizing new uploads.I'll be checking in on the logs daily for the time being to make sure it does't seem to be getting stuck.
If your docs haven't built and you're impatient (understandably so!), please don't bump your package version to kick the priority -- that'll just put more metadata on the server we don't need. Instead, you can build and upload your own docs as in this sample shell script:
https://github.com/ekmett/lens/blob/master/scripts/hackage-docs.sh
Best, Gershom
On Fri, Nov 11, 2016 at 11:43 AM, Gershom B
wrote: Sorry — the builder for docs has been in rough shape and we’re working on it. In this case it got stuck due to disk space issues and monitoring didn’t catch it. Its now running again but doing some catching up. (And lots of stuff in the queue it failed to build due to other problems needs to be replaced there, which is a manual process at the moment if we don’t want the queue as a whole to just drown out any new things at all with backlog). It would be much better to expose more queues and have better monitoring (and real prioritization), and another volunteer to help ben out on this would be very welcome (please contact me if you’re interested — the code lives at https://github.com/haskell/hackage-server/blob/master/ BuildClient.hs and there’s some uncommitted work on queuing as well).
—gerhsom
On November 8, 2016 at 8:37:19 PM, Patrick Pelletier ( code@funwithsoftware.org) wrote:
This is only my second time uploading a package to Hackage, so I don't yet have a feel for how it's supposed to go. I uploaded normalization-insensitive-2.0.0.1 about 24 hours ago:
https://hackage.haskell.org/package/normalization-insensitive-2.0.0.1
Under "Status", it says "Docs pending". (And the module names are all non-clickable.) Is it normal to take this long to build the docs? Is there some way to find out where in the queue my job is? Is this an indication that something has gone wrong? How do I fix it?
Thanks,
--Patrick
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Good question! The current incarnation of the builder doesn’t build candidate documentation — this is something we should change/implement. (Not sure if it ever built candidate documentation, actually?) The builder itself (I just checked) is running just fine and generating documentation for fully released packages. Best, Gershom On January 24, 2017 at 1:22:05 AM, Harendra Kumar (harendra.kumar@gmail.com) wrote:
I am not sure if the problem has been resolved or if it has resurfaced again after getting resolved. I uploaded a package recently and the docs have not been generated even after a day. Here is the link to the package candidate that I uploaded:
https://hackage.haskell.org/package/unicode-transforms-0.2.1/candidate
Is there a place where I can see any debug logs or diagnostic messages to figure out what went wrong?
-harendra
On 16 November 2016 at 23:31, Gershom B wrote:
As an update -- the builder had a few other issues besides disk-space in terms of getting a better retry policy in place, and properly prioritizing new over older packages, etc. I think as of this morning they're finally all worked through and it seems to be slowly chugging through the backlog in a reasonable order, prioritizing new uploads.I'll be checking in on the logs daily for the time being to make sure it does't seem to be getting stuck.
If your docs haven't built and you're impatient (understandably so!), please don't bump your package version to kick the priority -- that'll just put more metadata on the server we don't need. Instead, you can build and upload your own docs as in this sample shell script:
https://github.com/ekmett/lens/blob/master/scripts/hackage-docs.sh
Best, Gershom
On Fri, Nov 11, 2016 at 11:43 AM, Gershom B wrote:
Sorry — the builder for docs has been in rough shape and we’re working on it. In this case it got stuck due to disk space issues and monitoring didn’t catch it. Its now running again but doing some catching up. (And lots of stuff in the queue it failed to build due to other problems needs to be replaced there, which is a manual process at the moment if we don’t want the queue as a whole to just drown out any new things at all with backlog). It would be much better to expose more queues and have better monitoring (and real prioritization), and another volunteer to help ben out on this would be very welcome (please contact me if you’re interested — the code lives at https://github.com/haskell/hackage-server/blob/master/ BuildClient.hs and there’s some uncommitted work on queuing as well).
—gerhsom
On November 8, 2016 at 8:37:19 PM, Patrick Pelletier ( code@funwithsoftware.org) wrote:
This is only my second time uploading a package to Hackage, so I don't yet have a feel for how it's supposed to go. I uploaded normalization-insensitive-2.0.0.1 about 24 hours ago:
https://hackage.haskell.org/package/normalization-insensitive-2.0.0.1
Under "Status", it says "Docs pending". (And the module names are all non-clickable.) Is it normal to take this long to build the docs? Is there some way to find out where in the queue my job is? Is this an indication that something has gone wrong? How do I fix it?
Thanks,
--Patrick
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

It did build candidate documentation earlier for me. Not sure if something
changed recently. I always uploaded candidates, and when the documentation
was generated, I reviewed it and then published the candidate.
-harendra
On 24 January 2017 at 12:43, Gershom B
Good question!
The current incarnation of the builder doesn’t build candidate documentation — this is something we should change/implement. (Not sure if it ever built candidate documentation, actually?)
The builder itself (I just checked) is running just fine and generating documentation for fully released packages.
Best, Gershom
On January 24, 2017 at 1:22:05 AM, Harendra Kumar (harendra.kumar@gmail.com) wrote:
I am not sure if the problem has been resolved or if it has resurfaced again after getting resolved. I uploaded a package recently and the docs have not been generated even after a day. Here is the link to the package candidate that I uploaded:
https://hackage.haskell.org/package/unicode-transforms-0.2.1/candidate
Is there a place where I can see any debug logs or diagnostic messages to figure out what went wrong?
-harendra
On 16 November 2016 at 23:31, Gershom B wrote:
As an update -- the builder had a few other issues besides disk-space in terms of getting a better retry policy in place, and properly prioritizing new over older packages, etc. I think as of this morning they're finally all worked through and it seems to be slowly chugging through the backlog in a reasonable order, prioritizing new uploads.I'll be checking in on the logs daily for the time being to make sure it does't seem to be getting stuck.
If your docs haven't built and you're impatient (understandably so!), please don't bump your package version to kick the priority -- that'll just put more metadata on the server we don't need. Instead, you can build and upload your own docs as in this sample shell script:
https://github.com/ekmett/lens/blob/master/scripts/hackage-docs.sh
Best, Gershom
On Fri, Nov 11, 2016 at 11:43 AM, Gershom B wrote:
Sorry — the builder for docs has been in rough shape and we’re working on it. In this case it got stuck due to disk space issues and monitoring didn’t catch it. Its now running again but doing some catching up. (And lots of stuff in the queue it failed to build due to other problems needs to be replaced there, which is a manual process at the moment if we don’t want the queue as a whole to just drown out any new things at all with backlog). It would be much better to expose more queues and have better monitoring (and real prioritization), and another volunteer to help ben out on this would be very welcome (please contact me if you’re interested — the code lives at https://github.com/haskell/hackage-server/blob/master/ BuildClient.hs and there’s some uncommitted work on queuing as well).
—gerhsom
On November 8, 2016 at 8:37:19 PM, Patrick Pelletier ( code@funwithsoftware.org) wrote:
This is only my second time uploading a package to Hackage, so I don't yet have a feel for how it's supposed to go. I uploaded normalization-insensitive-2.0.0.1 about 24 hours ago:
https://hackage.haskell.org/package/normalization- insensitive-2.0.0.1
Under "Status", it says "Docs pending". (And the module names are all non-clickable.) Is it normal to take this long to build the docs? Is there some way to find out where in the queue my job is? Is this an indication that something has gone wrong? How do I fix it?
Thanks,
--Patrick
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Right — we changed the builder to improve stability and handle things chronologically to avoid backlog when things got out of wack. But in the process I think we lost the candidate building. I filed a ticket to improve this: https://github.com/haskell/hackage-server/issues/568 Thanks again for the report! —Gershom On January 24, 2017 at 3:00:45 AM, Harendra Kumar (harendra.kumar@gmail.com) wrote:
It did build candidate documentation earlier for me. Not sure if something changed recently. I always uploaded candidates, and when the documentation was generated, I reviewed it and then published the candidate.
-harendra
On 24 January 2017 at 12:43, Gershom B wrote:
Good question!
The current incarnation of the builder doesn’t build candidate documentation — this is something we should change/implement. (Not sure if it ever built candidate documentation, actually?)
The builder itself (I just checked) is running just fine and generating documentation for fully released packages.
Best, Gershom
On January 24, 2017 at 1:22:05 AM, Harendra Kumar (harendra.kumar@gmail.com) wrote:
I am not sure if the problem has been resolved or if it has resurfaced again after getting resolved. I uploaded a package recently and the docs have not been generated even after a day. Here is the link to the package candidate that I uploaded:
https://hackage.haskell.org/package/unicode-transforms-0.2.1/candidate
Is there a place where I can see any debug logs or diagnostic messages to figure out what went wrong?
-harendra
On 16 November 2016 at 23:31, Gershom B wrote:
As an update -- the builder had a few other issues besides disk-space in terms of getting a better retry policy in place, and properly prioritizing new over older packages, etc. I think as of this morning they're finally all worked through and it seems to be slowly chugging through the backlog in a reasonable order, prioritizing new uploads.I'll be checking in on the logs daily for the time being to make sure it does't seem to be getting stuck.
If your docs haven't built and you're impatient (understandably so!), please don't bump your package version to kick the priority -- that'll just put more metadata on the server we don't need. Instead, you can build and upload your own docs as in this sample shell script:
https://github.com/ekmett/lens/blob/master/scripts/hackage-docs.sh
Best, Gershom
On Fri, Nov 11, 2016 at 11:43 AM, Gershom B wrote:
Sorry — the builder for docs has been in rough shape and we’re working on it. In this case it got stuck due to disk space issues and monitoring didn’t catch it. Its now running again but doing some catching up. (And lots of stuff in the queue it failed to build due to other problems needs to be replaced there, which is a manual process at the moment if we don’t want the queue as a whole to just drown out any new things at all with backlog). It would be much better to expose more queues and have better monitoring (and real prioritization), and another volunteer to help ben out on this would be very welcome (please contact me if you’re interested — the code lives at https://github.com/haskell/hackage-server/blob/master/ BuildClient.hs and there’s some uncommitted work on queuing as well).
—gerhsom
On November 8, 2016 at 8:37:19 PM, Patrick Pelletier ( code@funwithsoftware.org) wrote:
This is only my second time uploading a package to Hackage, so I don't yet have a feel for how it's supposed to go. I uploaded normalization-insensitive-2.0.0.1 about 24 hours ago:
https://hackage.haskell.org/package/normalization- insensitive-2.0.0.1
Under "Status", it says "Docs pending". (And the module names are all non-clickable.) Is it normal to take this long to build the docs? Is there some way to find out where in the queue my job is? Is this an indication that something has gone wrong? How do I fix it?
Thanks,
--Patrick
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

You are right, it is broken for candidates but works for package upload. I
just uploaded the package directly and documentation was generated fine.
Thanks,
harendra
On 24 January 2017 at 13:36, Gershom B
Right — we changed the builder to improve stability and handle things chronologically to avoid backlog when things got out of wack. But in the process I think we lost the candidate building. I filed a ticket to improve this:
https://github.com/haskell/hackage-server/issues/568
Thanks again for the report!
—Gershom
It did build candidate documentation earlier for me. Not sure if something changed recently. I always uploaded candidates, and when the documentation was generated, I reviewed it and then published the candidate.
-harendra
On 24 January 2017 at 12:43, Gershom B wrote:
Good question!
The current incarnation of the builder doesn’t build candidate documentation — this is something we should change/implement. (Not sure if it ever built candidate documentation, actually?)
The builder itself (I just checked) is running just fine and generating documentation for fully released packages.
Best, Gershom
On January 24, 2017 at 1:22:05 AM, Harendra Kumar (harendra.kumar@gmail.com) wrote:
I am not sure if the problem has been resolved or if it has resurfaced again after getting resolved. I uploaded a package recently and the docs have not been generated even after a day. Here is the link to the
candidate that I uploaded:
https://hackage.haskell.org/package/unicode-transforms-0. 2.1/candidate
Is there a place where I can see any debug logs or diagnostic messages to figure out what went wrong?
-harendra
On 16 November 2016 at 23:31, Gershom B wrote:
As an update -- the builder had a few other issues besides disk-space in terms of getting a better retry policy in place, and properly prioritizing new over older packages, etc. I think as of this morning they're finally all worked through and it seems to be slowly chugging through the backlog in a reasonable order, prioritizing new uploads.I'll be checking in on the logs daily for the time being to make sure it does't seem to be getting stuck.
If your docs haven't built and you're impatient (understandably so!), please don't bump your package version to kick the priority --
just put more metadata on the server we don't need. Instead, you can build and upload your own docs as in this sample shell script:
https://github.com/ekmett/lens/blob/master/scripts/hackage-docs.sh
Best, Gershom
On Fri, Nov 11, 2016 at 11:43 AM, Gershom B wrote:
Sorry — the builder for docs has been in rough shape and we’re working on it. In this case it got stuck due to disk space issues and monitoring didn’t catch it. Its now running again but doing some catching up. (And lots of stuff in the queue it failed to build due to other problems needs to be replaced there, which is a manual process at the moment if we don’t want the queue as a whole to just drown out any new things at all with backlog). It would be much better to expose more queues and have better monitoring (and real prioritization), and another volunteer to help ben out on this would be very welcome (please contact me if you’re interested — the code lives at https://github.com/haskell/ hackage-server/blob/master/ BuildClient.hs and there’s some uncommitted work on queuing as well).
—gerhsom
On November 8, 2016 at 8:37:19 PM, Patrick Pelletier ( code@funwithsoftware.org) wrote: > This is only my second time uploading a package to Hackage, so I don't > yet have a feel for how it's supposed to go. I uploaded > normalization-insensitive-2.0.0.1 about 24 hours ago: > > https://hackage.haskell.org/package/normalization- insensitive-2.0.0.1 > > Under "Status", it says "Docs pending". (And the module names are all > non-clickable.) Is it normal to take this long to build the docs? Is > there some way to find out where in the queue my job is? Is
> indication that something has gone wrong? How do I fix it? > > Thanks, > > --Patrick > > _______________________________________________ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to
On January 24, 2017 at 3:00:45 AM, Harendra Kumar ( harendra.kumar@gmail.com) wrote: package that'll this an post.
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Patrick Pelletier writes:
This is only my second time uploading a package to Hackage, so I don't yet have a feel for how it's supposed to go. I uploaded normalization-insensitive-2.0.0.1 about 24 hours ago:
https://hackage.haskell.org/package/normalization-insensitive-2.0.0.1
Under "Status", it says "Docs pending". (And the module names are all non-clickable.) Is it normal to take this long to build the docs? Is there some way to find out where in the queue my job is? Is this an indication that something has gone wrong? How do I fix it?
Should it be normal? No. Has it been normal in the past? Sadly yes. In fact, due to some unfortunately interactions with our CDN it was not uncommon for builds to silently fail and never be reattempted. However, we have been slowly plugging away at the builder to try to sort out the common failure modes, including this one. One of the challenges has been working out how to retry the builds that have previously failed (of which there are many) without harming build latencies for new uploads. The problem is that there are two classes of failed documentation builds, a) Expected, reproducible failures (e.g. the builder environment missing native library dependencies). b) Various sporatic failures such as the CDN issue noted above While ideally we would like to rebuild only those packages falling in class (b), it's not so easy to accomplish this. Consequently I instead opted to retry all failed builds (over 17000 packages) and queue the work in such a way to prioritize new uploads. Unfortunately, my previous attempt at fair scheduling wasn't quite behaving as expected. However, this morning I took another look at this and things should now be back to normal. Let me know if you still see issues. Cheers, - Ben
participants (12)
-
Adam Bergmark
-
Ben Gamari
-
Chris Smith
-
Francesco Ariis
-
Gershom B
-
Harendra Kumar
-
Herbert Valerio Riedel
-
Imants Cekusins
-
Michael Snoyman
-
Patrick Pelletier
-
Saurabh Nanda
-
Tom Ellis