Well said points. 1) perhaps opening a ticket on ghc trac for your problem is a good next step. That way folks who are better at reading trac than email can help! 2) if the pattern synonyms branch gets merged in, you'll have to upstream the associated changes to haddock too right? On Monday, January 6, 2014, Mateusz Kowalczyk wrote:

On 28/12/13 16:53, Mateusz Kowalczyk wrote:

...

Greetings,

I'm trying to validate HEAD and I care that Haddock is built alongside it (so --no-haddock is not an option). I get the following errors listed at the bottom of this e-mail. How can I validate so that it all builds?

From what I understand, to validate I should: * Have a stable compiler in my PATH (7.6.3) * go to top level directory * run ‘sh validate’

Am I missing steps?

== Start post-build package check Timestamp 2013-12-28 05:00:55 UTC for /home/shana/.ghc/i386-linux-7.7.20131227/package.conf.d/package.cache Timestamp 2013-12-28 05:00:55 UTC for /home/shana/.ghc/i386-linux-7.7.20131227/package.conf.d (same as cache) using cache: /home/shana/.ghc/i386-linux-7.7.20131227/package.conf.d/package.cache Timestamp 2013-12-28 05:22:27 UTC for /home/shana/programming/ghc/inplace/lib/package.conf.d/package.cache Timestamp 2013-12-28 05:22:27 UTC for /home/shana/programming/ghc/inplace/lib/package.conf.d (same as cache) using cache: /home/shana/programming/ghc/inplace/lib/package.conf.d/package.cache There are problems in package xhtml-3000.2.1: dependency "base-4.7.0.0-578628bf142f9304d05ce5581b5f8d76" doesn't exist There are problems in package ghc-paths-0.1.0.9: dependency "base-4.7.0.0-578628bf142f9304d05ce5581b5f8d76" doesn't exist

The following packages are broken, either because they have a problem listed above, or because they depend on a broken package. xhtml-3000.2.1 ghc-paths-0.1.0.9

Ping. I need GHC to validate. Here's what I'm trying to achieve: as you might know, I worked on Haddock over summer, rewriting the whole parser, adding tests, fixing bugs, adding features. As Haddock ships with GHC however (and is technically a GHC HQ package), we can not merge it without making sure that GHC can build and validate with the changes.

This has been a problem for me and Simon Hengel for quite a while. We now have a branch with preliminary changes on https://github.com/sol/haddock/tree/new-parser . We can not even begin to try to merge the new features if the parser they are built upon is not merged. With the recent calls to push out a 7.8 release candidate, I think we're running out of time to get this in (or is it too late already?). It is not the first time we've been asking for help here!

Can someone say what are the steps I should take to get an OK from the GHC HQ that we can push new-parser onto master? If we miss 7.8, the next opportunity will be 7.10, because to get a new Haddock version you also need a new compiler, which people only get during stable releases. There's still a lot of work to be done on Haddock and I think it's understandable that I don't want to do work on what effectively is an ‘outdated version’. I'm fine with changes being rejected because they are deemed not good enough for some specific reason, but I'd hate the changes to not make it because I can't get a confirmation from GHC HQ that it's safe to do so.

Thanks, hope to hear from someone soon.

-- Mateusz K. _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org javascript:; http://www.haskell.org/mailman/listinfo/ghc-devs

i'd really recommend asking on #ghc and filing a ticket on trac preemptively. Different people reply better on different channels On Tue, Jan 7, 2014 at 12:23 AM, Mateusz Kowalczyk wrote:

On 07/01/14 04:17, Carter Schonwald wrote:

...

Well said points. 1) perhaps opening a ticket on ghc trac for your problem is a good next step. That way folks who are better at reading trac than email can help!

I'll do so tomorrow if I don't get any replies with tips.

...

2) if the pattern synonyms branch gets merged in, you'll have to upstream the associated changes to haddock too right?

It's not a show-stopper if Haddock can't document something. In fact there are many things it can't document already (GADT type constructors are an easy one). If someone writes the pattern synonym stuff for existing Haddock it's not a problem. The proposed changes from new-parser don't touch the parts that pattern synonyms would and if they did, it'd be easy to merge. Usually GHC HQ folk patch up Haddock when they change API so that it can still compile but everything extra tends to be a ‘if we can get it to document the new bleeding-edge feature, then great, if not, someone will make a ticket later’.

I actually attempted to make Haddock work with some extra stuff that it currently can't document but because it so heavily depends on GHC, I need my GHC tree validating for that too.

-- Mateusz K.

| Ping. I need GHC to validate. Here's what I'm trying to achieve: as you | might know, I worked on Haddock over summer, rewriting the whole parser, | adding tests, fixing bugs, adding features. As Haddock ships with GHC | however (and is technically a GHC HQ package), we can not merge it | without making sure that GHC can build and validate with the changes. | | This has been a problem for me and Simon Hengel for quite a while. We | now have a branch with preliminary changes on | https://github.com/sol/haddock/tree/new-parser . We can not even begin | to try to merge the new features if the parser they are built upon is | not merged. With the recent calls to push out a 7.8 release candidate, I | think we're running out of time to get this in (or is it too late | already?). It is not the first time we've been asking for help here! Actually I didn't know that you were asking to get something into 7.8. Haddock is maintained by David Waern and Simon Marlow, so the question of when to merge your changes into the main Haddock HEAD is up to them, not GHC HQ. We'll simply ship whatever Haddock we have when we cut the release candidate. (I know there is still some fuzz about when that will be; Austin is figuring that out now.) I'm copying David and Simon. Simon

| David stepped down and Simon Marlow has a long time ago too! It is now | Simon Hengel who maintains it. OK, well perhaps you can immediately push a change to haddock.cabal to reflect this? That's how we know. | Is it by now too late for 7.8? I'm afraid Simon H is away without much | access to technology until the 20th. Realistically that would push 7.8 RC to the end of Jan. Why would that be better than pushing to head just after the 7.8 release? Will 7.8 users see a big improvement if it was in? What do others think? S

Hi Mateusz, I remember your email and I believe I responded with the OK at the time - my impression was that it was ready to be merged and would shortly be done after that, but I didn't hear anything back about it. I apologize for my dropping the ball. As for your actual error - ghc-paths is only used in Haddock when it's not built in the GHC tree (as per the cabal file,) so I find it very suspicious that your package check is mentioning it at all (it's not mentioned anywhere else in any GHC sources.) Can you verify that it's there with `./inplace/bin/ghc-pkg list`? I'm not even sure how it could possibly get involved. Finally, can you be more specific about exactly how you tested these changes with your modified Haddock? I presume it was something like: $ ... clone ghc source ... $ cd ghc $ ... get extra stuff with ./sync-all ... $ cd utils/haddock $ ... use git to grab your code from github ... $ cd ../.. $ sh ./validate But I'd like to make sure I know exactly what's going on. I can try testing your branch later today. On Tue, Jan 7, 2014 at 4:29 AM, Mateusz Kowalczyk wrote:

On 07/01/14 10:20, Simon Peyton Jones wrote:

...

| David stepped down and Simon Marlow has a long time ago too! It is now | Simon Hengel who maintains it.

OK, well perhaps you can immediately push a change to haddock.cabal to reflect this? That's how we know.

I will try later but I think I don't have permissions. I can at best push to Simon's branch where he would periodically push to the GHC hosted repository (or perhaps it would get pulled from, I do not know).

...

| Is it by now too late for 7.8? I'm afraid Simon H is away without much | access to technology until the 20th.

Realistically that would push 7.8 RC to the end of Jan. Why would that be better than pushing to head just after the 7.8 release? Will 7.8 users see a big improvement if it was in? What do others think?

The changes were mostly there for user benefit. The markup can now be escaped much better. If we can validate what's on Simon's new-parser branch reasonably quickly, we might even be able to push in the new features: new mark up, nested paragraphs, better lists, headers… I'm trying to push for 7.8 because Haddock ships with GHC and 7.8 is the stable release that everyone will be using in couple of months time. If the changes don't get into 7.8, we'll have to wait for the next stable release for the users to benefit. Is this incorrect? I was always under the impression that the only Haddock releases we can reasonably make are with stable GHC releases. Of course, anyone can compile HEAD and generate the docs for their own viewing but for example, Hackage will run stable compiler and all the docs will still be using old Haddock.

I'd love to hear that I'm wrong about this and that Haddock releases separate from GHC are possible but I don't think that's the case.

...

S

-- Mateusz K. _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

Yes, the skipped tests are normal. The testsuite has a concept of tests being built a certain 'way' - for example, you might test a piece of code by making sure it works compiled with -threaded, non-threaded, profiling, the LLVM backend, or any combination of those, etc. So a single *test* gives rise to multiple *test cases*. When you run validate, it runs it in a 'fast' mode by default as opposed to the slow mode. The fast mode only runs a subset of the overall test cases - it runs the most basic tests per file, which generally gives a pretty good indication as to what is going on. Also, the performance failures you're seeing are (I speculate) due to out of date performance numbers. Sometimes these numbers go up or down just due to code churn, but they're sometimes finnicky, because they may depend on the exact time a major GC happens or something. So a small wibble can cause them to sometimes occasionally fail. In any case, these results seem to indicate your branch looks quite OK, so I can try to merge this soon, if you think it is actually complete and ready. On Tue, Jan 7, 2014 at 12:13 PM, Mateusz Kowalczyk wrote:

On 07/01/14 14:42, Austin Seipp wrote:

...

Hi Mateusz,

I remember your email and I believe I responded with the OK at the time - my impression was that it was ready to be merged and would shortly be done after that, but I didn't hear anything back about it. I apologize for my dropping the ball.

We contacted you because we thought it wouldn't be this much trouble to get an OK from the validate process. The code was technically more or less ready months ago, although Simon has been making some changes here and there.

...

As for your actual error - ghc-paths is only used in Haddock when it's not built in the GHC tree (as per the cabal file,) so I find it very suspicious that your package check is mentioning it at all (it's not mentioned anywhere else in any GHC sources.) Can you verify that it's there with `./inplace/bin/ghc-pkg list`? I'm not even sure how it could possibly get involved.

Finally, can you be more specific about exactly how you tested these changes with your modified Haddock? I presume it was something like:

I had ran it on a as-is tree so that I could compare the results from before and after I put my changes in. I had just ran validate yesterday again (after sync-all) and I no longer get this package failure!

...

$ ... clone ghc source ... $ cd ghc $ ... get extra stuff with ./sync-all ... $ cd utils/haddock $ ... use git to grab your code from github ... $ cd ../.. $ sh ./validate

As I mention, it was on unchanged tree but this is how I'll do it when testing the changes.

...

But I'd like to make sure I know exactly what's going on. I can try testing your branch later today.

I think the original issue is now gone. I do get 8 unexpected failures and about 11000+ skipped tests! Is this normal? Should I be filing bugs? Should I create a separate thread? Can someone look at my log? You can download it from [1], it's about 8MB. If GHC itself compiles the test information into some files you'd prefer, please let me know, I simply redirected all output from the validate script into a log.

[1]: http://fuuzetsu.co.uk/misc/validatelog

-- Mateusz K.

-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

It's worth mentioning that Gergo's Pattern Synonym work lightly touches Haddock as well, so perhaps it's worth ensuring nothing conflicts there as well: https://github.com/gergoerdi/ghc-haddock - I'm not sure which should be merged first (Gergo's patch has some validate failures that need to be fixed up, so I imagine yours might make it first.) On Tue, Jan 7, 2014 at 12:39 PM, Mateusz Kowalczyk wrote:

On 07/01/14 18:21, Austin Seipp wrote:

...

Yes, the skipped tests are normal. The testsuite has a concept of tests being built a certain 'way' - for example, you might test a piece of code by making sure it works compiled with -threaded, non-threaded, profiling, the LLVM backend, or any combination of those, etc. So a single *test* gives rise to multiple *test cases*.

When you run validate, it runs it in a 'fast' mode by default as opposed to the slow mode. The fast mode only runs a subset of the overall test cases - it runs the most basic tests per file, which generally gives a pretty good indication as to what is going on.

Also, the performance failures you're seeing are (I speculate) due to out of date performance numbers. Sometimes these numbers go up or down just due to code churn, but they're sometimes finnicky, because they may depend on the exact time a major GC happens or something. So a small wibble can cause them to sometimes occasionally fail.

In any case, these results seem to indicate your branch looks quite OK, so I can try to merge this soon, if you think it is actually complete and ready.

These are the numbers from the clean tree. I will now merge in my changes, validate again, run Haddock test suite and let you know how it went. If I see similar results, I'll assume it's fine.

I greatly appreciate the help I've been getting on this thread.

@Simon H. Do you think that the new features could be merged fairly soon too, if the basic parser stuff checks out? Does anything extra need doing?

-- Mateusz K.

-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

On 07/01/14 20:15, Ian Lynagh wrote:

On Tue, Jan 07, 2014 at 06:39:36PM +0000, Mateusz Kowalczyk wrote:

...

On 07/01/14 18:21, Austin Seipp wrote:

...

Also, the performance failures you're seeing are (I speculate) due to out of date performance numbers. Sometimes these numbers go up or down just due to code churn, but they're sometimes finnicky, because they may depend on the exact time a major GC happens or something. So a small wibble can cause them to sometimes occasionally fail.

These are the numbers from the clean tree.

The haddock perf numbers look pretty bad, especially the peak_megabytes_allocated:

=====> haddock.base(normal) 429 of 3855 [0, 0, 0] peak_megabytes_allocated value is too high: Expected peak_megabytes_allocated: 139 +/-1% Actual peak_megabytes_allocated: 180

=====> haddock.Cabal(normal) 430 of 3855 [0, 1, 0] peak_megabytes_allocated value is too high: Expected peak_megabytes_allocated: 89 +/-1% Actual peak_megabytes_allocated: 150

=====> haddock.compiler(normal) 431 of 3855 [0, 2, 0] max_bytes_used value is too high: Expected peak_megabytes_allocated: 663 +/-1% Actual peak_megabytes_allocated: 794

I think it would be worth working out what's going on before merging more haddock changes.

Thanks Ian

Hi Ian, Is there any guidance on how these tests are performed? More importantly, is there any log of how the performance changed over time? Is it Haddock's fault that it has become slower or is it the cause of GHC changes? PS: If there's no performance over time log, it might be worth introducing something! -- Mateusz K.

On 07/01/14 21:20, Austin Seipp wrote:

For the record and other people reading - after a quick discussion on IRC, it simply looks like the 32-bit peak_megabytes_allocated numbers for those tests probably weren't updated at the same time as the 64bit ones, leaving them out of date.

I have now validated GHC with the new Haddock stuff in place. You can see the new log at [1]. The end result is the same as validation on a tree without changes: same 8 tests failing. I have also built and ran Haddock's own tests with HEAD and they now all check out. The branch at [2] should now be ready to be merged into upstream Haddock. If someone could merge that in, that'd be great. This is the new parser which contains few bug fixes. We have more changes than this which include user-visible features and new documentation. I'll prepare and validate those for you tomorrow and bother you some more. Let me know if anything needs changing. Thanks! [1]: http://fuuzetsu.co.uk/misc/validateloghaddock [2]: https://github.com/sol/haddock/tree/new-parser -- Mateusz K.

On 08/01/14 07:46, Austin Seipp wrote:

Excellent, thank you. We should really fix the 32bit performance numbers too I think, based on what we discussed on IRC earlier. Would you like to submit a patch for that too please? You can find the numbers in testsuite/tests/perf/haddock/all.T.

I have no idea how to determine the new numbers. I could probably do it with some guidance. Is there a wiki link or something of the sort? Is there a special set up I need? I suppose you want it in a Trac ticket.

Also, is there any new documentation we should need for this? Is all the new stuff properly documented somewhere? Etc.

There are some slight semantics changes between the old and new parser. A good example is the ability to now escape things properly. In the past, ‘/foo\/bar/’ would actually only treat ‘foo’ as italics. As this should have been the original behaviour to begin with, the documentation is now actually correct. We were very careful to not make changes which would compromise old documentation. In fact, the first few commits include a parser which pretty much replicates the behaviour of the old one, with all the bugs and such! For the features (not in the new-parser branch, I will prepare the appropriate branch after I can validate it and give it a final look; the perf tests should probably be update after this is merged rather than before), I have updated Haddock's own documentation. It will be included in the branch. Anything that's no longer correct/relevant has been changed. Both Simon H and myself have access to the the Haddock documentation hosted on haskell.org so we can update that after the merges. I also hope to create a (web) tool which would allow one to write/paste in some Haskell to allow the user to compare between the Haddock versions as well as probably writing up a quick ‘migration’ guide but it's nothing that needs to be in the repository itself. After the features are merged in, it's all done from our side and anyone is free to hack on top of the changes, which is great. Perhaps I can get back to filling feature requests and bug fixes. Regarding PatternSynonyms, Erdi seems to be happy to make sure his changes to Haddock merge on top of what we send so we don't have to worry about that. So for now, hold tight until I can prepare the feature branch. I imagine you should be hearing about me with a link in about 12-16 hours. Thanks -- Mateusz K.

On 10/01/14 10:01, Mateusz Kowalczyk wrote:

Hi all,

I have now merged in the new parser and new features onto a single branch. I'm having some issues validating with HEAD at the moment (#8661, unrelated problem) but while I get that sorted out, someone might want to try validating with Haddock changes on their own platform.

The full branch is at [1]. I have squashed the changes to what I feel is the minimum number of commits until they completely stop making sense. It should apply cleanly on top of current Haddock master branch. The documentation is updated so you can read about what changed. Feel free to ask any questions.

I will post again once I can confirm that the branch validates for me without any new test failures.

Thanks for your patience.

[1]: https://github.com/Fuuzetsu/haddock/tree/new-features

This is just a simple follow up to say that the changes don't seem to break anything new on 32-bit Linux. I provide my validate logs before[1] and after[2] Haddock changes. Here's a word of warning: previously, when the mark-up wasn't 100% clear, we'd get a parse error and no documentation for the whole package. The new parser no longer does this and instead does its best to parse and present everything. This means that any Haddock parse failures should be reported as bugs. As you can see in [1], there were some parse failures in the past (look for ‘doc comment parse failed’) and they will now be rendered. This means the documentation might look bad in those places so it's probably worth while visiting those places and having a look. On an upside, at least we now have documentation for those packages. Validation was ran on commit 15a3de1288fe9d055f3dc92d554cb59b3528fa30 including #8661 fixes. Here's the relevant tail of the logs:

Unexpected results from: TEST="lazy-bs-alloc T1969 T3064 T4801 T3294 T5498 haddock.Cabal haddock.compiler haddock.base"

OVERALL SUMMARY for test run started at Fri Jan 10 12:45:39 2014 GMT 0:17:23 spent to go through 3861 total tests, which gave rise to 15072 test cases, of which 11547 were skipped

28 had missing libraries 3432 expected passes 56 expected failures

0 caused framework failures 0 unexpected passes 9 unexpected failures

Unexpected failures: deriving/should_fail T5498 [stderr mismatch] (normal) perf/compiler T1969 [stat too good] (normal) perf/compiler T3064 [stat not good enough] (normal) perf/compiler T3294 [stat not good enough] (normal) perf/compiler T4801 [stat not good enough] (normal) perf/haddock haddock.Cabal [stat not good enough] (normal) perf/haddock haddock.base [stat not good enough] (normal) perf/haddock haddock.compiler [stat not good enough] (normal) perf/should_run lazy-bs-alloc [stat too good] (normal)

gmake[2]: Leaving directory `/home/shana/programming/ghc/testsuite/tests' gmake[1]: Leaving directory `/home/shana/programming/ghc/testsuite/tests' == Start post-testsuite package check Timestamp 2014-01-10 12:45:36.897842164 UTC for /home/shana/programming/ghc/bindisttest/install dir/lib/ghc-7.7.20140109/package.conf.d/package.cache Timestamp 2014-01-10 12:45:36 UTC for /home/shana/programming/ghc/bindisttest/install dir/lib/ghc-7.7.20140109/package.conf.d (older than cache) using cache: /home/shana/programming/ghc/bindisttest/install dir/lib/ghc-7.7.20140109/package.conf.d/package.cache == End post-testsuite package check ------------------------------------------------------------------- Oops! Looks like you have some unexpected test results or framework failures. Please fix them before pushing/sending patches. -------------------------------------------------------------------

The failures are the same in both logs. Thanks! [1]: http://fuuzetsu.co.uk/misc/segfix [2]: http://fuuzetsu.co.uk/misc/segfixhaddock -- Mateusz K.