[GHC] #11551: Get doctests into testsuite
#11551: Get doctests into testsuite -------------------------------------+------------------------------------- Reporter: lwm | Owner: Type: task | Status: new Priority: normal | Milestone: Component: Core | Version: 7.10.3 Libraries | Keywords: | Operating System: Unknown/Multiple documentation, doctest | Architecture: | Type of failure: Documentation Unknown/Multiple | bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- It seems like a bit of a shame not to have Mr.Orlitzky's lovely doctests not running with the testsuite. Or perhaps to at least have some sort of `make doctest`. If there is some interest to get this working, I'd like to take a look at submitting a patch. Here are the previous documentation patches I can find: https://phabricator.haskell.org/p/mjo/ I can see the following files that have doctests: {{{ ruby-2.2.1 ~/work/contributing/ghc (master) [12:35:44] lwm$ grep -R "=== __Examples__" libraries/base/ | uniq grep: libraries/base/Data/Maybe.hs:-- ==== __Examples__ libraries/base/Data/Either.hs:-- ==== __Examples__ libraries/base/Data/List.hs:-- ==== __Examples__ libraries/base/Data/Functor.hs:-- ==== __Examples__ libraries/base/Data/Bool.hs:-- ==== __Examples__ libraries/base/Data/Char.hs:-- ==== __Examples__ libraries/base/GHC/Unicode.hs:-- ==== __Examples__ libraries/base/Text/Printf.hs:-- ==== __Examples__ }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11551 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11551: Get doctests into testsuite -------------------------------------+------------------------------------- Reporter: lwm | Owner: Type: task | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.10.3 Resolution: | Keywords: | documentation, doctest Operating System: Unknown/Multiple | Architecture: Type of failure: Documentation | Unknown/Multiple bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by thomie): Go for it. Either create a separate `doctest` target in the testsuite (see `testsuite/Makefile`, `testsuite/mk/test.mk` and `testsuite/tests/Makefile`), and have `validate` call that. Or add it simply as a `run_command` test. Think about what should happen when the user doesn't have `doctest` installed. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11551#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11551: Get doctests into testsuite -------------------------------------+------------------------------------- Reporter: lwm | Owner: Type: task | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.10.3 Resolution: | Keywords: | documentation, doctest Operating System: Unknown/Multiple | Architecture: Type of failure: Documentation | Unknown/Multiple bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): lwm, is the trouble just integration into the build system or is there a deeper issue here? I quickly tried running `doctest` on some modules in `base` and very quickly ran into issues with multiple `base` versions (e.g. GHC desugared `do` syntax to imply a constraint of the `Monad` class provided by the compiler, not the `Monad` class provided by the `base` library which I was building. This suggests to me that even getting `doctest` running against `base` will take some effort by someone. Have you had different experiences, lwm? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11551#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11551: Get doctests into testsuite -------------------------------------+------------------------------------- Reporter: lwm | Owner: Type: task | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.10.3 Resolution: | Keywords: | documentation, doctest Operating System: Unknown/Multiple | Architecture: Type of failure: Documentation | Unknown/Multiple bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by lwm): bgamari, thanks for the input.
Have you had different experiences, lwm?
No, that's pretty much exactly what happened to me. I didn't get as deep as you did with regards to the desugared output. I have since been playing with trying to run doctest against base. I think I will get some logs to illustrate the problem and create an issue over at the doctest repo. It's definitely been blocking me. Otherwise, do you have some recommendations? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11551#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11551: Get doctests into testsuite -------------------------------------+------------------------------------- Reporter: lwm | Owner: Type: task | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.10.3 Resolution: | Keywords: | documentation, doctest Operating System: Unknown/Multiple | Architecture: Type of failure: Documentation | Unknown/Multiple bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by phadej): you have to compile doctests against the same compiler you are testing with. - A compiled ghc master - patched ghc-paths: https://github.com/simonmar/ghc-paths/pull/9 (not sure if actually necessary) - compiled doctest using just compiled ghc - `.../build/doctest/doctest libraries/base/Data/Maybe.hs` - profit {{{ Examples: 44 Tried: 44 Errors: 0 Failures: 0 }}} Maybe, Char and Either pass, Functor got: {{{ ### Failure in libraries/base/Data/Functor.hs:49: expression `show <$> Nothing' expected: "Nothing" but got: "" "\ESC[;1m<interactive>:31:6: \ESC[;1m\ESC[31merror:\ESC[;1m" " Ambiguous occurrence \8216<$>\8217" " It could refer to either \8216Data.Functor.<$>\8217," " defined at libraries/base/Data/Functor.hs:73:1" " or \8216Prelude.<$>\8217," " imported from \8216Prelude\8217 (and originally defined in \8216Data.Functor\8217)\ESC[0m" }}} nasty colorcodes! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11551#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11551: Get doctests into testsuite
-------------------------------------+-------------------------------------
Reporter: lwm | Owner:
Type: task | Status: new
Priority: normal | Milestone:
Component: Core Libraries | Version: 7.10.3
Resolution: | Keywords:
| documentation, doctest
Operating System: Unknown/Multiple | Architecture:
Type of failure: Documentation | Unknown/Multiple
bug | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s):
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#11551: Get doctests into testsuite -------------------------------------+------------------------------------- Reporter: lwm | Owner: Type: task | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.10.3 Resolution: | Keywords: | documentation, doctest Operating System: Unknown/Multiple | Architecture: Type of failure: Documentation | Unknown/Multiple bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by phadej): With https://gist.github.com/phadej/fdd74f4b4b40dc0a758ca6192cd66e83 I can successfully run the doctests in `base`. There's a hack, it picks files based on whether there is `$setup`, which IMHO is not so bad assumption. If someone could give a pointer how to integrate it with `validate`, I probably can do it. Yet, i'd like to have ability to run the doctests in standalone way, as editing docs / doctests shouldn't require rebuilds of the `base` etc. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11551#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11551: Get doctests into testsuite -------------------------------------+------------------------------------- Reporter: lwm | Owner: Type: task | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.10.3 Resolution: | Keywords: | documentation, doctest Operating System: Unknown/Multiple | Architecture: Type of failure: Documentation | Unknown/Multiple bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by lwm): Oh, that is awesome phadej, well done! Yes, would be nice for it to be stand alone from base. I'll try and get a doctest run on my local with your Gist ... -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11551#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC