On Wed, 2008-12-10 at 15:30 +0100, Sean Leather wrote:

[Responding to an old (but still relevant) thread...]

I'm getting this error as well, both in 2.3.0 and 2.4.1.

Suppose that I wanted to generate documentation for everything in my Cabal package except for the modules with Template Haskell in them. Is it enough to add {-# OPTIONS_HADDOCK hide #-} to each of these modules? I tried, but it didn't pan out like I hoped. I'm still getting the same error. Is there something else I can do?

As I understand it, using hide like that just means there will be no documentation page for that module. However it still has to parse that module since things in it may be re-exported from other modules and have documentation inserted into the docs for those modules. For example the documentation for the Prelude works this way, it's mostly re-exported from internal hidden GHC.* modules. (It may occasionally be broken but that's the way it's supposed to work). Duncan

On Thu, 2008-12-11 at 11:29 +0100, Sean Leather wrote:

Now, I'm trying to use #ifndef __HADDOCK__ / #endif around those same modules, and I'm still running into the same internal Haddock error:

...

haddock: internal Haddock or GHC error: Maybe.fromJust: Nothing

I've been using "cabal haddock" to run haddock on my package. I also get the same error using haddock directly:

...

$ haddock -odir=tmp --debug --verbose --html Generics/EMGM.hs haddock: internal Haddock or GHC error: Maybe.fromJust: Nothing

Any other suggestions?

Sorry, you'll have to ask David. Have you filed a ticket for this in the haddock trac so that he doesn't forget? http://trac.haskell.org/haddock/ Duncan

On Thu, 2008-12-11 at 16:26 +0100, Sean Leather wrote:

The above is true, but then I ran it with --optghc=-D__HADDOCK__ and that fixed the problem. You see, I didn't know who was actually defining the __HADDOCK__ macro. Now, after looking at code for Haddock and Cabal, I realize it's Cabal.

Does Cabal 1.6.0.1 actually define this macro? I don't see it listed when I run cabal-install with --verbose=3, but I don't know if it's supposed to be printed.

For haddock-0.x, Cabal cpp's all the modules with -D__HADDOCK__ because the old haddock cannot parse all sorts of things. However for haddock-2.x it is important not to use -D__HADDOCK__ because the hacks that people added for haddock-0.x would make haddock-2.x fail. For example they'd simply omit entire declarations. While haddock-0.x didn't really care and parsed and renamed in a sloppy way, haddoxk-2.x is basically ghc and so if the module does not compile then it also cannot be processed via haddock-2.x. The proper solution is to make haddock not fail when it encounters TH code or whatever the original problem was. It can ignore it for now, it just has to not fail. Duncan

On Thu, 2008-12-11 at 18:20 +0100, Sean Leather wrote:

For haddock-0.x, Cabal cpp's all the modules with -D__HADDOCK__ because the old haddock cannot parse all sorts of things.

However for haddock-2.x it is important not to use -D__HADDOCK__ because the hacks that people added for haddock-0.x would make haddock-2.x fail. For example they'd simply omit entire declarations. While haddock-0.x didn't really care and parsed and renamed in a sloppy way, haddoxk-2.x is basically ghc and so if the module does not compile then it also cannot be processed via haddock-2.x.

That's quite a presumption there. I can certainly write a module that compiles and produces documentation for Haddock but that is different when compiled into binary form. Even without this particular problem, I can see that being potentially useful.

Sure, but it'd have to be a different cpp flag because the existing one is used for a different purpose.

The proper solution is to make haddock not fail when it encounters TH code or whatever the original problem was. It can ignore it for now, it just has to not fail.

That's certainly the ideal solution; however, it is not there right now.

I would be happy to work around it if I could, but I can't. As far as I can tell, I can't pass any flags to Haddock via the Cabal file. I would love to tell Hackage to run Haddock like so, "cabal haddock --haddock-option=--optghc=-D__HADDOCK__", but I don't know how.

This adventure has exposed a major problem with Cabal that I have run into several times. It supports a limited set of external tools (e.g. Haddock, Alex, Happy, etc.), but it supports them in a limited way. Primarily, it is not possible to pass options to these tools (at least) through the configure script.

Don't you mean the other way around? It's not possible to pass extra options to these tools via the .cabal file and it's only possible via configure --$prog-options=

Also, regarding transparency with Hackage: is it possible to determine exactly which tools (e.g. Haddock version) are being used on the server? I would like to see a list on the website of the exact operations that will be applied to my package once uploaded.

You'd have to ask Ross about what happens on the build server. With the new hackage-server design, it will be possible for authors to upload docs they have generated themselves when the generic doc builder fails for whatever reason. Duncan

On Thu, 2008-12-11 at 23:32 +0100, Sean Leather wrote:

> That's quite a presumption there. I can certainly write a module that > compiles and produces documentation for Haddock but that is different > when compiled into binary form. Even without this particular problem, > I can see that being potentially useful.

Sure, but it'd have to be a different cpp flag because the existing one is used for a different purpose.

But what purpose does __HADDOCK__ serve if not to inform a file of code that haddock is currently processing it?

To inform a file that haddock-0.x is currently processing it.

Perhaps even better would be a __HADDOCK_VERSION__ definition that allows for code to work through changes in the tool.

Yes, and files would have to actually test that version number. That's why we cannot just do -D__HADDOCK__=2, because existing modules just do: #ifdef __HADDOCK__ which means they would present broken code to haddock-2.

While CPP may not be the most type-safe tool in the world and definitely not ideal for text macro substitution, it does do this simple kind of job very well.

Sure. Let's see what David thinks. If he thinks is possible to fix these kinds of things where haddock is not covering the whole GHC AST (at least to the degree where it can ignore bits it does not understand). If that's not realistic in the short term then we probably should introduce some haddock version macro so that people can workaround it. Duncan

2008/12/12 Duncan Coutts :

Let's see what David thinks. If he thinks is possible to fix these kinds of things where haddock is not covering the whole GHC AST (at least to the degree where it can ignore bits it does not understand). If that's not realistic in the short term then we probably should introduce some haddock version macro so that people can workaround it.

Haddock already tries to filter out declarations that it can't (or shouldn't) handle. I think that works well, although it's possible it can be improved. If we find a case where more filtering is needed, it's an easy fix to make. In this case, we can't filter out the TH declarations (since the code might then not compile), but we can perhaps fix the root-cause in the short-term (I'm not sure about that, but I hope so). So let's wait with the flag until we find something that can't be filtered out and for which we don't have a short-term fix? David

I would be happy to work around it if I could, but I can't. As far as I can tell, I can't pass any flags to Haddock via the Cabal file. I would love to tell Hackage to run Haddock like so, "cabal haddock --haddock-option=--optghc=-D__HADDOCK__", but I don't know how.

Let's suppose that I do actually want to define __HADDOCK__ for my library. Can I do this with a user-defined hook using the Cabal library? main :: IO () main = defaultMainWithHooks hooks where hooks = simpleUserHooks { haddockHook = haddockHook' } -- Define CPP __HADDOCK__ macro when running haddock. haddockHook' p l h f = do putStrLn ("f=\"" ++ show f ++ "\"") putStrLn ("f'=\"" ++ show f' ++ "\"") haddock p l (hookedPreProcessors h) f' where -- The Haddock flag to pass a flag to GHC to define the macro. *define__HADDOCK__ = ("haddock",["--optghc=-D__HADDOCK__"])* -- Add the flag to the the other flags. f' = f `mappend` emptyHaddockFlags { haddockProgramArgs = [define__HADDOCK__] } I got the value for define__HADDOCK__ from passing the "--haddock-option=--optghc=-D__HADDOCK__" flag at the command line. If I run "cabal haddock --verbose=3", I get: f="HaddockFlags {haddockProgramPaths = [], haddockProgramArgs = [], haddockHoogle = Flag False, haddockHtmlLocation = NoFlag, haddockExecutables = Flag False, haddockInternal = Flag False, haddockCss = NoFlag, haddockHscolour = Flag False, haddockHscolourCss = NoFlag, haddockDistPref = Flag "dist", haddockVerbosity = Flag Deafening}" f'="HaddockFlags {haddockProgramPaths = [], haddockProgramArgs = * [("haddock",["--optghc=-D__HADDOCK__"])]*, haddockHoogle = Flag False, haddockHtmlLocation = NoFlag, haddockExecutables = Flag False, haddockInternal = Flag False, haddockCss = NoFlag, haddockHscolour = Flag False, haddockHscolourCss = NoFlag, haddockDistPref = Flag "dist", haddockVerbosity = Flag Deafening}" [...] ("/.../haddock",["--html", ... My flag shows up in f' but not in the actual call to haddock. Thus, it doesn't work. With "cabal haddock --verbose=3 --haddock-option=--optghc=-D__HADDOCK__", I get: f="HaddockFlags {haddockProgramPaths = [], haddockProgramArgs = [* ("haddock",["--optghc=-D__HADDOCK__"])]*, haddockHoogle = Flag False, haddockHtmlLocation = NoFlag, haddockExecutables = Flag False, haddockInternal = Flag False, haddockCss = NoFlag, haddockHscolour = Flag False, haddockHscolourCss = NoFlag, haddockDistPref = Flag "dist", haddockVerbosity = Flag Deafening}" f'="HaddockFlags {haddockProgramPaths = [], haddockProgramArgs = * [("haddock",["--optghc=-D__HADDOCK__"]),("haddock",["--optghc=-D__HADDOCK__"])] *, haddockHoogle = Flag False, haddockHtmlLocation = NoFlag, haddockExecutables = Flag False, haddockInternal = Flag False, haddockCss = NoFlag, haddockHscolour = Flag False, haddockHscolourCss = NoFlag, haddockDistPref = Flag "dist", haddockVerbosity = Flag Deafening}" [...] ("/.../haddock",[*"--optghc=-D__HADDOCK__"*,"--html", ... Now, the command-line flag shows up in all 3. And, of course, it works. Is there something wrong with my hook code? Regards, Sean

Call the original haddockHook with the updated flags rather than the haddock command.

-- Define CPP __HADDOCK__ macro when running haddock. haddockHook' p l h f = do putStrLn ("f=\"" ++ show f ++ "\"") putStrLn ("g=\"" ++ show g ++ "\"") *haddockHook* simpleUserHooks p l h g where -- The Haddock flag to pass a flag to GHC to define the macro. define__HADDOCK__ = ("haddock",["--optghc=-D__HADDOCK__"]) -- Add the flag to the the other flags. g = f `mappend` emptyHaddockFlags { haddockProgramArgs = [define__HADDOCK__] } No change in output. Regards, Sean

On Fri, 2008-12-12 at 01:01 +0000, Duncan Coutts wrote:

On Fri, 2008-12-12 at 01:01 +0100, Sean Leather wrote:

...

Call the original haddockHook with the updated flags rather than the haddock command.

...

No change in output.

Ah, sorry I misread the code. This works:

Or simpler: import Distribution.Simple import Distribution.Simple.LocalBuildInfo (LocalBuildInfo(withPrograms)) import Distribution.Simple.Program (userSpecifyArgs) main = defaultMainWithHooks simpleUserHooks { haddockHook = \pkg lbi h f -> let progs = userSpecifyArgs "haddock" ["--optghc=-D__HADDOCK__"] (withPrograms lbi) in haddockHook simpleUserHooks pkg lbi { withPrograms = progs } h f }

...

I would be happy to work around it if I could, but I can't. As far as I can tell, I can't pass any flags to Haddock via the Cabal file. I would love to tell Hackage to run Haddock like so, "cabal haddock --haddock-option=--optghc=-D__HADDOCK__", but I don't know how.

Let's suppose that I do actually want to define __HADDOCK__ for my library. Can I do this with a user-defined hook using the Cabal library?

You might want to define it conditionally, depending on version. Cabal supports package-version macros, and haddock installs both a package (the paths are wrong in the package description, but you can find the version number there) and an executable for itself, but the package version macros are not available in Setup.hs. Still, you might find something useful in the discussion for this ticket: Cabal should support Cabal-version-dependent Setup.hs http://hackage.haskell.org/trac/hackage/ticket/326 Claus

Still, you might find something useful in the discussion for this ticket:

Cabal should support Cabal-version-dependent Setup.hs http://hackage.haskell.org/trac/hackage/ticket/326

or, more directly: http://www.haskell.org/pipermail/cabal-devel/2008-August/003576.html so, if you add the haddock package to your build-depends, that might give you a MIN_VERSION_haddock(_,_,_) already, without setup fiddling - Duncan? Then again, haddock depends on ghc and specific versions of other packages, so you might not want to depend on haddock.. Looks like one of those frustrating corners of packaging - haddock wants to be up to date wrt ghc, so claims not to need a macro; but it isn't complete, ghc keeps developing, so the macro that shouldn't be needed in theory would be useful in practice - cabal supports package version macros, but they aren't available everywhere, due to the way they are implemented - tools like haddock aren't tracked by cabal - it would be nice if every tool executable also installed a tool package with version/ path information (similar to ghc-paths for ghc), as that would be tracked by cabal - package version dependent settings in .cabal/Setup.hs would be useful; apart from the special case of cabal-version-dependent code in Setup.hs, perhaps something like this, to set options depending on package availability if flag(haddock2) build-depends: haddock > 2 haddock-options: -D__HADDOCK2__ only that options fields are limited to a predefined set, not including haddock? Shouldn't options fields be available for every use of .cabal, like haddocking? Claus

...

I've been using "cabal haddock" to run haddock on my package. I also get the same error using haddock directly:

...

$ haddock -odir=tmp --debug --verbose --html Generics/EMGM.hs haddock: internal Haddock or GHC error: Maybe.fromJust: Nothing

Have you filed a ticket for this in the haddock trac so that he doesn't forget? http://trac.haskell.org/haddock/

There seemed to be perhaps one or two tickets related, but I wasn't sure if any exactly matched the issue, so I created a new one with a minimal testcase. http://trac.haskell.org/haddock/ticket/68 Regards, Sean

The fromJust error is a bug, of course, however, the underlying problem is a bit more difficult: Haddock doesn't generate any code, it only typechecks. If the code uses Template Haskell, however, the typechecker will have to run Haskell code and potentially this code will have to come from a module of the same package. If the code indeed comes from the same package, fixing the fromJust error will just lead to GHCi linker error, since Haddock didn't generate any code for these. Here are a couple of solutions and non-solutions: - Detect whether a package uses TH by looking at the ghc flags and the OPTIONS pragmas. This might work in many cases, but: - We cannot generate Bytecode for all modules, because unboxed tuples and foreign exports are not supported by GHCi. This drawback would only affect a few packages, though, and cpp magic could hide the tricky parts from Haddock. - The bigger issue is security. TH can run arbitrary IO actions so, by default, we should just fail, and only enable it if the user says so. - Skip modules using TH. Won't work, because other modules may depend on those modules. We also cannot know which modules may require TH if the flag is set globally. - Disallow TH. That'll require a way to use #ifdef's to show alternative code. I guess, that's the current workaround. - Do a best-effort Renaming pass and run the typechecker without trying to resolve Splices. Lot's of work and may result to incorrect documentation, so I'm not sure it's worth it. Any other? / Thomas 2008/12/12 Sean Leather :

...

...

...

I've been using "cabal haddock" to run haddock on my package. I also get the same error using haddock directly:

...

$ haddock -odir=tmp --debug --verbose --html Generics/EMGM.hs haddock: internal Haddock or GHC error: Maybe.fromJust: Nothing

Have you filed a ticket for this in the haddock trac so that he doesn't forget? http://trac.haskell.org/haddock/

There seemed to be perhaps one or two tickets related, but I wasn't sure if any exactly matched the issue, so I created a new one with a minimal testcase.

http://trac.haskell.org/haddock/ticket/68

And for anyone who later comes upon this thread seeking an answer to a similar problem, this is apparently a bug in the GHC API on which Haddock is dependent:

http://hackage.haskell.org/trac/ghc/ticket/2739

Regards, Sean

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- Push the envelope. Watch it bend.

2008/12/12 Sean Leather :

On Fri, Dec 12, 2008 at 13:22, Thomas Schilling wrote:

...

The fromJust error is a bug, of course, however, the underlying problem is a bit more difficult:

Haddock doesn't generate any code, it only typechecks. If the code uses Template Haskell, however, the typechecker will have to run Haskell code and potentially this code will have to come from a module of the same package. If the code indeed comes from the same package, fixing the fromJust error will just lead to GHCi linker error, since Haddock didn't generate any code for these.

Thanks for enlightening us, Thomas.

...

Here are a couple of solutions and non-solutions:

[...]

...

Any other?

What about the eventual (maybe never?) solution of collecting comments and types for Haddock after splicing in code? This is more long term, perhaps, but in the end ideal if it can be made to work.

This is what Haddock does, actually. It works when the spliced-in code comes from another package (although this causes another problem, currently), and we should make it work also for your case where the code comes from the same package. With some improvements to the GHC API (and Haddock), we should be able to fix this. David