cabal: question on compiling build prerequisites

This may be a silly question, but I can't seem to find the answer in the documentation. I would like to create a Cabal package where some of the source files use a custom pre-processor. The pre-processor is itself a Haskell program, let's say MyPreProc.hs in the package's root directory. I figured out how to activate the preprocessor, for example, cabal build --ghc-options="-F -pgmF ./MyPreProc" I also figured out how to achieve the same effect using hooks. However, there is one problem: how can I tell cabal to actually compile MyPreProc.hs before any of the commands using the preprocessor are run? The package's Setup.hs file is automatically compiled during "cabal configure". I would like MyPreProc.hs to be compiled at the same time as well - or at least some time before the proprocessor needs to be used. The .cabal file contains all kinds of related variables, such as "build-depends" (but only packages can be listed here, not programs to be compiled), "other-modules" (but these modules would already require the pre-processor, so the pre-processor should be compiled first), "build-tools" (but this only refers to external programs). My current solution is to make the pre-processor a shell script, rather than a Haskell program. But this is ugly and not portable (won't work in Windows, for example). To clarify: my custom preprocessor is local to this package. It has no independent value, so it would not make sense to publish it as a separate package just so that I can add it to build-depends. Is there a known solution to this problem? Thanks, -- Peter

On Fri, 6 Dec 2013, Peter Selinger wrote:
My current solution is to make the pre-processor a shell script, rather than a Haskell program. But this is ugly and not portable (won't work in Windows, for example).
To clarify: my custom preprocessor is local to this package. It has no independent value, so it would not make sense to publish it as a separate package just so that I can add it to build-depends.
Is there a known solution to this problem?
I have not tried this solution but I would guess that it is a good idea to put the preprocessor in a separate package anyway, since it might have its own build-depends.

Thanks for your reply! The preprocessor is essentially a 20-line Haskell program depending on Prelude. It's similar to the kind of ad-hoc programs that one might use within the "test" and "benchmark" targets. So putting it in a separate package would be overkill, I think. I wonder if I can somehow run the compiler from a post-configure hook. Hm. Looks like I would have to call Distribution.Simple.Build.buildExe with some magic arguments, to invoke the compiler. Maybe I will keep experimenting. If there's a better solution, please let me know! Thanks, -- Peter Henning Thielemann wrote:
On Fri, 6 Dec 2013, Peter Selinger wrote:
My current solution is to make the pre-processor a shell script, rather than a Haskell program. But this is ugly and not portable (won't work in Windows, for example).
To clarify: my custom preprocessor is local to this package. It has no independent value, so it would not make sense to publish it as a separate package just so that I can add it to build-depends.
Is there a known solution to this problem?
I have not tried this solution but I would guess that it is a good idea to put the preprocessor in a separate package anyway, since it might have its own build-depends.

On Sat, 7 Dec 2013, Peter Selinger wrote:
The preprocessor is essentially a 20-line Haskell program depending on Prelude. It's similar to the kind of ad-hoc programs that one might use within the "test" and "benchmark" targets. So putting it in a separate package would be overkill, I think.
Maybe it can be done without a preprocessor? In this case the discussion would better fit to haskell-cafe. For Cabal discussions there is also http://www.haskell.org/mailman/listinfo/cabal-devel

2013/12/7 Henning Thielemann
I have not tried this solution but I would guess that it is a good idea to put the preprocessor in a separate package anyway, since it might have its own build-depends.
I thought about a similar scenario recently, and I would like to hear if there is a clean cabal solution for this, too. Ripping out a helper program which has only a single use case (support for the "real" package) and which totally has to stay in sync with that "real" package would not be very nice. It would just be an artifact of a (missing?) cabal feature. Note that e.g. Template Haskell is not a solution for all needs (portability, XML=>Haskell translations, etc.), so support for some preprocessing step would be nice.

On Mon, 9 Dec 2013, Sven Panne wrote:
I thought about a similar scenario recently, and I would like to hear if there is a clean cabal solution for this, too. Ripping out a helper program which has only a single use case (support for the "real" package) and which totally has to stay in sync with that "real" package would not be very nice. It would just be an artifact of a (missing?) cabal feature.
The question is, how far should this feature reach? What if the preprocessor needs a preprocessor itself? E.g. a preprocessor might be built using a parser generator, then you need this parser generator as build-tool for the preprocessor build-tool.

On 2013-12-09 11:00, Henning Thielemann wrote:
On Mon, 9 Dec 2013, Sven Panne wrote:
I thought about a similar scenario recently, and I would like to hear if there is a clean cabal solution for this, too. Ripping out a helper program which has only a single use case (support for the "real" package) and which totally has to stay in sync with that "real" package would not be very nice. It would just be an artifact of a (missing?) cabal feature.
The question is, how far should this feature reach? What if the preprocessor needs a preprocessor itself? E.g. a preprocessor might be built using a parser generator, then you need this parser generator as build-tool for the preprocessor build-tool.
This just means that the build-tool definition should allow for recursion, I think. In fact, at least one build system that I know of supports this in a very general way via recursive project definitions. (The tool is the somewhat inaccurately named Simple Build Tool/SBT, but the non-simple nature of it doesn't have anything in particular to do with this feature.) Regards,

Henning Thielemann wrote:
On Fri, 6 Dec 2013, Peter Selinger wrote:
My current solution is to make the pre-processor a shell script, rather than a Haskell program. But this is ugly and not portable (won't work in Windows, for example).
To clarify: my custom preprocessor is local to this package. It has no independent value, so it would not make sense to publish it as a separate package just so that I can add it to build-depends.
Is there a known solution to this problem?
I have not tried this solution but I would guess that it is a good idea to put the preprocessor in a separate package anyway, since it might have its own build-depends.
I'd like to report on how I resolved these issues in the end. In a way, I followed your suggestion after all. But rather than putting just the preprocessor in a separate package, I put my entire build system in a separate package. Actually, this turned out rather nicely, because now other packages can use it too. The build system in question is now the package superdoc: http://hackage.haskell.org/package/superdoc Its purpose is to extend Haddock's markup language with new markup: for example, superscripts, boldface, images, and Unicode. It's implemented as a bunch of UserHooks for a Setup.hs script and a preprocessor. What makes this particular use of a preprocessor interesting is that the preprocessor is only used for building documentation, not for building the actual code. Because it is now in a separate package, other packages can use it transparently (the preprocessor will already be installed). An example of a package using superdoc is newsynth: http://hackage.haskell.org/package/newsynth. The only slight complication is bootstrapping: Superdoc's own documentation uses Superdoc markup. This creates a chicken-and-egg situation, where the preprocessor may not yet be installed by the time it needs to be used. It cannot be resolved with "build-tools" or similar settings, because the library is a prerequisite of the preprocessor, and the preprocessor is a prerequisite of the documentation of the library (but not the library itself). To resolve this, I bit the bullet and used shell scripts as wrappers for invoking the preprocessor via runhaskell. I provided both .sh and .bat versions of the shell script (as well as code to decide which one to run), so hopefully this is now fully portable. Tested under Linux and Windows. Thanks for all your feedback and suggestions! -- Peter

Hi Peter,
Peter Selinger
The .cabal file contains all kinds of related variables, such as "build-depends" (but only packages can be listed here, not programs to be compiled), "other-modules" (but these modules would already require the pre-processor, so the pre-processor should be compiled first), "build-tools" (but this only refers to external programs).
According to Duncan Coutts [1], 'build-tools' should also work for internal executables. [1] https://github.com/haskell/cabal/issues/1541#issuecomment-30155513

Hi Sven,
On Thu, Dec 12, 2013 at 11:47 AM, Sven Panne
2013/12/11 Mikhail Glushenkov
: According to Duncan Coutts [1], 'build-tools' should also work for internal executables.
Is there a Haskell package on Hackage which is successfully using it this way? I'm just curious to see the details...
I've created a small example: https://gist.github.com/23Skidoo/7930870 -- () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments

On Thu, 12 Dec 2013, Mikhail Glushenkov wrote:
Hi Sven,
On Thu, Dec 12, 2013 at 11:47 AM, Sven Panne
wrote: 2013/12/11 Mikhail Glushenkov
: According to Duncan Coutts [1], 'build-tools' should also work for internal executables.
Is there a Haskell package on Hackage which is successfully using it this way? I'm just curious to see the details...
I've created a small example: https://gist.github.com/23Skidoo/7930870
I have added a reference to: http://www.haskell.org/haskellwiki/Cabal/Developer-FAQ#Custom_preprocessors
participants (5)
-
Bardur Arantsson
-
Henning Thielemann
-
Mikhail Glushenkov
-
selinger@mathstat.dal.ca
-
Sven Panne