Integrate GHC-API with Cabal
Dear cafe, I am currently working on a project [1] which aims to index the code published on Stackage in order to provide a "code example" database. Basically, for each library, I want to generate a real-world example corpus for each exported symbol. Before generating this database, I need to retrieve the exported symbol of a package. So far, I have been gathering the dependencies and integrating them in the GHC pkg database using an external stack call, I then parse the cabal file and load the exposed modules using the GHC API and gather the exported symbols*. It works pretty reliably on the simple packages, however, I end up with some missing dynamic flags for some more advanced packages (missing c includes, c libraries, ASM flags, etc.). I then started to parse and load these missing attributes to GHC until I stepped back for a moment and realized I was re-implementing cabal build. I then looked for a way to "hook" my GHC API code in cabal build. This would allow cabal to both handle the dependencies gathering as well as setting up the correct GHC dyn-flags. The only resource I found was [2]. It's really clever, however, looks a bit hacky to me. Is there a better way to perform this kind of "hook"? [1] https://git.alternativebit.fr/NinjaTrappeur/Exhs [2] http://blog.ezyang.com/2017/02/how-to-integrate-ghc-api-programs-with-cabal/ * I am aware of the hoogle index generated by haddock. However, this index is missing when the haddock documentation of a package cannot be generated.
cabal-install package includes useful modules. With these modules it is possible to read the actual build info. These modules could be added to the Cabal API. What are the reasons cabal-install can not be added as a project dependency? Can some modules be moved from cabal-install to Cabal library? On 25 July 2018 at 11:33, Félix Baylac <felix@alternativebit.fr> wrote:
Dear cafe,
I am currently working on a project [1] which aims to index the code published on Stackage in order to provide a "code example" database. Basically, for each library, I want to generate a real-world example corpus for each exported symbol.
Before generating this database, I need to retrieve the exported symbol of a package. So far, I have been gathering the dependencies and integrating them in the GHC pkg database using an external stack call, I then parse the cabal file and load the exposed modules using the GHC API and gather the exported symbols*.
It works pretty reliably on the simple packages, however, I end up with some missing dynamic flags for some more advanced packages (missing c includes, c libraries, ASM flags, etc.). I then started to parse and load these missing attributes to GHC until I stepped back for a moment and realized I was re-implementing cabal build.
I then looked for a way to "hook" my GHC API code in cabal build. This would allow cabal to both handle the dependencies gathering as well as setting up the correct GHC dyn-flags. The only resource I found was [2]. It's really clever, however, looks a bit hacky to me.
Is there a better way to perform this kind of "hook"?
[1] https://git.alternativebit.fr/NinjaTrappeur/Exhs [2] http://blog.ezyang.com/2017/02/how-to-integrate-ghc-api- programs-with-cabal/
* I am aware of the hoogle index generated by haddock. However, this index is missing when the haddock documentation of a package cannot be generated. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
Hi, Thanks for the reply, I somehow missed your message.
cabal-install package includes useful modules. With these modules it is possible to read the actual build info. These modules could be added to the Cabal API.
As mentionned in another reply, there's even a library[1] retrieving this kind of informations.
What are the reasons cabal-install can not be added as a project dependency?
I can depend on it.
Can some modules be moved from cabal-install to Cabal library?
This is a really good question. Seems like exposing this kind of information could benefit some other people. I am not fimiliar with Cabal internals and have no idea whether this is possible nor the amount of work needed for this. I am trying to avoid digging yet another rabbit whole for now. [1]: https://hackage.haskell.org/package/cabal-helper
On 25 July 2018 at 11:33, Félix Baylac <felix@alternativebit.fr> wrote:
Dear cafe,
I am currently working on a project [1] which aims to index the code published on Stackage in order to provide a "code example" database. Basically, for each library, I want to generate a real-world example corpus for each exported symbol.
Before generating this database, I need to retrieve the exported symbol of a package. So far, I have been gathering the dependencies and integrating them in the GHC pkg database using an external stack call, I then parse the cabal file and load the exposed modules using the GHC API and gather the exported symbols*.
It works pretty reliably on the simple packages, however, I end up with some missing dynamic flags for some more advanced packages (missing c includes, c libraries, ASM flags, etc.). I then started to parse and load these missing attributes to GHC until I stepped back for a moment and realized I was re-implementing cabal build.
I then looked for a way to "hook" my GHC API code in cabal build. This would allow cabal to both handle the dependencies gathering as well as setting up the correct GHC dyn-flags. The only resource I found was [2]. It's really clever, however, looks a bit hacky to me.
Is there a better way to perform this kind of "hook"?
[1] https://git.alternativebit.fr/NinjaTrappeur/Exhs [2] http://blog.ezyang.com/2017/02/how-to-integrate-ghc-api- programs-with-cabal/
* I am aware of the hoogle index generated by haddock. However, this index is missing when the haddock documentation of a package cannot be generated. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
You can implement your tool as a source plugin and then it will work with any method of building. On Wed, 25 Jul 2018, 10:34 Félix Baylac, <felix@alternativebit.fr> wrote:
Dear cafe,
I am currently working on a project [1] which aims to index the code published on Stackage in order to provide a "code example" database. Basically, for each library, I want to generate a real-world example corpus for each exported symbol.
Before generating this database, I need to retrieve the exported symbol of a package. So far, I have been gathering the dependencies and integrating them in the GHC pkg database using an external stack call, I then parse the cabal file and load the exposed modules using the GHC API and gather the exported symbols*.
It works pretty reliably on the simple packages, however, I end up with some missing dynamic flags for some more advanced packages (missing c includes, c libraries, ASM flags, etc.). I then started to parse and load these missing attributes to GHC until I stepped back for a moment and realized I was re-implementing cabal build.
I then looked for a way to "hook" my GHC API code in cabal build. This would allow cabal to both handle the dependencies gathering as well as setting up the correct GHC dyn-flags. The only resource I found was [2]. It's really clever, however, looks a bit hacky to me.
Is there a better way to perform this kind of "hook"?
[1] https://git.alternativebit.fr/NinjaTrappeur/Exhs [2] http://blog.ezyang.com/2017/02/how-to-integrate-ghc-api-programs-with-cabal/
* I am aware of the hoogle index generated by haddock. However, this index is missing when the haddock documentation of a package cannot be generated. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
Hi, On Wed, Jul 25, 2018 at 10:33:45AM +0200, Félix Baylac wrote:
Before generating this database, I need to retrieve the exported symbol of a package. So far, I have been gathering the dependencies and integrating them in the GHC pkg database using an external stack call, I then parse the cabal file and load the exposed modules using the GHC API and gather the exported symbols*.
It works pretty reliably on the simple packages, however, I end up with some missing dynamic flags for some more advanced packages (missing c includes, c libraries, ASM flags, etc.). I then started to parse and load these missing attributes to GHC until I stepped back for a moment and realized I was re-implementing cabal build.
I think you might be interrested in one of my packages, [cabal-helper](https://hackage.haskell.org/package/cabal-helper). It's geard more towards editor tooling but I think it should work for your use case too. Basically you'd have to do `cabal install --only-dependencies && cabal configure && cabal build` for each package you want to inspect and then cabal-helpers API boils down to giving you a list of GHC flags/module names you need to pass into the GHC API to make it work. (The `cabal build` step is needed to run preprocessors like hsc2hs and compile C dependencies, but once cabal has taken care of that you can usually just load the package without problems.) Here's some example code you can use to get started (from our tests): https://github.com/DanielG/cabal-helper/blob/7fc3b9d468a4a2997a7fed63e378567... Quoting from the above, the main part is: ``` let qe = mkQueryEnv dir (dir </> "dist") cs <- runQuery qe $ components $ (,,) <$> entrypoints <.> ghcOptions forM cs $ \(ep, opts, cn) -> do ... ``` where `dir` is the directory containing the cabal file of the project you're trying to load into GHC. The do block gets called for each component in the cabal project with 1) `ep` a datatype containing (among other things) the list of modules you need to pass to GHC, 2) the GHC flags `opts` and 3) `ch` the component name. API docs are over here: https://hackage.haskell.org/package/cabal-helper-0.8.0.2/docs/Distribution-H.... They're probably not all that much use if you're not familliar with Cabal's inner workings though so feel free to ask me questions. --Daniel
participants (4)
-
Daniel Gröber -
Félix Baylac -
Imants Cekusins -
Matthew Pickering