Interleaving per-component hooks with build
Hi, I am looking for a way to do custom preprocessing for each component (of type `UnitId -> IO ()`) but this preprocessing depends on the dependencies of the component being already built. So imagine a Cabal file which defines a library and an exe, with the exe depending on the library. Then, I would need the following sequence of events to happen during `buildHook`: 1. myPreprocessing unitIdOfLibrary 2. standard build of library 3. myPreprocessing unitIdOfExe 4. standard build of exe How can I achieve that without copy-pasting large parts of the internals of `buildHook simpleUserHooks`? Thanks, Gergo
Hi, I've made some progress on this, but I still have some questions. My new idea is to use withAllComponentsInBuildOrder to do the preprocessing before handing it over to buildHook simpleUserHooks. So then the next missing piece of the puzzle is how to arrange for buildHook simpleUserHooks to only build a single component: ``` myBuildHook pkg localInfo userHooks buildFlags = do withAllComponentsInBuildOrder pkg localInfo $ \c clbi -> do localInfo <- return $ restrictLocalInfo c clbi localInfo buildFlags <- return $ restrictBuildFlags c buildFlags bi <- preprocessBuildInfo localInfo buildFlags c pkg <- return $ updateBuildInfo c bi pkg buildHook simpleUserHooks pkg localInfo userHooks buildFlags ``` All the magic is supposed to happen in `restrictLocalInfo` and `restrictBuildFlags` to restrict the `LocalInfo` and the `BuildFlags` so that `buildHook simpleUserHooks` will only build the single given component. My first idea was to edit the `componentEnableSpec` / `componentGraph` / `componentNameMap` to be singleton lists: ``` restrictLocalInfo :: Component -> ComponentLocalBuildInfo -> LocalBuildInfo -> LocalBuildInfo restrictLocalInfo c clbi localInfo = localInfo { componentEnabledSpec = OneComponentRequestedSpec $ componentName c , componentGraph = G.insert clbi G.empty , componentNameMap = M.singleton (componentName c) [clbi] } ``` If I try this, building fails if some components are explicitly requested: setup: Cannot process the executable 'compucolor2-cpu-screen' because this package was configured only to build lib. Re-run configure with the argument executable 'compucolor2-cpu-screen' So next idea was to also delete the `buildArgs`: ``` restrictBuildFlags :: Component -> BuildFlags -> BuildFlags restrictBuildFlags c buildFlags = buildFlags { buildArgs = [] } ``` This gets us over building the lib, but now the exes fail: Preprocessing executable 'compucolor2-cpu-textscreen' for compucolor2-0.1.0.. Building executable 'compucolor2-cpu-textscreen' for compucolor2-0.1.0.. <command line>: cannot satisfy -package-id compucolor2-0.1.0-1E5h9nTsLCR9N8r4sKLlg5 (use -v for more information) My other idea is to NOT change the LocalInfo, and instead only edit `buildArgs` to contain only the current component. I haven't managed to get this quite right either: * If I only put the current component's name (via `prettyShow . componentName`) in the `buildArgs` list, then that fails for libraries: `prettyShow . componentName` for the library component is `"lib"`, but it seems that Cabal actually expects `"lib:packagename"` as the argument. * If I hack it by hardcoding the addition of `"lib:mylibname"` to the `buildArgs`, then the library is built and re-built and re-built for each component again and again. Of course, these "builds" finish quickly because nothing has changed, but it's still a nuisance. * The whole point of this exercise is to have access to already built libraries when preprocessing executables. But it seems that that is not as simple as adding `absolutePackaBgeDBPaths $ withPackageDB localInfo` to the package DB path when building, since that directory doesn't have a pkgconf file for the just-built library. So how do I use just-built dependency libraries? Thanks, Gergo On Wed, 3 Mar 2021, ÉRDI Gergő wrote:
I am looking for a way to do custom preprocessing for each component (of type `UnitId -> IO ()`) but this preprocessing depends on the dependencies of the component being already built. So imagine a Cabal file which defines a library and an exe, with the exe depending on the library. Then, I would need the following sequence of events to happen during `buildHook`:
1. myPreprocessing unitIdOfLibrary 2. standard build of library 3. myPreprocessing unitIdOfExe 4. standard build of exe
How can I achieve that without copy-pasting large parts of the internals of `buildHook simpleUserHooks`?
Let's put this in more concrete terms. Here is a simple model of my Setup.hs: https://gist.github.com/gergoerdi/2cdc4d984ddf039d650f6a98f2508a96 Here, I hope to run `justBeforeBuilding` for each `Component` just before building them (the actual building is to be done by `buildHook simpleUserHooks`). I print the package DBs, and their contents, in `justBeforeBuilding`. With a simple package that contains a library and an exe that depends on the library, the output is the following: ``` !!! Processing component CLibName LMainLibName !!! At this point, the package DB paths are: /home/cactus/prog/clash/bugs/cabal-component-hook/.stack-work/install/x86_64-linux-tinfo6/82781a0829e0e0da301c1db4825858dea8980a6a982b679eb178870c6c0ec1ee/8.10.4/pkgdb package.cache.lock package.cache /home/cactus/sdk/stack/snapshots/x86_64-linux-tinfo6/82781a0829e0e0da301c1db4825858dea8980a6a982b679eb178870c6c0ec1ee/8.10.4/pkgdb (snip) Preprocessing library for cabal-component-hook-0.1.0.. Building library for cabal-component-hook-0.1.0.. !!! Processing component CExeName (UnqualComponentName "bar") !!! At this point, the package DB paths are: /home/cactus/prog/clash/bugs/cabal-component-hook/.stack-work/install/x86_64-linux-tinfo6/82781a0829e0e0da301c1db4825858dea8980a6a982b679eb178870c6c0ec1ee/8.10.4/pkgdb package.cache.lock package.cache /home/cactus/sdk/stack/snapshots/x86_64-linux-tinfo6/82781a0829e0e0da301c1db4825858dea8980a6a982b679eb178870c6c0ec1ee/8.10.4/pkgdb package.cache.lock (snip) Preprocessing library for cabal-component-hook-0.1.0.. Building library for cabal-component-hook-0.1.0.. Preprocessing executable 'bar' for cabal-component-hook-0.1.0.. Building executable 'bar' for cabal-component-hook-0.1.0.. ``` After the whole process finishes, I do see my library's file in the package DB: ``` /home/cactus/prog/clash/bugs/cabal-component-hook/.stack-work/install/x86_64-linux-tinfo6/82781a0829e0e0da301c1db4825858dea8980a6a982b679eb178870c6c0ec1ee/8.10.4/pkgdb ├── cabal-component-hook-0.1.0-8kPvy0LMfqKAbvXmH5zWaP.conf ├── package.cache └── package.cache.lock ``` So my problem is that the package DB returned by `absolutePackageDBPaths $ withPackageDB localInfo` doesn't contain the just-build library, even though (supposedly) the library has already been built by that point. And then there is this mysterious second occurrence of "Preprocessing library for cabal-component-hook-0.1.0..Building library for cabal-component-hook-0.1.0.." while building (supposedly) just the exe. What's going on here? How can I arrange for `justBeforeBuilding` to have access to dependency libraries? Thanks, Gergo On Sun, 14 Mar 2021, ÉRDI Gergő wrote:
Hi,
I've made some progress on this, but I still have some questions. My new idea is to use withAllComponentsInBuildOrder to do the preprocessing before handing it over to buildHook simpleUserHooks. So then the next missing piece of the puzzle is how to arrange for buildHook simpleUserHooks to only build a single component:
``` myBuildHook pkg localInfo userHooks buildFlags = do withAllComponentsInBuildOrder pkg localInfo $ \c clbi -> do localInfo <- return $ restrictLocalInfo c clbi localInfo buildFlags <- return $ restrictBuildFlags c buildFlags bi <- preprocessBuildInfo localInfo buildFlags c pkg <- return $ updateBuildInfo c bi pkg buildHook simpleUserHooks pkg localInfo userHooks buildFlags ```
All the magic is supposed to happen in `restrictLocalInfo` and `restrictBuildFlags` to restrict the `LocalInfo` and the `BuildFlags` so that `buildHook simpleUserHooks` will only build the single given component.
My first idea was to edit the `componentEnableSpec` / `componentGraph` / `componentNameMap` to be singleton lists:
``` restrictLocalInfo :: Component -> ComponentLocalBuildInfo -> LocalBuildInfo -> LocalBuildInfo restrictLocalInfo c clbi localInfo = localInfo { componentEnabledSpec = OneComponentRequestedSpec $ componentName c , componentGraph = G.insert clbi G.empty , componentNameMap = M.singleton (componentName c) [clbi] } ```
If I try this, building fails if some components are explicitly requested:
setup: Cannot process the executable 'compucolor2-cpu-screen' because this package was configured only to build lib. Re-run configure with the argument executable 'compucolor2-cpu-screen'
So next idea was to also delete the `buildArgs`:
``` restrictBuildFlags :: Component -> BuildFlags -> BuildFlags restrictBuildFlags c buildFlags = buildFlags { buildArgs = [] } ```
This gets us over building the lib, but now the exes fail:
Preprocessing executable 'compucolor2-cpu-textscreen' for compucolor2-0.1.0.. Building executable 'compucolor2-cpu-textscreen' for compucolor2-0.1.0.. <command line>: cannot satisfy -package-id compucolor2-0.1.0-1E5h9nTsLCR9N8r4sKLlg5 (use -v for more information)
My other idea is to NOT change the LocalInfo, and instead only edit `buildArgs` to contain only the current component. I haven't managed to get this quite right either:
* If I only put the current component's name (via `prettyShow . componentName`) in the `buildArgs` list, then that fails for libraries: `prettyShow . componentName` for the library component is `"lib"`, but it seems that Cabal actually expects `"lib:packagename"` as the argument.
* If I hack it by hardcoding the addition of `"lib:mylibname"` to the `buildArgs`, then the library is built and re-built and re-built for each component again and again. Of course, these "builds" finish quickly because nothing has changed, but it's still a nuisance.
* The whole point of this exercise is to have access to already built libraries when preprocessing executables. But it seems that that is not as simple as adding `absolutePackaBgeDBPaths $ withPackageDB localInfo` to the package DB path when building, since that directory doesn't have a pkgconf file for the just-built library. So how do I use just-built dependency libraries?
Thanks, Gergo
On Wed, 3 Mar 2021, ÉRDI Gergő wrote:
I am looking for a way to do custom preprocessing for each component (of type `UnitId -> IO ()`) but this preprocessing depends on the dependencies of the component being already built. So imagine a Cabal file which defines a library and an exe, with the exe depending on the library. Then, I would need the following sequence of events to happen during `buildHook`:
1. myPreprocessing unitIdOfLibrary 2. standard build of library 3. myPreprocessing unitIdOfExe 4. standard build of exe
How can I achieve that without copy-pasting large parts of the internals of `buildHook simpleUserHooks`?
-- .--= ULLA! =-----------------. \ http://gergo.erdi.hu \ `---= gergo@erdi.hu =-------' Marijuana... because your friends just aren't that funny
On Mon, 15 Mar 2021, ÉRDI Gergő wrote:
So my problem is that the package DB returned by `absolutePackageDBPaths $ withPackageDB localInfo` doesn't contain the just-build library, even though (supposedly) the library has already been built by that point.
I've figured out the solution to this one (after not receiving any replies on this thread for almost a month...): I was missing the so-called "internal package DB". For anyone interested, the following Haskell module defines `withComponentHook`, a build hook transformator that allows each component's `BuildInfo` to be edited just before building them: https://github.com/gergoerdi/clashilator/blob/c2561e8465ba235aa4836e3bfae35f... Illustrating its use is the following module: https://github.com/gergoerdi/clashilator/blob/c2561e8465ba235aa4836e3bfae35f... I'm opening a Cabal ticket to hopefully start a conversation on adding something similar to Cabal itself. G
participants (1)
-
ÉRDI Gergő