Hi Ryan, hi Richard,

My (limited) understanding is that template-haskell is not reinstallable for the same reasons that base and ghc-prim aren't reinstallable: the GHC codebase directly wires in several definitions several functions, types, and data constructors from template-haskell. See THNames [1]. If one were to, say, use GHC 8.6 but install a different version of template-haskell than the one that came bundled with 8.6, then it's almost certain that lots of code in THNames would no longer work, since they could be referencing identifiers that no longer exist (if the new version of template-haskell doesn't have them).

Right, I'm mostly concerned about re-installing the *same* version again. For the motivation: template-haskell depends on pretty, deepseq and array. Let's assume there is some off-by-one issue in array that only affects my application right now. Now I'm going to patch my array package, but if my application depends on template-haskell, I end up with two different array packages in my dependency tree that are not identical. What I'd ideally like to do here is to re-install deepseq, pretty and template-haskell based on my fixed array package. Now I don't have two different array packeds in my dependencies anymore; it would however require me to be able to rebuild (the same version) of template-haskell.

I have to admit I don't have a strong grasp on what "reinstallable" implies. Does a package get the same hash after reinstalling? What could make a package not reinstallable? Why aren't packages reinstallable today? Why isn't ghc-prim reinstallable?

We can't re-install packages that depend on build-time values. The RTS and ghc-prim right now include files that are generated during the build process and they have no capability on their own to create those files, as such re-installing them is not possible right now.

My concern stems from the fact that ghc is interlinked with TH in at least two ways: - GHC imports definitions from template-haskell. But this is the same as the way GHC is involved with, say, `base`. - GHC also wires in some template-haskell definitions. This is the aspect I am worried about. Is `base` reinstallable? If so, then perhaps template-haskell could be, too.

Now especially with TH I might see an issue when *running/using* TH, as at that point the compiler and the produces code have to interact. That is we are compiling splices against a different template-haskell package than the compiler is built against. This is where I see *upgrading* (that is building a newer version of Template Haskell) could be an issue, but I feel I don't fully grasp why rebuilding the same version should pose an issue. If we go one step further and use iserv (-fexternal-interpreter) outright, then I think we could just rebuild iserv against the re-installed template-haskell, and would not even run into the above mentioned issue. Might this potentially even be allow us to upgrade template-haskell? Cheers, Moritz

The Note [Wirde-in packages] is relevant, and highlights how you need to compile and use new template-haskell, for an exercise having a chance to work. Importantly you have to compile new `template-haskell` with `-this-unit-id template-haskell`, and not `template-haskell-2.14.0.0-somehash`. For the reference: Certain packages are known to the compiler, in that we know about certain entities that reside in these packages, and the compiler needs to declare static Modules and Names that refer to these packages.  Hence the wired-in packages can't include version numbers in their package UnitId, since we don't want to bake the version numbers of these packages into GHC. So here's the plan.  Wired-in packages are still versioned as normal in the packages database, and you can still have multiple versions of them installed. To the user, everything looks normal. However, for each invocation of GHC, only a single instance of each wired-in package will be recognised (the desired one is selected via @-package@\/@-hide-package@), and GHC will internall pretend that it has the *unversioned* 'UnitId', including in .hi files and object file symbols. Unselected versions of wired-in packages will be ignored, as will any other package that depends directly or indirectly on it (much as if you had used @-ignore-package@). The affected packages are compiled with, e.g., @-this-unit-id base@, so that the symbols in the object files have the unversioned unit id in their name. Make sure you change 'Packages.findWiredInPackages' if you add an entry here. For `integer-gmp`/`integer-simple` we also change the base name to `integer-wired-in`, but this is fundamentally no different. See Note [The integer library] in PrelNames. - Oleg On 8.3.2019 16.05, Ryan Scott wrote:

If you're reinstalling the same template-haskell version, then I can't immediately think of anything that would go wrong. Then again, I'm not intimately familiar with all of the little details that might be relevant here.

Out of curiosity, are you asking this question because you've tried implementing this and failed somewhere? Or are you trying to figure out is there's some obvious problem with the idea that's been overlooked? In the absence of anyone chiming in with such an obvious problem, I'd be inclined to just try implementing it and seeing what happens. If it does break, that could provide some valuable insight.

Ryan S.

On Thu, Mar 7, 2019 at 10:12 PM Moritz Angermann mailto:moritz.angermann@gmail.com> wrote:

Hi Ryan, hi Richard,

> My (limited) understanding is that template-haskell is not reinstallable for the same reasons that base and ghc-prim aren't reinstallable: the GHC codebase directly wires in several definitions several functions, types, and data constructors from template-haskell. See THNames [1]. If one were to, say, use GHC 8.6 but install a different version of template-haskell than the one that came bundled with 8.6, then it's almost certain that lots of code in THNames would no longer work, since they could be referencing identifiers that no longer exist (if the new version of template-haskell doesn't have them).

Right, I'm mostly concerned about re-installing the *same* version again.  For the motivation: template-haskell depends on pretty, deepseq and array.  Let's assume there is some off-by-one issue in array that only affects my application right now.  Now I'm going to patch my array package, but if my application depends on template-haskell, I end up with two different array packages in my dependency tree that are not identical.  What I'd ideally like to do here is to re-install deepseq, pretty and template-haskell based on my fixed array package.  Now I don't have two different array packeds in my dependencies anymore; it would however require me to be able to rebuild (the same version) of template-haskell.

> I have to admit I don't have a strong grasp on what "reinstallable" implies. Does a package get the same hash after reinstalling? What could make a package not reinstallable? Why aren't packages reinstallable today? Why isn't ghc-prim reinstallable?

We can't re-install packages that depend on build-time values. The RTS and ghc-prim right now include files that are generated during the build process and they have no capability on their own to create those files, as such re-installing them is not possible right now.

> My concern stems from the fact that ghc is interlinked with TH in at least two ways: > - GHC imports definitions from template-haskell. But this is the same as the way GHC is involved with, say, `base`. > - GHC also wires in some template-haskell definitions. This is the aspect I am worried about. Is `base` reinstallable? If so, then perhaps template-haskell could be, too.

Now especially with TH I might see an issue when *running/using* TH, as at that point the compiler and the produces code have to interact.  That is we are compiling splices against a different template-haskell package than the compiler is built against.  This is where I see *upgrading* (that is building a newer version of Template Haskell) could be an issue, but I feel I don't fully grasp why rebuilding the same version should pose an issue.

If we go one step further and use iserv (-fexternal-interpreter) outright, then I think we could just rebuild iserv against the re-installed template-haskell, and would not even run into the above mentioned issue. Might this potentially even be allow us to upgrade template-haskell?

Cheers,  Moritz

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs