Hello Dan, https://ghc.haskell.org/trac/ghc/ticket/9265 I think summarizes the reason why we shifted to this. It's also important infrastructural groundwork for a more sophisticated module system in Haskell. Edward Excerpts from Dan Burton's message of 2015-05-01 11:54:05 -0700:

Can you give (or link to) the rationale for the internal representation change? What use cases (or new corner cases for ghc-7.10) justify the added complexity here?

-- Dan Burton

On Fri, May 1, 2015 at 10:09 AM, Edward Z. Yang wrote:

...

In GHC 7.10, we changed the internal representation of names to be based on package keys (base_XXXXXX) rather than package IDs (base-4.7.0.1), however, we forgot to update the Template Haskell API to track these changes. This lead to some bugs in TH code which was synthesizing names by using package name and version directly, e.g. https://ghc.haskell.org/trac/ghc/ticket/10279

We now propose the following changes to the TH API in order to track these changes:

1. Currently, a TH NameG contains a PkgName, defined as:

newtype PkgName = PkgName String

This is badly misleading, even in the old world order, since these needed version numbers as well. We propose that this be renamed to PkgKey:

newtype PkgKey = PkgKey String mkPackageKey :: String -> PackageKey mkPackageKey = PkgKey

2. Package keys are somewhat hard to synthesize, so we also offer an API for querying the package database of the GHC which is compiling your code for information about packages. So, we introduce a new abstract data type:

data Package packageKey :: Package -> PkgKey

and some functions for getting packages:

searchPackage :: String -- Package name -> String -- Version -> Q [Package]

reifyPackage :: PkgKey -> Q Package

We could add other functions (e.g., return all packages with a package name).

3. Commonly, a user wants to get the package key of the current package. Following Simon's suggestion, this will be done by augmenting ModuleInfo:

data ModuleInfo = ModuleInfo { mi_this_mod :: Module -- new , mi_imports :: [Module] }

We'll also add a function for accessing the module package key:

modulePackageKey :: Module -> PkgKey

And a convenience function for accessing the current module:

thisPackageKey :: Q PkgKey thisPackageKey = fmap (modulePackageKey . mi_this_mod) qReifyModule

thisPackage :: Q Package thisPackage = reifyPackage =<< thisPackageKey

Discussion period: 1 month

Thanks, Edward

(apologies to cc'd folks, I sent from my wrong email address) _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

Excerpts from Dan Burton's message of 2015-05-01 15:05:18 -0700:

#1 looks like harmless bikeshedding. (Keep the old mkPkgName and pkgString and `type PkgName = PkgKey` as synonyms for a deprecation cycle or two?).

It's not that harmless: there really was a semantic change, and most users of mkPkgName did, in fact, break when GHC 7.10 was released (this change was prompted by two separate reports of TH code which was synthesizing these IDs breaking.)

As for #2, searchPackage struck me as odd at first. What could a TH user do with a list of packages that are all "the same version"? As specified, it's rather opaque. However, it occurs to me that if the Package data type has some information about, for example, the [Package] that it was built against, then perhaps some useful things could be done via TH with this information.

Yes. Admittedly, I have a hard time thinking of uses for this function, but it doesn't cost us too much to add. Edward

Being able to get all the packages that have a particular name sounds good! Note that if "-hide-package" / "-hide-all-packages" is used, I wouldn't want those packages in the results.

This is not entirely clear; you could imagine a package also coming with an 'exposed' bit, so you could filter for exposed/not exposed pacakges.

Like Dan, I'm a little less keen on #1. Here's why:

* PkgName is in the internal "Language.Haskell.TH.Syntax" module

* It isn't actually documented what it means, so it's meaning can be freely bent. There's no guarantee that you should be able to generate them.

* I couldn't find any examples of its usage that would be broken by this new semantics. I've done a search on github[1] to find usages of PkgName, and I only found one use[2] that uses PkgName which would be affected by this change. This example is in the TH lib itself, and so would hopefully be fixed by this change.

On the other hand, since it's rarely used, such an API breakage wouldn't be that impactful, so it's not that big of a deal.

Actually, this proposal was prompted by two separate reports of breakage: https://github.com/ekmett/lens/issues/496 https://ghc.haskell.org/trac/ghc/ticket/10279 So, like it or not, people seem to be depending on this API, which means it's worth at least discussing a little when we change it :) Cheers, Edward

Hi Michael,

Being able to get all the packages that have a particular name sounds good! Note that if "-hide-package" / "-hide-all-packages" is used, I wouldn't want those packages in the results.

Same here.

Like Dan, I'm a little less keen on #1. Here's why:

[...]

* I couldn't find any examples of its usage that would be broken by this new semantics. I've done a search on github[1] to find usages of PkgName, and I only found one use[2] that uses PkgName which would be affected by this change. This example is in the TH lib itself, and so would hopefully be fixed by this change.

Did you find any uses where an existing PkgName is reused to construct a new name? I believe this would be an important data point for deciding whether renaming the constructor is a good idea or not. Basically I'm asking whether the majority of uses of PkgName "out in the wild" right now is correct or incorrect. Cheers, Bertram

I didn't search for that, since I was searching for explicit uses of PkgName. The results for NameG are pretty interesting, though:

https://github.com/search?l=haskell&q=NameG&type=Code&utf8=%E2%9C%93

It seems like most of the results are either `Lift`ing a PkgName or re-using a PkgName from another name. However, there are a fair number of uses which explicitly construct the name, though. In particular, there are a fair number of usages of this to generate references to the tuple constructors in GHC.Prim.

Interestingly, these usages of (mkPkgName "ghc-prim") don't have version numbers at all. This implies that the behavior is to select the version of the package which is currently being used.

Well, the reason they don't have version numbers is because ghc-prim (and a few other packages) are internally considered wired-in packages, so their package keys are always some specific, hard-coded value. There is no "implied behavior", it's just an implementation detail that this TH API happens to expose. (There is one concrete consequence of this, which is that it's not possible to have two versions of ghc-prim simultaneously in the same executable.) Prior to 7.10, this manifested as ghc-prim having PkgName "ghc-prim", but lens having PkgName "lens-0.3.2".

So, how about for this rare-ish case where there we're linking against two different versions of the foo package, and the TH manually generates a (PkgName "foo"), we generate an ambiguity error.

If we force users to search for and specify fully unambiguous package names then that pushes the boilerplate of package selection out to TH usage. It seems like pretty much everyone will want "select the version of the package that I'm using." That said, I'm also in favor of adding the function mentioned in #2, for those cases where the default isn't sufficient (I'm anticipating very few of these).

I looked at the results. https://github.com/bitemyapp/hackage-packages/blob/fd9649f426254c0581bd97678... This is just very wrong, they should clearly switch to thisPackageKey when we have it https://github.com/s9gf4ult/letqq/blob/472d665cf64ec60b5f02f200c2b4c54fc6580... I'm not really sure what this is supposed to be, besides some meta-meta-programming library. It's clearly prototype code and I don't think they will mind if it stops working. https://github.com/suhailshergill/liboleg/blob/57673d01c66ab9f284579a40aed05... https://github.com/bitemyapp/hackage-packages/blob/fd9649f426254c0581bd97678... So, this piece of code is something that really ought to be in the standard library, if it isn't already. Basically, the problem is that when you quote an identifier, e.g. [| foo |], you get the *un-renamed* syntax (a variable foo) rather than the renamed syntax (somepkg:Data.Foo.foo). This is very useful, too useful to be in another library. https://github.com/andrep/hoc/blob/bfd0391bf0dab4bda5d6a5f7845fab19f8e4b9a9/... https://github.com/mokus0/hoc/blob/b6fa3906b8e1e61bed0435a8d497a132e5905227/... Wouldn't be broken. https://github.com/DavidAlphaFox/ghc/blob/6a5d9fa147b8abc370159d87e9c3dac871... This (and the next page of results) are from the template-haskell library. We can fix these ourselves when we make these changes. To sum up, I still think we should rename, and I think we should also add some functionality to the standard library for getting the renamed version of a TH syntax tree. Edward

I didn't search for that, since I was searching for explicit uses of PkgName. The results for NameG are pretty interesting, though: https://github.com/search?l=haskell&q=NameG&type=Code&utf8=%E2%9C%93 It seems like most of the results are either `Lift`ing a PkgName or re-using a PkgName from another name. However, there are a fair number of uses which explicitly construct the name, though. In particular, there are a fair number of usages of this to generate references to the tuple constructors in GHC.Prim. Very good exercise! Looking for how an existing API is used (perhaps in a clumsy way, because of the inadequacies of the existing API) is a good guide to improving it. e.g. If tuple names are an issue, let’s provide a TH API for getting their names!! Simon From: Libraries [mailto:libraries-bounces@haskell.org] On Behalf Of Michael Sloan Sent: 02 May 2015 23:46 To: Bertram Felgenhauer Cc: Haskell Libraries Subject: Re: Template Haskell changes to names and package keys On Sat, May 2, 2015 at 2:38 AM, Bertram Felgenhauer mailto:bertram.felgenhauer@googlemail.com> wrote: Hi Michael,

Being able to get all the packages that have a particular name sounds good! Note that if "-hide-package" / "-hide-all-packages" is used, I wouldn't want those packages in the results.

Same here.

Like Dan, I'm a little less keen on #1. Here's why:

[...]

* I couldn't find any examples of its usage that would be broken by this new semantics. I've done a search on github[1] to find usages of PkgName, and I only found one use[2] that uses PkgName which would be affected by this change. This example is in the TH lib itself, and so would hopefully be fixed by this change.

Did you find any uses where an existing PkgName is reused to construct a new name? I believe this would be an important data point for deciding whether renaming the constructor is a good idea or not. Basically I'm asking whether the majority of uses of PkgName "out in the wild" right now is correct or incorrect. I didn't search for that, since I was searching for explicit uses of PkgName. The results for NameG are pretty interesting, though: https://github.com/search?l=haskell&q=NameG&type=Code&utf8=%E2%9C%93 It seems like most of the results are either `Lift`ing a PkgName or re-using a PkgName from another name. However, there are a fair number of uses which explicitly construct the name, though. In particular, there are a fair number of usages of this to generate references to the tuple constructors in GHC.Prim. Interestingly, these usages of (mkPkgName "ghc-prim") don't have version numbers at all. This implies that the behavior is to select the version of the package which is currently being used. Considering how much more convenient this is than manually looking up package names, I'm strongly in favor of keeping PkgName and its current behavior, and extending it to handle package keys. Ideally, also documenting the different formats it expects. In particular, "name", "name-version", and "name-version-key" (I'm guessing). The issue here is that we may be linking against two different packages which are named the same thing, right? This was always a possibility, and having package keys just makes it so that we need the ability to disambiguate packages beyond version numbers. So, how about for this rare-ish case where there we're linking against two different versions of the foo package, and the TH manually generates a (PkgName "foo"), we generate an ambiguity error. If we force users to search for and specify fully unambiguous package names then that pushes the boilerplate of package selection out to TH usage. It seems like pretty much everyone will want "select the version of the package that I'm using." That said, I'm also in favor of adding the function mentioned in #2, for those cases where the default isn't sufficient (I'm anticipating very few of these). -Michael Cheers, Bertram _______________________________________________ Libraries mailing list Libraries@haskell.orgmailto:Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

On Tue, May 5, 2015 at 10:50 AM, Edward Z. Yang wrote:

...

Very good exercise! Looking for how an existing API is used (perhaps in a clumsy way, because of the inadequacies of the existing API) is a good guide to improving it.

e.g. If tuple names are an issue, let’s provide a TH API for getting

Excerpts from Simon Peyton Jones's message of 2015-05-05 01:15:50 -0700: their names!!

Hello Simon,

The right and proper way of getting a tuple name (well, constructor really, but that's the only reason people want names) should be:

[| (,) |]

But people sometimes don't want to use quotes, e.g. as in https://github.com/ekmett/lens/issues/496 where they want to work with stage1 GHC.

So in this case, https://ghc.haskell.org/trac/ghc/ticket/10382 (making quotes work with stage1 GHC) will help a lot.

Language.Haskell.TH already exports tupleTypeName tupleDataName

Hello Richard, Thanks for the comments. I've created a wiki page here: https://ghc.haskell.org/trac/ghc/wiki/TemplateHaskell/PackageKeyChanges

packageVersionString :: Package -> String -- maybe use a newtype here? or the proper Cabal datatype? packageName :: Package -> String -- use the old PkgName here? or is that confusing?

packageDependencies :: Package -> Q [Package] -- could be very useful in printing debugging information. Then, if you're writing a library that -- is sensitive to dependency versions, you can print this info out when you panic

OK, we can plan on adding this information.

I'm -1 on #3 if it will break code. You can already get current package information through the `qLocation` function.

This is true. However, qLocation returns a String. Should we change it to return a PkgKey? (Could break users of the information.) I don't know know if the name and version should get newtyped. If we do newtype them, should qLocation also be updated accordingly? Thanks, Edward