[GHC] #10952: Use IPids instead of package keys in HPC tix files

#10952: Use IPids instead of package keys in HPC tix files -------------------------------------+------------------------------------- Reporter: mgsloan | Owner: Type: bug | Status: new Priority: low | Milestone: Component: Code Coverage | Version: 7.10.2 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | -------------------------------------+------------------------------------- With GHC 7.10.2, HPC module names are stored in the format "package_key/Module.Name", rather than "package_id/Module.Name". In stack, we restrict the coverage results to just the library modules by using the "--include" flag. For GHC 7.10, this means I need to ask ghc- pkg what the package key is in order to figure out what to pass in for "-- include" (see this issue: https://github.com/commercialhaskell/stack/issues/785). I thought this was done correctly, but now I'm seeing some cases where coverage is reporting (0/0) due to the wrong package key being passed in... I think I need to run the ghc-pkg query with the package's ipid rather than name.. So, I need to have the correct ipid somewhere, in order to run ghc-pkg describe, parse its results to find out what the package key is, and then finally pass this in as a CLI argument. The hpc program's CLI is already woefully error-prone (see https://ghc.haskell.org/trac/ghc/ticket/10951), but this makes it plain hellish. Is there a better way to do what I need to do here? I think the right approach to resolving this is to put the full ipid in the HPC files. Then, the hpc program should be modified to allow "-- include" argument to take multiple package identification formats - ipid, package identifier, or package name. If the full package name was included, I could at least do some custom tix file munging based on just the package name. As things stand, the only way I can meaningfully operate on tix files is if I have the correct package key. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10952 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10952: Use IPids instead of package keys in HPC tix files -------------------------------------+------------------------------------- Reporter: mgsloan | Owner: Type: bug | Status: new Priority: low | Milestone: Component: Code Coverage | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): -------------------------------------+------------------------------------- Description changed by mgsloan: Old description:
With GHC 7.10.2, HPC module names are stored in the format "package_key/Module.Name", rather than "package_id/Module.Name". In stack, we restrict the coverage results to just the library modules by using the "--include" flag. For GHC 7.10, this means I need to ask ghc- pkg what the package key is in order to figure out what to pass in for " --include" (see this issue: https://github.com/commercialhaskell/stack/issues/785).
I thought this was done correctly, but now I'm seeing some cases where coverage is reporting (0/0) due to the wrong package key being passed in... I think I need to run the ghc-pkg query with the package's ipid rather than name.. So, I need to have the correct ipid somewhere, in order to run ghc-pkg describe, parse its results to find out what the package key is, and then finally pass this in as a CLI argument. The hpc program's CLI is already woefully error-prone (see https://ghc.haskell.org/trac/ghc/ticket/10951), but this makes it plain hellish.
Is there a better way to do what I need to do here?
I think the right approach to resolving this is to put the full ipid in the HPC files. Then, the hpc program should be modified to allow "-- include" argument to take multiple package identification formats - ipid, package identifier, or package name.
If the full package name was included, I could at least do some custom tix file munging based on just the package name. As things stand, the only way I can meaningfully operate on tix files is if I have the correct package key.
New description: With GHC 7.10.2, HPC module names are stored in the format "package_key/Module.Name", rather than "package_id/Module.Name". In stack, we restrict the coverage results to just the library modules by using the "--include" flag. For GHC 7.10, this means I need to ask ghc- pkg what the package key is in order to figure out what to pass in for "-- include" (see this issue: https://github.com/commercialhaskell/stack/issues/785). I thought this was done correctly, but now I'm seeing some cases where coverage is reporting (0/0) due to the wrong package key being passed in... I think I need to run the ghc-pkg query with the package's ipid rather than name.. So, I need to have the correct ipid somewhere, in order to run ghc-pkg describe, parse its results to find out what the package key is, and then finally pass this in as a CLI argument. The hpc program's CLI is already woefully error-prone (see https://ghc.haskell.org/trac/ghc/ticket/10951), but this makes it plain hellish. Is there a better way to do what I need to do here? I think the right approach to resolving this is to put the full ipid in the HPC files. Then, the hpc program should be modified to allow "-- include" argument to take multiple package identification formats - ipid, package identifier, or package name. This should also make it possible to remove the explici If the full package name was included, I could at least do some custom tix file munging based on just the package name. As things stand, the only way I can meaningfully operate on tix files is if I have the correct package key. There isn't enough information in the files to perform operations based on package name. -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10952#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10952: Use IPids instead of package keys in HPC tix files -------------------------------------+------------------------------------- Reporter: mgsloan | Owner: Type: bug | Status: new Priority: low | Milestone: Component: Code Coverage | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): -------------------------------------+------------------------------------- Changes (by thomie): * cc: ezyang (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10952#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10952: Use IPids instead of package keys in HPC tix files -------------------------------------+------------------------------------- Reporter: mgsloan | Owner: Type: bug | Status: new Priority: low | Milestone: Component: Code Coverage | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): -------------------------------------+------------------------------------- Comment (by ezyang): mgsloan, how much of this do you want to fix for the upcoming 8.0 release, as opposed to backports for 7.10? I think your life will get better when we unify IPIDs and PackageKeys, so that there will no longer be any distinction. If you're also willing to wait for 8.0, I am sure there are many affordances that could be added to hpc's API to make things better. Ultimately, I do think it is right for GHC to output Tix files based on the "package key" (which happened to just be the "package id" in the past, but only accidentally). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10952#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10952: Use IPids instead of package keys in HPC tix files -------------------------------------+------------------------------------- Reporter: mgsloan | Owner: Type: bug | Status: new Priority: low | Milestone: Component: Code Coverage | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): -------------------------------------+------------------------------------- Comment (by mgsloan): Hi ezyang! I'm glad to hear that IPIDs and PackageKeys will be unified. Does that mean that the tix module names will include the full package name + possibly the version number? I don't think there is much value in backporting things to 7.10, I just want to see this fixed in 8.0. So, if this will be fixed as a side effect of that unification, and the unification is sure to happen, feel free to close this ticket. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10952#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10952: Use IPids instead of package keys in HPC tix files -------------------------------------+------------------------------------- Reporter: mgsloan | Owner: Type: bug | Status: new Priority: low | Milestone: Component: Code Coverage | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Feuerbach): I've just run into this issue. Here's my problem. I want to merge tix files produced by running unit tests (via "stack test") and integration tests (via running actual executables that use the libraries being tested). As it happens, stack (via Cabal) runs tests by using an in-place version of the library, while the executable is using the properly installed version. This prevents me from merging the tix files and causes a lot of headache. If my understanding is correct, IPIDs by themselves won't help me; hpc will also need to learn to merge data from different installations of the same package. I do need this for 7.10 (not necessarily the official release; I can build ghc myself, although a release would be nice). I can also invest a bit of time into this if someone (Ed?) gave me directions. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10952#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10952: Use IPids instead of package keys in HPC tix files -------------------------------------+------------------------------------- Reporter: mgsloan | Owner: Type: bug | Status: new Priority: low | Milestone: Component: Code Coverage | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by ezyang): mgsloan: To answer your seven weeks ago question: tix module names will include the full package name + version number + hash (they will just be what you think of as the IPID). I am not sure this DIRECTLY solves your problem; you still need an IPID, but you can avoid going through ghc-pkg. (Unless you need to support backwards compatibility). Feuerbach: I don't quite understand your problem. Is the assigned package key from the inplace and the installed version actually different? I don't think should be true in 7.10; if it is that's some sort of bug. Is this giving you grief for 7.8 as well? I can advise on package key/IPID related issues, but I know very little about the hpc tool. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10952#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10952: Use IPids instead of package keys in HPC tix files -------------------------------------+------------------------------------- Reporter: mgsloan | Owner: Type: bug | Status: new Priority: low | Milestone: Component: Code Coverage | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Feuerbach): Is the assigned package key from the inplace and the installed version actually different? Yes it is. The two versions also have different package ids (which may be the cause; I'm not sure about what goes into a package key): {{{ --- ./lib/.stack- work/dist/x86_64-linux/Cabal-1.22.4.0/package.conf.inplace/presta- lib-1.0-inplace.conf 2015-11-24 09:32:13.518081305 +0200 +++ ./.stack-work/install/x86_64-linux/lts-3.3/7.10.2/pkgdb/presta- lib-1.0-1e1cf758eabdea7023a4cecbff20ce1f.conf 2015-11-24 09:29:31.266011461 +0200 @@ -1,7 +1,7 @@ name: presta-lib version: 1.0 -id: presta-lib-1.0-inplace -key: prest_1NxNAwYoYvk6w7VMhPEYyX +id: presta-lib-1.0-1e1cf758eabdea7023a4cecbff20ce1f +key: prest_8KWvpj6fKCvE66cwDmJopU }}} Is this giving you grief for 7.8 as well? Well, the issue being discussed wasn't present in ghc 7.8, so, no, it wasn't. (And we also don't use 7.8 at work anymore.) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10952#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10952: Use IPids instead of package keys in HPC tix files -------------------------------------+------------------------------------- Reporter: mgsloan | Owner: Type: bug | Status: new Priority: low | Milestone: Component: Code Coverage | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by mgsloan): ezyang: Thanks for the infos! It's fine if it doesn't directly solve the problem, as long as there's enough information to be able to munge the resulting tix files in reasonable ways. As is, we can't really tell which package a particular tix entry refers to unless we know about the specific package key it uses. @Feuerbach: Yes, that's a rather unfortunate Cabal bug - https://github.com/haskell/cabal/issues/2870 - which is fixed in the development version. I don't think stack can do anything about it. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10952#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10952: Use IPids instead of package keys in HPC tix files -------------------------------------+------------------------------------- Reporter: mgsloan | Owner: Type: bug | Status: new Priority: low | Milestone: Component: Code Coverage | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Feuerbach): Wow thanks Michael, that's great news! Building stack with Cabal HEAD is much easier than patching and building ghc. And I've verified that doing so indeed fixes my problem, so this ticket shouldn't affect me much anymore. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10952#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10952: Use IPids instead of package keys in HPC tix files -------------------------------------+------------------------------------- Reporter: mgsloan | Owner: Type: bug | Status: new Priority: low | Milestone: Component: Code Coverage | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by mgsloan): @Feuerbach Great! @ezyang Feel free to close this ticket if the changes you describe are sure to be in ghc 8. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10952#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC