Following up on this. Would there be interest in improving HPC and adding these features? Ivan On Thu, 24 Oct 2024 at 10:30, Ivan Perez wrote:

Ryan -- thanks!

Café, GHC Devs -- is anyone actively working on HPC?

Thanks,

Ivan

PS. Putting David Binder in CC since he seems to have been one of the most active contributors as of lately.

On Thu, 24 Oct 2024 at 07:57, Ryan Scott wrote:

...

Hi Ivan,

Short answer: I'm not aware of a way to automate the process of excluding certain definitions from cabal-generated HPC coverage reports.

Long answer: there is a way to manually modify HPC coverage reports using the hpc overlay command. Sadly, most the existing documentation on this command [1] is pretty sparse, so the only way I was able to discover this was by reading the original HPC paper [2] (in that paper, `hpc overlay` is called `hpc-makemtix`) and by finding an example of how to use `hpc overlay` in GHC's test suite [3]. To translate that example into prose: suppose you have the following program (hpc001.hs):

main = print (const "Hello" "World")

Here, the expression "World" is never evaluated, so by default HPC will claim that there is no coverage for it. This is pretty silly, however, so we'd like to instruct HPC not to warn about "World". To do so, first compile the example above with coverage enabled and run it:

$ ghc -fhpc hpc001.hs $ ./hpc001

This should generate an hpc001.tix file. If you run `hpc markup hpc001.tix` and look at the resulting HTML report, you'll see that it warns about "World". Let's fix that. To do so, we have to define an HPC overlay file. The example given in GHC's test suite (sample_overlay.ovr) is:

module "Main" { inside "main" { tick "\"World\"" on line 1; } }

While I can't find a reference for the syntax that HPC overlay files use, this particular example is pretty self-explanatory: don't report about the expression "World" on line 1, which is found inside the `main` function in the `Main` module. In order to make use of this overlay file, we first have to convert it to a .tix file:

$ hpc overlay sample_overlay.ovr > sample_overlay1.tix

Then we have to combine it with our original .tix file:

$ hpc combine hpc_sample.tix sample_overlay1.tix > total1.tix

If we run `hpc markup total1.txt`, we now see that HPC no longer warns about "World", just as we'd hoped for.

That is where my understanding of this feature ends, however. In particular, I'm not aware of a better way to generate overlay files (other than to write them by hand, which still feels somewhat laborious), and I'm not aware of any deeper integration with cabal so that these overlay files would be picked up and automatically applied (e.g., for reporting coverage on Hackage). Given that the HPC documentation for `hpc overlay` [1] labels this as an experimental feature, I'd be surprised if such tooling or integration existed. I agree that it would be cool if did, however.

Best,

Ryan ----- [1] https://hpc-bin.readthedocs.io/en/latest/hpc.html#hpc-overlay-and-hpc-draft [2] https://dl.acm.org/doi/abs/10.1145/1291201.1291203 [3] https://gitlab.haskell.org/ghc/ghc/-/blob/e39c8c993c1da534c5893ca418d1fa4cbb...

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

On Sat, Dec 28, 2024 at 07:50:20PM -0500, Ben Gamari wrote:

Ivan Perez writes:

...

Following up on this.

Would there be interest in improving HPC and adding these features?

Indeed David Binder is currently the one thinking most about HPC. The general idea here sounds reasonable, although there is a some design work needed to bridge the gap between the idea and a supportable, user-facing feature.

Indeed. FWIW, this thread reminds me of a curiousity with HPC vs. case analysis involving "otherwise". case foo of X | ... -> ... | otherwise -> ... HPC always flags incomplete coverage of "otherwise", because it always evaluated True! :-) -- Viktor.

On Sun, Dec 29, 2024 at 12:13:35PM -0800, Ivan Perez wrote:

Wrt otherwise, yes, I also found that a bit confusing.

Well, it isn't that confusing, it is a boolean function that has never been tested well enough to observe the effect of it returning False. :-) It is slightly surprising when first observed, but clear enough why this was flagged. -- Viktor.

On Sun, Dec 29, 2024 at 01:32:48PM -0800, amindfv--- via Haskell-Cafe wrote:

On Mon, Dec 30, 2024 at 07:32:01AM +1100, Viktor Dukhovni wrote:

...

On Sun, Dec 29, 2024 at 12:13:35PM -0800, Ivan Perez wrote:

...

Wrt otherwise, yes, I also found that a bit confusing.

Well, it isn't that confusing, it is a boolean function that has never been tested well enough to observe the effect of it returning False. :-) It is slightly surprising when first observed, but clear enough why this was flagged.

It's clear enough why this would happen, but it's serving the compiler, not the user.

I don't entirely disagree, though I also learned something useful (that is perhaps after the fact obvious, but only with some experience) from "otherwise" being flagged, which is that HPC covers evaluatio of expressions, not just execution of lines of code, and that for expressions that are branch heads it also checks whether both paths are taken. ... So it was educational (the first time). One of course nowdays define: patten Else :: Bool pattern Else = True and use "Else" instead of otherwise, and then perhaps the impossible branch not taken would not be flagged?

Here's another annoying example:

main :: IO () main = do _ <- foo pure ()

The unit ("()") is marked as "never executed" and highlighted in yellow, and expression coverage falls below 100%. While that may literally be true, this falls - for the programmer just trying to get something done - under the category of "why am I looking at this?"

HPC is a great tool and a few human-aware improvements could really help quality of life (and HPC's adoption, imo).

I really like HPC. But sure, some twkeaks might go a long way. -- Viktor.