[GHC] #14711: Machine readable output of coverage
#14711: Machine readable output of coverage -------------------------------------+------------------------------------- Reporter: Koterpillar | Owner: (none) Type: feature | Status: new request | Priority: normal | Milestone: Component: Code Coverage | Version: 8.2.2 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- When integrating with a CI system, it might be useful to fail the build, alert or take other actions based on coverage percentage. To do that, hpc needs to output a summary coverage report in a machine readable format. For example, the current output: {{{ 38% expressions used (1779/4624) 57% boolean coverage (16/28) 56% guards (9/16), 6 always True, 1 always False 58% 'if' conditions (7/12), 2 always False, 3 unevaluated 100% qualifiers (0/0) 7% alternatives used (103/1410) 75% local declarations used (102/136) 40% top-level declarations used (185/457) }}} can be represented as: {{{ { expressions: { total: 4624, used: 1779, percentage: 0.38 }, boolean: { total: 28, used: 16, percentage: 0.57 }, <...> } }}} This output can be then parsed to decide whether to fail the build, produce a coverage graph over time, etc. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#14711: Machine readable output of coverage -------------------------------------+------------------------------------- Reporter: Koterpillar | Owner: (none) Type: feature request | Status: new Priority: normal | Milestone: Component: Code Coverage | Version: 8.2.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Indeed this sounds quite useful. Let me know if you want help taking a stab at this. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:1> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#14711: Machine readable output of coverage -------------------------------------+------------------------------------- Reporter: Koterpillar | Owner: (none) Type: feature request | Status: new Priority: normal | Milestone: Component: Code Coverage | Version: 8.2.2 Resolution: | Keywords: newcomers Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * keywords: => newcomers -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:2> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#14711: Machine readable output of coverage -------------------------------------+------------------------------------- Reporter: Koterpillar | Owner: (none) Type: feature request | Status: new Priority: normal | Milestone: Component: Code Coverage | Version: 8.2.2 Resolution: | Keywords: newcomers Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Koterpillar): I'll try to do this, reading through the newcomers documentation currently. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:3> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#14711: Machine readable output of coverage -------------------------------------+------------------------------------- Reporter: Koterpillar | Owner: (none) Type: feature request | Status: new Priority: normal | Milestone: Component: Code Coverage | Version: 8.2.2 Resolution: | Keywords: newcomers Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by jml): There's already an `--xml-output` flag for `report`: {{{ $ hpc help report Usage: hpc report [OPTION] .. <TIX_FILE> [<MODULE> [<MODULE> ..]] Output textual report about program coverage Options: --per-module show module level detail --decl-list show unused decls --exclude=[PACKAGE:][MODULE] exclude MODULE and/or PACKAGE --include=[PACKAGE:][MODULE] include MODULE and/or PACKAGE --srcdir=DIR path to source directory of .hs files multi-use of srcdir possible --hpcdir=DIR append sub-directory that contains .mix files default .hpc [rarely used] --reset-hpcdirs empty the list of hpcdir's [rarely used] --xml-output show output in XML --verbosity=[0-2] verbosity level, 0-2 default 1 $ hpc version hpc tools, version 0.67 $ hpc report $(stack path --stack-yaml=stack-8.2.yaml --local-hpc-root )/graphql-api/graphql-api-tests/graphql-api-tests.tix --hpcdir=$(stack path --stack-yaml=stack-8.2.yaml --dist-dir)/hpc/ --xml-output <?xml version="1.0" encoding="UTF-8"?> <coverage name="/Users/jml/src/graphql-api/.stack- work/install/x86_64-osx/lts-10.4/8.2.2/hpc/graphql-api/graphql-api-tests /graphql-api-tests.tix"> <summary> <exprs boxes="3774" count="2290"/> <booleans boxes="19" true="4" false="0" count="15"/> <guards boxes="13" true="4" false="0" count="9"/> <conditionals boxes="4" true="0" false="0" count="4"/> <qualifiers boxes="2" true="0" false="0" count="2"/> <alts boxes="342" count="168"/> <local boxes="84" count="69"/> <toplevel boxes="1051" count="366"/> </summary> </coverage> }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:4> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#14711: Machine readable output of coverage -------------------------------------+------------------------------------- Reporter: Koterpillar | Owner: (none) Type: feature request | Status: new Priority: normal | Milestone: Component: Code Coverage | Version: 8.2.2 Resolution: | Keywords: newcomers Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by potomak): Can we consider this task as invalid or does it require to include also a JSON representation of the coverage output? -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:5> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#14711: Machine readable output of coverage -------------------------------------+------------------------------------- Reporter: Koterpillar | Owner: (none) Type: feature request | Status: new Priority: normal | Milestone: Component: Code Coverage | Version: 8.2.2 Resolution: | Keywords: newcomers Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4522 Wiki Page: | -------------------------------------+------------------------------------- Changes (by jproyo): * differential: => Phab:D4522 Comment: Hi Team, I am a newcomer and i have been trying to add this functionality. I have committed a review to Phabricator D4522 but i am not getting build pass because since i added a new flag to `hpc report` output command help i had to change `libraries/hpc` submodule testsuite in order to get the test pass successfully. I have committed the review with arc diff as the wiki indicates but it is complaining because my submodule changes are not in the upstream. Obviously i didnt want to push my submodule changes to not affect master branch build until review is done. Apart from this i dont have push permissions. How can i commit a review to Phab which contains changes in some component, in this case hpc, and in a submodule either? Best, -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:6> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#14711: Machine readable output of coverage -------------------------------------+------------------------------------- Reporter: Koterpillar | Owner: jproyo Type: feature request | Status: new Priority: normal | Milestone: Component: Code Coverage | Version: 8.2.2 Resolution: | Keywords: newcomers Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4522 Wiki Page: | -------------------------------------+------------------------------------- Changes (by Phyx-): * owner: (none) => jproyo -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:7> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#14711: Machine readable output of coverage -------------------------------------+------------------------------------- Reporter: Koterpillar | Owner: jproyo Type: feature request | Status: patch Priority: normal | Milestone: Component: Code Coverage | Version: 8.2.2 Resolution: | Keywords: newcomers Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4522 Wiki Page: | Phab:4523 -------------------------------------+------------------------------------- Changes (by Phyx-): * status: new => patch * differential: Phab:D4522 => Phab:D4522 Phab:4523 -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:8> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#14711: Machine readable output of coverage -------------------------------------+------------------------------------- Reporter: Koterpillar | Owner: jproyo Type: feature request | Status: patch Priority: normal | Milestone: Component: Code Coverage | Version: 8.2.2 Resolution: | Keywords: newcomers Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4522 Wiki Page: | Phab:D4523 -------------------------------------+------------------------------------- Changes (by Phyx-): * differential: Phab:D4522 Phab:4523 => Phab:D4522 Phab:D4523 -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:9> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#14711: Machine readable output of coverage -------------------------------------+------------------------------------- Reporter: Koterpillar | Owner: jproyo Type: feature request | Status: patch Priority: normal | Milestone: Component: Code Coverage | Version: 8.2.2 Resolution: | Keywords: newcomers Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4522 Wiki Page: | Phab:D4523 Phab:D4524 -------------------------------------+------------------------------------- Changes (by jproyo): * differential: Phab:D4522 Phab:D4523 => Phab:D4522 Phab:D4523 Phab:D4524 -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:10> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#14711: Machine readable output of coverage -------------------------------------+------------------------------------- Reporter: Koterpillar | Owner: jproyo Type: feature request | Status: patch Priority: normal | Milestone: Component: Code Coverage | Version: 8.2.2 Resolution: | Keywords: newcomer Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4522 Wiki Page: | Phab:D4523 Phab:D4524 -------------------------------------+------------------------------------- Changes (by monoidal): * keywords: newcomers => newcomer -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:11> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
participants (1)
-
GHC