[GHC] #16233: HIE file generation is inefficient

#16233: HIE file generation is inefficient -------------------------------------+------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.7 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Generating HIE files eats up a lot of memory and takes a long time. There are at least two big parts to this: * Every expression in the AST is desugared via `deSugarExpr` for the purpose of extracting a type. GHC ends up doing the same work multiple times for the same nodes. Ex: in order to desugar `HsApp _ fun arg`, GHC desugars both `fun` and `arg`, yet the HIE code doesn't cache that (so it ends up desugaring `fun` and `arg` at least once more). * There is a lot of stuff being written into these HIE files, making them quite big (even after compressing/deduplicating type information). One data point: the largest HIE file at time of writing is `Parser.hie`, which is 7.2MB, compared to just 58K for `Parser.hi`Q Writing large files is naturally going to take up memory and be slow. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16233 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16233: HIE file generation is inefficient -------------------------------------+------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.7 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #15320 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by harpocrates): * related: => #15320 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16233#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16233: HIE file generation is inefficient -------------------------------------+------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.7 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #15320 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by harpocrates): Since 5ed48d25decc9dec29659482644b136cff91606e, we avoid most of the expensive calls to `deSugarExpr`. For a handful of expressions (ex: `HsLam`), the type already exists in the extensions field, so we use that. For some other expressions that are prone to the quartic desugaring performance (eg. `HsApp`), we've just disabled getting their type information completely for now. The rest (ex. `HsVar`), we still perform the desugaring. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16233#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16233: HIE file generation is inefficient -------------------------------------+------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.7 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #15320 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by int-index): It seems like a great motivation to do something along the lines of the design in Phab:D5047 Simon, what do you think? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16233#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16233: HIE file generation is inefficient
-------------------------------------+-------------------------------------
Reporter: harpocrates | Owner: (none)
Type: bug | Status: new
Priority: normal | Milestone:
Component: Compiler | Version: 8.7
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture:
| Unknown/Multiple
Type of failure: None/Unknown | Test Case:
Blocked By: | Blocking:
Related Tickets: #15320 | Differential Rev(s):
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
participants (1)
-
GHC