[GHC] #9557: Deriving instances is slow
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Keywords: | Operating System: Architecture: Unknown/Multiple | Unknown/Multiple Difficulty: Unknown | Type of failure: Blocked By: | None/Unknown Related Tickets: | Test Case: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Let's take [https://raw.githubusercontent.com/haskell-suite/haskell-src- exts/8e0153d33b66add96fd5606614ce7938d2029510/src/Language/Haskell/Exts/Annotated/Syntax.hs this file] (from haskell-src-exts) and build it with -O0. On my machine it takes 55s. If we remove deriving of all classes except Functor (which is the only one used by the module itself), the time will drop down to 4s. Different classes vary in their compile-time cost, but there's no single culprit. Among the costly ones are Generic, Data, Show, Ord. haskell-src-exts users are very annoyed by its long compile time, about 10 minutes, especially because we don't compile with -O0 by default. It would be nice to get this fixed. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Changes (by slyfox): * cc: slyfox (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Changes (by jstolarek): * cc: jan.stolarek@… (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: closed Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: duplicate | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: #8731 Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Changes (by carter): * status: new => closed * resolution: => duplicate * related: => #8731 Comment: I think this is likely a dupe of #8731 This is a problem that impacts many many projects directly and indirectly (and epecshows up when folks ) I'm marking it as a duplicate for now. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: closed Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: duplicate | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: #8731 Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by Feuerbach): carter: why do you think they are the same? There, the reason is the large number of field selectors. Here, I was able to check that removing some of the class derivations reduces the compile time significantly. I'm not convinced that fixing that issue will necessarily fix this one; they may (or may not) be inefficiencies in different parts of code. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: #8731 Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Changes (by Feuerbach): * status: closed => new * resolution: duplicate => -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: #8731 Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by rwbarton): Deriving a lot of instances for a lot of large types generates a lot of code. Language.Haskell.Exts.Annotated.Syntax desugared to a program with 126,396 terms and took 24.2 seconds to build (with -O0). By comparison, the longest (in terms of lines, roughly 2100) module Language.Haskell.Exts.Annotated.ExactPrint desugared to a program with only 12,181 terms and it took 2.7 seconds to build. So, I don't think there is anything wrong with the speed of deriving instances specifically. Now, it's possible that instance deriving could be made to generate smaller code than it currently does for some particular classes... but I didn't notice anything amiss scanning through the generated instances for Decl (with `-dsuppress-all -ddump-deriv -ddump-to-file`). So, I don't really think deriving is to blame here. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: #8731 Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by simonpj): It undoubtedly is a bit unexpected if simply adding `deriving( Show )` or whatever, which seems such a little thing, generates seriously large amounts of object code. If someone feels able to investigate I would suggest: * Find out if any particular class is to blame. How much of the code is generated by which classes? * Study the generated code to see if it could be abstracted, so that instead of lots of code, there were calls to some suitable shared functions. Obviously there is a performance/code-size issue to worry about, but my guess is that while `Eq` and `Ord` might be performance-critical, `Read` and `Show` are not. But it would require some thoughtful investigation. Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: #8731 Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by rwbarton): I performed the following list of modifications to the program and recorded the size in terms after desugaring after each step: ||=step=||=program size (terms)=|| ||original|| 126396|| ||remove everything but data decls|| 122877|| ||remove deriving Generic|| 106339|| ||remove deriving Data|| 78544|| ||remove deriving Traversable|| 70886|| ||remove deriving Foldable|| 59332|| ||remove deriving Functor|| 55041|| ||remove deriving Show|| 42848|| ||remove deriving Ord|| 11044|| ||remove deriving Eq|| 2244|| ||remove deriving Typeable|| 1140|| So, the cost of each class is ||=class=||=terms=|| ||Ord|| 31804|| ||Data|| 27795|| ||Generic|| 16538|| ||Show|| 12193|| ||Foldable|| 11554|| ||Eq|| 8800|| ||Traversable|| 7658|| ||Functor|| 4291|| ||Typeable|| 1104|| So no individual class is adding a particularly egregious amount of code. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: #8731 Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by nomeata): For those who want to play around with the module: You can fetch it from http://hackage.haskell.org/package/haskell-src- exts-1.16.0/src/src/Language/Haskell/Exts/Annotated/Syntax.hs and it is has no dependencies inside `haskell-src-exts`, so you can just download and compile it in isolation. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: #8731 Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by nomeata):
Study the generated code to see if it could be abstracted, so that instead of lots of code, there were calls to some suitable shared functions.
Looking at the `Ord` code (which seems to be the largest): * It seems that it could be made much smaller using `<>` (which is `EQ <> x = x` and `y <> _ = y` otherwise) instead of nested case expressions. I guess there would be a performance hit, though. * Do we really have to provide separate definitions for `compare` and `<` and `<=` and `>` and `>=`? Maybe using the default implementation for all methods but `compare` is good enough, and could bring code size down considerably. Overreaching spontaneous idea: Add a method `generalCompare :: r -> r -> r -> a -> a -> r` to the `Ord` class. Implement that in deriving clauses, and have the default implementations use that. (e.g. `(<) = generalCompare True False False`). Should be faster than using `compare` + pattern matching. OTOH. all constructors of `Comparing` are static values, so there is probably not much to win here. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: #8731 Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by nomeata): Obviously, people have considered my second suggest before, see `Note [Do not rely on compare]` in [source:ghc/compiler/typecheck/TcGenDeriv.lhs#L300]... -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: #8731 Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by Feuerbach): Is there an easy way to check what exactly takes up the time: generating the ast, renaming/typechecking it, generating the output code? Or do I have to compile ghc with profiling for that? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: #8731 Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Changes (by gidyn): * cc: gideon@… (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: #8731 | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by edsko): * cc: edsko (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: #8731 | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by gidyn): * cc: gideon@… (removed) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: #8731 | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by gidyn): * cc: gidyn (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: #8731 | Differential Revisions: -------------------------------------+------------------------------------- Changes (by thomie): * failure: None/Unknown => Compile-time performance bug -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: #8731 #10858 | Differential Revisions: -------------------------------------+------------------------------------- Changes (by nomeata): * related: #8731 => #8731 #10858 Comment: See #10858 for some ideas around the specific case of the `Ord` instance. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:18 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: #8731 #10858 | Differential Revisions: -------------------------------------+------------------------------------- Changes (by jstolarek): * cc: jan.stolarek@… (removed) * cc: jstolarek (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: #8731 #10858 | Differential Rev(s): -------------------------------------+------------------------------------- Changes (by hvr): * cc: hvr (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: deriving-perf Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: #8731 #10858 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by ezyang): * keywords: => deriving-perf -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: deriving-perf Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: #8731 #10858 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by erikd): * cc: erikd (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:22 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: deriving-perf Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: #8731 #10858 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * cc: RyanGlScott (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:23 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9557: Deriving instances is slow -------------------------------------+------------------------------------- Reporter: Feuerbach | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: deriving-perf Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: #8731 #10858 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by michalt): * cc: michalt (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9557#comment:24 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC