[GHC] #11725: Performance Regression from 7.8.3 to 7.10.3

#11725: Performance Regression from 7.8.3 to 7.10.3 -------------------------------------+------------------------------------- Reporter: dominic | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 Keywords: | Operating System: Linux Architecture: | Type of failure: Runtime Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Some time ago I wrote a little test program for random number generation. Under 7.8.3 I get {{{ Total time 9.03s ( 9.15s elapsed) }}} Under 7.10.3 I get {{{ Total time 24.773s ( 25.288s elapsed) }}} Now of course it could be the libraries that I am using rather than GHC itself so I have tried to make them as similar as possible. For 7.8.3 I have {{{ build-depends: base ==4.7.0.1, mtl ==2.1.3.1, primitive == 0.6, mwc-random == 0.13.3.2, vector == 0.10.12.3, random ==1.1, random-fu == 0.2.6.2, random-source == 0.3.0.6 }}} For 7.10.3 I have {{{ build-depends: base ==4.8.2.0, mtl ==2.2, primitive == 0.6, mwc-random == 0.13.3.2, vector == 0.10.12.3, random ==1.1, random-fu == 0.2.6.2, random-source == 0.3.0.6 }}} So the only differences are in mtl and base. I don’t seem to be able to coax cabal into using mtl-2.2 for 7.8.3 with all the other required libraries. {{{ dominic@ghcPerformance:~$ cabal install 'random-source ==0.3.0.6' 'random- fu ==0.2.6.2' 'random ==1.1' 'primitive ==0.6' 'mwc-random ==0.13.3.2' 'mtl ==2.2' 'vector ==0.10.12.3' --with-ghc=ghc-7.10.3 Resolving dependencies... All the requested packages are already installed: mtl-2.2 mwc-random-0.13.3.2 primitive-0.6 random-1.1 random-fu-0.2.6.2 random-source-0.3.0.6 vector-0.10.12.3 Use --reinstall if you want to reinstall anyway. dominic@ghcPerformance:~$ cabal install 'random-source ==0.3.0.6' 'random- fu ==0.2.6.2' 'random ==1.1' 'primitive ==0.6' 'mwc-random ==0.13.3.2' 'mtl ==2.2' 'vector ==0.10.12.3' --with-ghc=ghc-7.8.3 Resolving dependencies... cabal: Could not resolve dependencies: trying: random-source-0.3.0.6/installed-70e... (user goal) next goal: mtl (user goal) rejecting: mtl-2.2.1, 2.2.0.1 (global constraint requires ==2.2) rejecting: mtl-2.2/installed-cc5..., 2.2 (conflict: random-source => mtl==2.1.3.1/installed-8bc...) rejecting: mtl-2.1.3.1/installed-8bc..., 2.1.3.1, 2.1.2, 2.1.1, 2.1, 2.0.1.1, 2.0.1.0, 2.0.0.0, 1.1.1.1, 1.1.1.0, 1.1.0.2, 1.1.0.1, 1.1.0.0, 1.0 (global constraint requires ==2.2) Backjump limit reached (change with --max-backjumps). }}} Cabal seems to be telling me that random-source-0.3.0.6 is the problem but if I look at the constraints for that package here https://hackage.haskell.org/package/random-source then I see {{{ mtl (>=1 && <3) }}} I am not sure how to proceed from here. I’d like to solve this myself but I don’t want to start building versions of ghc to see which change caused the regression without first eliminating mtl. Any ideas would be gratefully received. {{{ {-# LANGUAGE TemplateHaskell #-} {-# LANGUAGE GADTs #-} {-# LANGUAGE FlexibleInstances #-} import Data.Random import Data.Random.Source import qualified System.Random.MWC as MWC import Control.Monad.Reader import Control.Monad.Primitive $(monadRandom [d| instance (PrimMonad m, s ~ PrimState m) => MonadRandom (ReaderT (MWC.Gen s) m) where getRandomWord16 = ask >>= lift . MWC.uniform getRandomWord32 = ask >>= lift . MWC.uniform getRandomWord64 = ask >>= lift . MWC.uniform |]) testUniform :: MonadRandom m => Int -> m [Double] testUniform n = replicateM (fromIntegral n) (sample stdUniform) n :: Int n = 10^7 main :: IO () main = do seed <- MWC.create xs <- runReaderT (testUniform n) seed print (sum xs / fromIntegral n) }}} This cabal file will build this on 7.8.3 {{{ name: PerfTest8 version: 0.1.0.0 homepage: TBD license: MIT author: Dominic Steinitz maintainer: idontgetoutmuch@gmail.com category: System build-type: Simple cabal-version: >=1.10 executable Random8 main-is: TestMwcViaRandomSource.hs build-depends: base ==4.7.0.1, mtl ==2.1.3.1, primitive == 0.6, mwc-random == 0.13.3.2, vector == 0.10.12.3, random ==1.1, random-fu == 0.2.6.2, random-source == 0.3.0.6 default-language: Haskell2010 }}} This cabal file will build this on 7.10.3 {{{ name: PerfTest10 version: 0.1.0.0 homepage: TBD license: MIT author: Dominic Steinitz maintainer: idontgetoutmuch@gmail.com category: System build-type: Simple cabal-version: >=1.10 executable Random10 main-is: TestMwcViaRandomSource.hs build-depends: base ==4.8.2.0, mtl ==2.2, primitive == 0.6, mwc-random == 0.13.3.2, vector == 0.10.12.3, random ==1.1, random-fu == 0.2.6.2, random-source == 0.3.0.6 default-language: Haskell2010 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11725 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11725: Performance Regression from 7.8.3 to 7.10.3 -------------------------------------+------------------------------------- Reporter: dominic | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 Resolution: | Keywords: Operating System: Linux | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dominic): It's entirely possible that this has been solved in 8.0.1 so I have built this compiler on OS X 10.11.3. At the moment I can't build one of the packages on which my example depends {{{ ~/Dropbox/Private/Random8 $ ~/Library/Haskell/ghc-7.10.3/lib/cabal- install-1.25.0.0/bin/cabal install flexible-defaults --constraint ="template-haskell==2.11.0.0" --with-ghc=/Users/dom/ghc/inplace/bin/ghc- stage2 ~/Library/Haskell/ghc-7.10.3/lib/cabal-install-1.25.0.0/bin/cabal install flexible-defaults --constraint="template-haskell==2.11.0.0" --with- ghc=/Users/dom/ghc/inplace/bin/ghc-stage2 Resolving dependencies... cabal: Entering directory '/var/folders/3p/m593dprn5snbjz6sc45c77f80000gn/T/cabal-tmp-99610 /flexible-defaults-0.0.0.2' Configuring flexible-defaults-0.0.0.2... Building flexible-defaults-0.0.0.2... Preprocessing library flexible-defaults-0.0.0.2... [1 of 3] Compiling Language.Haskell.TH.FlexibleDefaults.Solve ( src/Language/Haskell/TH/FlexibleDefaults/Solve.hs, dist/dist-sandbox- 59d6d2c7/build/Language/Haskell/TH/FlexibleDefaults/Solve.o ) src/Language/Haskell/TH/FlexibleDefaults/Solve.hs:13:1: warning: The import of ‘Data.Monoid’ is redundant except perhaps to import instances from ‘Data.Monoid’ To import instances alone, use: import Data.Monoid() [2 of 3] Compiling Language.Haskell.TH.FlexibleDefaults.DSL ( src/Language/Haskell/TH/FlexibleDefaults/DSL.hs, dist/dist-sandbox- 59d6d2c7/build/Language/Haskell/TH/FlexibleDefaults/DSL.o ) src/Language/Haskell/TH/FlexibleDefaults/DSL.hs:79:82: error: Not in scope: type constructor or class ‘InlineSpec’ src/Language/Haskell/TH/FlexibleDefaults/DSL.hs:90:32: error: Not in scope: type constructor or class ‘InlineSpec’ src/Language/Haskell/TH/FlexibleDefaults/DSL.hs:121:14: error: Not in scope: type constructor or class ‘InlineSpec’ cabal: Leaving directory '/var/folders/3p/m593dprn5snbjz6sc45c77f80000gn/T /cabal-tmp-99610/flexible-defaults-0.0.0.2' Failed to install flexible-defaults-0.0.0.2 cabal: Error: some packages failed to install: flexible-defaults-0.0.0.2 failed during the building phase. The exception was: ExitFailure 1 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11725#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11725: Performance Regression from 7.8.3 to 7.10.3 -------------------------------------+------------------------------------- Reporter: dominic | Owner: dominic Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 Resolution: | Keywords: Operating System: Linux | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by dominic): * owner: => dominic -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11725#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11725: Performance Regression from 7.8.3 to 7.10.3 -------------------------------------+------------------------------------- Reporter: dominic | Owner: dominic Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 Resolution: | Keywords: Operating System: Linux | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dominic): See also https://ghc.haskell.org/trac/ghc/ticket/6166 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11725#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11725: Performance Regression from 7.8.3 to 7.10.3 -------------------------------------+------------------------------------- Reporter: dominic | Owner: dominic Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 Resolution: | Keywords: Operating System: Linux | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): A 3x performance regression looks bad. If you can pin it down more precisely it would be a big help. Thanks! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11725#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11725: Performance Regression from 7.8.3 to 7.10.3 -------------------------------------+------------------------------------- Reporter: dominic | Owner: dominic Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 Resolution: | Keywords: Operating System: Linux | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dominic): Here's my proposed approach to pinning it down. 1. Build ghc-8.0.1-rc2 and run my program under it. If the regression is gone then I propose to do no further investigation. 2. git bisect the sources from 7.8.3 to 7.10.3 until I find the change that causes the regression. Does that sound reasonable? Is there any guidance on how to do (2) efficiently and effectively? At the moment I cannot get my sources to compile under ghc-8.0.1-rc2 so my first mini-step is to solve this. I should point out that this is somewhat tangential to my day job although we use RNGs extremely heavily in our application. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11725#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11725: Performance Regression from 7.8.3 to 7.10.3 -------------------------------------+------------------------------------- Reporter: dominic | Owner: dominic Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 Resolution: | Keywords: Operating System: Linux | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dominic): Good news! After a heroic battle fighting cabal, template-haskell, unnecessarily restrictive upper bounds and the ghc build system on OS X 10.11.3, in which the aforementioned almost won, I managed to get my test program to compile under ghc-8.0.1-rc2. Under ghc-7.10.2 (NB not comparable with the above as that was done on a different machine) I get {{{ Total time 12.764s ( 13.242s elapsed) }}} Under ghc-8.0.1-rc2 I get {{{ Total time 2.896s ( 3.290s elapsed) }}} So better even than 7.83. For info I ran my comparison of 7.8.3 against 7.10.3 on {{{ processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 63 model name : Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz stepping : 2 microcode : 0xffffffff cpu MHz : 2397.169 cache size : 30720 KB physical id : 0 siblings : 1 core id : 0 cpu cores : 1 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq ss\ se3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm fsgsbase bmi1 avx2 smep bmi2 erms xsaveopt bugs : bogomips : 4794.33 clflush size : 64 cache_alignment : 64 address sizes : 42 bits physical, 48 bits virtual power management: }}} and I ran my comparison of 7.10.3 againt 8.0.1-rc2 on {{{ Intel(R) Core(TM) i7-3720QM CPU @ 2.60GHz }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11725#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11725: Performance Regression from 7.8.3 to 7.10.3 -------------------------------------+------------------------------------- Reporter: dominic | Owner: dominic Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 Resolution: | Keywords: Operating System: Linux | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dominic): There seems little point in keeping this ticket open but how do we stop this sort of performance regression happening in the future? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11725#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11725: Performance Regression from 7.8.3 to 7.10.3 -------------------------------------+------------------------------------- Reporter: dominic | Owner: dominic Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 Resolution: | Keywords: Operating System: Linux | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): It's hard saying without knowing more precisely what the performance regression was due to. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11725#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11725: Performance Regression from 7.8.3 to 7.10.3 -------------------------------------+------------------------------------- Reporter: dominic | Owner: dominic Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 Resolution: | Keywords: Operating System: Linux | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dominic): I don't propose to do any archeology myself so feel free to close this. I refer you to https://mail.haskell.org/pipermail/haskell- cafe/2016-January/122660.html though. I may start a suite of my own performance tests as I would prefer not to be caught out again. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11725#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

I don't propose to do any archeology myself so feel free to close this. I refer you to https://mail.haskell.org/pipermail/haskell- cafe/2016-January/122660.html though. I may start a suite of my own
#11725: Performance Regression from 7.8.3 to 7.10.3 -------------------------------------+------------------------------------- Reporter: dominic | Owner: dominic Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 Resolution: | Keywords: Operating System: Linux | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Replying to [comment:9 dominic]: performance tests as I would prefer not to be caught out again. If you can come up with any tests that have minimal build dependencies (ideally none although boot libraries are acceptable) please contribute them to `nofib`. We are in bad need of better coverage. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11725#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11725: Performance Regression from 7.8.3 to 7.10.3 -------------------------------------+------------------------------------- Reporter: dominic | Owner: dominic Type: bug | Status: closed Priority: normal | Milestone: 8.0.1 Component: Compiler | Version: 7.10.3 Resolution: fixed | Keywords: Operating System: Linux | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => closed * resolution: => fixed * milestone: => 8.0.1 Comment: This seems to be one of many performance regressions in 7.10 which has mysteriously improved with 8.0. There have been a variety of performance improvements made in 8.0; it would be great to know which of these helped here. You, the reader, can help here by identifying which commit(s) are responsible for this improvement. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11725#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC