#15938: Hadrian's recompilation check is extremely slow -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.8.1 Component: Build System | Version: 8.6.2 (Hadrian) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Hmm, if I'm not mistake the problem gets worse the farther into the build you get. For instance, I let the build continue and interrupted it at what I would guess is the 50% mark (while building `libraries/pretty`). Restarting it takes now around 40 seconds before any output is produced. The profile looks somewhat similar to the previous profiles, {{{ Thu Nov 29 18:32 2018 Time and Allocation Profiling Report (Final) hadrian +RTS -I0 -qg -l -s -p -A32M -RTS --lint --directory /home/ben/ghc/ghc total time = 39.23 secs (39231 ticks @ 1000 us, 1 processor) total alloc = 51,375,453,360 bytes (excludes profiling overheads) COST CENTRE MODULE SRC %time %alloc ?== Development.Shake.Internal.FilePattern src/Development/Shake/Internal/FilePattern.hs:(226,1)-(230,34) 34.4 15.8 split Data.List.Extra src/Data/List/Extra.hs:(569,1)-(571,48) 18.0 24.2 unionWith General.TypeMap src/General/TypeMap.hs:34:1-97 17.6 27.4 foldr Data.ByteString.UTF8 Data/ByteString/UTF8.hs:(171,1)-(173,40) 4.7 7.5 getUserRuleInternal Development.Shake.Internal.Core.Rules src/Development/Shake/Internal/Core/Rules.hs:(62,1)-(75,48) 3.4 2.3 decode Data.ByteString.UTF8 Data/ByteString/UTF8.hs:(68,1)-(124,32) 2.4 10.1 catch_ Control.Exception.Extra src/Control/Exception/Extra.hs:106:1-32 2.0 1.7 &%> Development.Shake.Internal.Rules.Files src/Development/Shake/Internal/Rules/Files.hs:(163,1)-(180,42) 1.8 1.3 firstJustM Control.Monad.Extra src/Control/Monad/Extra.hs:(243,1)-(244,69) 1.7 1.3 defaultRuleFile Development.Shake.Internal.Rules.File src/Development/Shake/Internal/Rules/File.hs:(191,1)-(194,96) 1.2 0.0 hashPtrWithSalt Data.Hashable.Class Data/Hashable/Class.hs:(723,1)-(725,23) 1.1 0.0 }}} For what it's worth, a very significant fraction of the profile weight comes from: {{{ packageRules Rules src/Rules.hs:(88,1)-(123,46) 4045 0 0.0 0.0 62.5 58.3 compilePackage Rules.Compile src/Rules/Compile.hs:(13,1)-(36,77) 4046 0 0.0 0.0 61.4 57.4 %> Development.Shake.Internal.Rules.File src/Development/Shake/Internal/Rules/File.hs:566:1-143 4047 0 0.3 0.0 13.4 8.3 ?== Development.Shake.Internal.FilePattern src/Development/Shake/Internal/FilePattern.hs:(226,1)-(230,34) 4053 0 8.0 1.2 13.1 8.3 split Data.List.Extra src/Data/List/Extra.hs:(569,1)-(571,48) 4054 78480846 5.1 7.1 5.1 7.1 shakeOpenDatabase Development.Shake.Database src/Development/Shake/Database.hs:(58,1)-(69,24) 4051 0 0.0 0.0 0.0 0.0 open Development.Shake.Internal.Core.Run src/Development/Shake/Internal/Core/Run.hs:(73,1)-(81,23) 4052 0 0.0 0.0 0.0 0.0 &%> Development.Shake.Internal.Rules.Files src/Development/Shake/Internal/Rules/Files.hs:(163,1)-(180,42) 4055 0 1.8 1.3 48.1 49.1 ?== Development.Shake.Internal.FilePattern src/Development/Shake/Internal/FilePattern.hs:(226,1)-(230,34) 4062 2225801 25.6 14.2 37.8 30.3 split Data.List.Extra src/Data/List/Extra.hs:(569,1)-(571,48) 4063 184314560 12.1 16.0 12.1 16.0 isRelativePattern Development.Shake.Internal.FilePattern src/Development/Shake/Internal/FilePattern.hs:(152,1)-(155,27) 12746 2225801 0.0 0.0 0.0 0.0 fileNameToString Development.Shake.Internal.FileName src/Development/Shake/Internal/FileName.hs:38:1-55 12747 0 0.1 0.0 8.0 17.5 fileNameToByteString Development.Shake.Internal.FileName src/Development/Shake/Internal/FileName.hs:41:1-37 12748 2225801 0.0 0.0 0.0 0.0 toString Data.ByteString.UTF8 Data/ByteString/UTF8.hs:53:1-29 12750 2225801 0.1 0.0 7.9 17.5 foldr Data.ByteString.UTF8 Data/ByteString/UTF8.hs:(171,1)-(173,40) 12751 40082494 4.7 7.5 7.9 17.5 uncons Data.ByteString.UTF8 Data/ByteString/UTF8.hs:(166,1)-(167,38) 12752 40082494 0.8 0.0 3.1 10.0 decode Data.ByteString.UTF8 Data/ByteString/UTF8.hs:(68,1)-(124,32) 12753 40082494 2.3 10.0 2.3 10.0 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15938#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler