#8144: Interface hashes include time stamp of included .h file when CPP is used -------------------------------------------------+------------------------- Reporter: nh2 | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time performance bug | Unknown/Multiple Test Case: | Difficulty: Blocking: | Unknown | Blocked By: | Related Tickets: -------------------------------------------------+------------------------- Changes (by akio): * cc: tkn.akio@… (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of included .h file when CPP is used -------------------------------------------------+------------------------- Reporter: nh2 | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time performance bug | Unknown/Multiple Test Case: | Difficulty: Blocking: | Unknown | Blocked By: | Related Tickets: -------------------------------------------------+------------------------- Comment (by simonpj): That certainly sounds wrong. Would anyone care to investigate? Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of included .h file when CPP is used -------------------------------------------------+------------------------- Reporter: nh2 | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time performance bug | Unknown/Multiple Test Case: | Difficulty: Blocking: | Unknown | Blocked By: | Related Tickets: -------------------------------------------------+------------------------- Comment (by nh2): ''UsageFile'' was introduced with ''addDependentFile'' in https://github.com/ghc/ghc/commit/b994313. The modification time being only (obtained) in second resolution is what triggers the inconsistent behaviour. Regarding that commit, I am also not sure how https://github.com/ghc/ghc/commit/b994313#L3R1283 works: {{{ checkModUsage _this_pkg UsageFile{ usg_file_path = file, usg_mtime = old_mtime } = do new_mtime <- liftIO $ getModificationTime file return $ old_mtime /= new_mtime }}} I might be wrong, but it looks to me as if this can only ever return False (''UpToDate'' in current GHC head) '''because''' the mtime is so coarse - in other words if your computer is too slow, a second will pass and this will never be considered up to date? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of dependent files (UsageFile mtime) -------------------------------------------------+------------------------- Reporter: nh2 | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time performance bug | Unknown/Multiple Test Case: | Difficulty: Blocking: | Unknown | Blocked By: | Related Tickets: -------------------------------------------------+------------------------- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of dependent files (UsageFile mtime) -------------------------------------------------+------------------------- Reporter: nh2 | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time performance bug | Unknown/Multiple Test Case: | Difficulty: Blocking: | Unknown | Blocked By: | Related Tickets: -------------------------------------------------+------------------------- Comment (by nh2): Replying to [comment:8 jstolarek]:

nh2, what platform are you on?

Linux on all the GHCs I mentioned above. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of dependent files (UsageFile mtime) -------------------------------------------------+------------------------- Reporter: nh2 | Owner: nh2 Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time performance bug | Unknown/Multiple Test Case: | Difficulty: Blocking: | Unknown | Blocked By: | Related Tickets: -------------------------------------------------+------------------------- Changes (by nh2): * owner: => nh2 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of dependent files (UsageFile mtime) -------------------------------------------------+------------------------- Reporter: nh2 | Owner: nh2 Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time performance bug | Unknown/Multiple Test Case: | Difficulty: Blocking: | Unknown | Blocked By: | Related Tickets: -------------------------------------------------+------------------------- Comment (by Austin Seipp ): In [changeset:677820ee3a3aadbf2ed414deb3926381d94b13a8/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="677820ee3a3aadbf2ed414deb3926381d94b13a8" Fix interface hashes including time stamp of dependent files. Fixes #8144. Before, the modification time of e.g. #included files (and everything that ends up as a UsageFile, e.g. via addDependentFile) was taken as input for the interface hash of a module. This lead to different hashes for identical inputs on every compilation. We now use file content hashes instead. This changes the interface file format. You will get "Binary.get(Usage): 50" when you try to do an incremental using .hi files that were created with a GHC 7.7 (only) older than this commit. To calculate the md5 hash (`Fingerprint`) of a file in constant space, there now is GHC.Fingerprint.getFileHash, and a fallback version for older GHCs that needs to load the file into memory completely (only used when compiling stage1 with an older GHC). Signed-off-by: Austin Seipp }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of dependent files (UsageFile mtime) -------------------------------------------------+------------------------- Reporter: nh2 | Owner: nh2 Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time performance bug | Unknown/Multiple Test Case: | Difficulty: Blocking: | Unknown | Blocked By: | Related Tickets: -------------------------------------------------+------------------------- Comment (by Austin Seipp ): In [changeset:b6a572b4272274efe117d5e04f4b9d49e124009c/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="b6a572b4272274efe117d5e04f4b9d49e124009c" Add some more comments to UsageFile. This brings them up to date with the changes in #8144. Signed-off-by: Austin Seipp }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of dependent files (UsageFile mtime) -------------------------------------------------+------------------------- Reporter: nh2 | Owner: nh2 Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time performance bug | Unknown/Multiple Test Case: | Difficulty: Blocking: | Unknown | Blocked By: | Related Tickets: -------------------------------------------------+------------------------- Comment (by thoughtpolice): Thanks, I've merged and I'm validating now. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of dependent files (UsageFile mtime) -------------------------------------------------+------------------------- Reporter: nh2 | Owner: nh2 Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time performance bug | Unknown/Multiple Test Case: | Difficulty: Blocking: | Unknown | Blocked By: | Related Tickets: -------------------------------------------------+------------------------- Comment (by nh2): A backport to 7.6 can be found at https://github.com/nh2/ghc/releases/tag /fix-interface-hashes-mtime-7.6.3-release-tarball-tag (based on 7.6.3 release tarball, 64 bit Linux ghc binary also provided). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of dependent files (UsageFile mtime) -------------------------------------------------+------------------------- Reporter: nh2 | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | testcase Type of failure: Compile-time performance bug | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: | Unknown | Blocked By: | Related Tickets: -------------------------------------------------+------------------------- Changes (by ezyang): * keywords: => testcase * owner: nh2 => * status: closed => new * resolution: fixed => Comment: nh2: Would it be possible to get a test case on this one? (Or is there one already?) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of dependent files (UsageFile mtime) -------------------------------------+------------------------------------- Reporter: nh2 | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: testcase Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by rodlogic): Should this be closed or is a test case still pending? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:23 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of dependent files (UsageFile mtime) -------------------------------------+------------------------------------- Reporter: nh2 | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: testcase Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by nh2): I added a test case at https://github.com/nh2/ghc/compare/testcase-8144. Please pull. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:25 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of dependent files (UsageFile mtime) -------------------------------------+------------------------------------- Reporter: nh2 | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: testcase Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by thomie): Test LGTM. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:27 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of dependent files (UsageFile mtime) -------------------------------------+------------------------------------- Reporter: nh2 | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: 7.10.1 Component: Compiler | Version: 7.6.3 Resolution: | Keywords: testcase Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by thoughtpolice): Here's a WIP patch: https://github.com/ghc/ghc/commit/a3bf4bfae694572b0bf886fcb1191d066d242aed on `wip/8144` -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:29 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of dependent files (UsageFile mtime) -------------------------------------+------------------------------------- Reporter: nh2 | Owner: Type: bug | Status: infoneeded Priority: high | Milestone: 7.12.1 Component: Compiler | Version: 7.6.3 Resolution: | Keywords: testcase Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by nomeata): * priority: normal => high Comment: Also, from reading the patch, it is not clear to me if the filename is also part of the ABI hash. It should not be, as build directory names may change, e.g. on automatic build machines. Also see http://bugs.debian.org/785282 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:32 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8144: Interface hashes include time stamp of dependent files (UsageFile mtime) -------------------------------------+------------------------------------- Reporter: nh2 | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: 7.12.1 Component: Compiler | Version: 7.6.3 Resolution: | Keywords: testcase Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by rwbarton): nomeata, sorry if I am missing something obvious, but this ticket seems to be about the time stamp not appearing in the interface hash, so the fact that this ticket is just waiting for a test is not incompatible with the interface hash including the file name. Currently we need the .h file name in the interface file so that we can check whether the file fingerprint matches, and we use the interface hash to determine whether to write a new interface file, so it's not immediately obvious that we can avoid putting the file name in the interface hash. Maybe you'd like to create a new ticket about finding some way to avoid this? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8144#comment:34 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler