[GHC] #13375: ghc-pkg "cache is out of date" message breaks testsuite
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Other Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Since the merge of the package database locking business, `ghc-pkg dump` spews a message on `stderr` on Windows, {{{ $ "C:/msys64/home/ben/ghc/inplace/bin/ghc-pkg.exe" dump WARNING: cache is out of date: C:\msys64\home\ben\ghc\inplace\lib\package.conf.d\package.cache ghc will see an old view of this package db. Use 'ghc-pkg recache' to fix. name: xhtml version: 3000.2.1 id: xhtml-3000.2.1 ... }}} When the testsuite driver sees this it (namely `testutil.getStdout`) assumes something has gone terribly wrong and aborts. From a cursory look it's not clear how we got into this situation; the file indeed exists and was certainly touched during the build. However, running `ghc-pkg recache` indeed seems to fix the issue. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by arybczak): I compiled ghc from scratch and here's how inplace/lib/package.conf.d changes: 10:18 - compilation start <package.conf.d directory is created somewhere here> 10:25 - rts registered 10:33 - all other core packages registered (I think that's after ghc- stage1 is built) 10:54 - a lock file is created (after ghc-stage2 is built) The thing is that ghc-stage2, when run, reads package db and creates a lock file, which wasn't there yet because all previous modifications of package db were done using ghc-pkg with no locking support. Then, ghc-pkg fails its naive modtime check (i.e. modtime of package.conf.d is higher than package.cache precisely because ghc created lock file when reading the database) and hence the warning. Note that read lock is acquired on Windows only, that's why this doesn't happen on other platforms. I think we can either make the modtime check in ghc-pkg less naive (i.e. compare max(modtime of .conf files) to modtime of package.cache instead of modtime of the directory) or just call recache command. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:1> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: infoneeded Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by arybczak): * status: new => infoneeded -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:2> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: infoneeded Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): It seems to me like making the modtime check less naive is the right solution here. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:3> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: infoneeded Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): I have a fix in Phab:D3289. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:4> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: infoneeded Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Ben Gamari <ben@…>): In [changeset:"30d69f404ba102da94423836f86fbec2fb4adaf9/ghc" 30d69f4/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="30d69f404ba102da94423836f86fbec2fb4adaf9" ghc-pkg: Consider .conf files when computing package db mtime We can no longer use the mtime of the containing directory since it now contains a lock file in addition to the .cache and .conf files. Fixes #13375. Test Plan: Validate on Windows Reviewers: austin, arybczak Subscribers: rwbarton, thomie Differential Revision: https://phabricator.haskell.org/D3289 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:5> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: closed Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: infoneeded => closed * resolution: => fixed -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:6> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: closed Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by marinelli): Is it still open? I've just tried listing and dumping packages with the both release candidate versions, obtaining the same result, e.g.: {{{ $ ghc-pkg list WARNING: cache is out of date: /opt/apps/ghc/20170510-8.2.1-rc2-1/lib/ghc-8.2.0.20170507/package.conf.d/package.cache ghc will see an old view of this package db. Use 'ghc-pkg recache' to fix. /opt/apps/ghc/20170510-8.2.1-rc2-1/lib/ghc-8.2.0.20170507/package.conf.d Cabal-2.0.0.0 array-0.5.1.2 ---✂--- }}} I am working with a Debian distribution (testing channel) on a btrfs file system, and I've used the x86_64-deb8 builds of GHC. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:7> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: closed Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by arybczak): What are the steps to reproduce? -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:8> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: closed Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by marinelli): Here they are: {{{ $ mkdir -p /tmp/ghc $ wget -q -P /tmp/ghc/ 'https://downloads.haskell.org/~ghc/8.2.1-rc2/ghc-8.2.0.20170507-x86_64-deb8-...' $ cd /tmp/ghc/ $ tar xf ghc-8.2.0.20170507-x86_64-deb8-linux.tar.xz $ cd ./ghc-8.2.0.20170507/ $ sh configure --prefix=/opt/ghc $ make install $ env PATH=/opt/ghc/bin:$PATH ghc --version The Glorious Glasgow Haskell Compilation System, version 8.2.0.20170507 $ env PATH=/opt/ghc/bin:$PATH ghc-pkg list WARNING: cache is out of date: /opt/ghc/lib/ghc-8.2.0.20170507/package.conf.d/package.cache ghc will see an old view of this package db. Use 'ghc-pkg recache' to fix. /opt/ghc/lib/ghc-8.2.0.20170507/package.conf.d Cabal-2.0.0.0 array-0.5.1.2 ---✂--- }}} The file config.log is here https://gist.github.com/anonymous/05fc2191ffd6a49eb8f3108b17d07d51 -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:9> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: closed Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by arybczak): The problem is that `make install` script, after installing all packages, does this: `for f in '/home/unknown/Media/GHC/8.2.1_rc2/lib/ghc-8.2.0.20170507/package.conf.d'/*; do create () { touch "$1" && chmod 644 "$1" ; } && create "$f"; done` which puts modification time of several .conf packages ahead of package.cache. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:10> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: closed Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Good catch, arybczak! I suppose we could just touch the cache at the end of `install`. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:11> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3569 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: closed => new * differential: => Phab:D3569 * resolution: fixed => -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:12> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: patch Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3569 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => patch -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:13> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: patch Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3569 Wiki Page: | -------------------------------------+------------------------------------- Comment (by marinelli): Wouldn't it be better to rebuild it? We'll certainly obtain a package.cache file that is consistent. We could use something like this after that for-loop in ghc.mk: {{{ "$(INSTALLED_GHC_PKG_REAL)" --global-package-db "$(INSTALLED_PACKAGE_CONF)" recache }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:14> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: patch Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3569 Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Fair point, I suppose that would be a bit more foolproof. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:15> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: bgamari Type: bug | Status: patch Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3569 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * owner: (none) => bgamari -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:16> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: bgamari Type: bug | Status: patch Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3569 Wiki Page: | -------------------------------------+------------------------------------- Comment (by Ben Gamari <ben@…>): In [changeset:"d6686a254293442a633482eae7ca78be968bef58/ghc" d6686a25/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="d6686a254293442a633482eae7ca78be968bef58" Ensure package.cache is newer than registration files after make install Rebuild package.cache to ensure that it's newer than the package database registration files, avoiding out-of-date cache warnings from ghc-pkg. See #13375. Test Plan: `make install`, run `ghc-pkg list`, look for out-of-date cache warning Reviewers: austin Subscribers: rwbarton, thomie GHC Trac Issues: #13375 Differential Revision: https://phabricator.haskell.org/D3569 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:17> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: bgamari Type: bug | Status: closed Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3569 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: patch => closed * resolution: => fixed Comment: Merged to `ghc-8.2` as 4ea0868737cfce7051bc10a731d5de152c93fde5. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:18> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13375: ghc-pkg "cache is out of date" message breaks testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: bgamari Type: bug | Status: closed Priority: highest | Milestone: 8.2.1 Component: ghc-pkg | Version: 8.0.1 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: #13194 | Differential Rev(s): Phab:D3569 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * related: => #13194 -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13375#comment:19> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
participants (1)
-
GHC