[GHC] #15576: Hadrian puts its build tree in the wrong place
#15576: Hadrian puts its build tree in the wrong place -------------------------------------+------------------------------------- Reporter: simonpj | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- I have a source tree in my slow, network-mounted backed-up filestore `/home/simonpj/ghc`, and a separate link-tree in my fast, physically- connected, non-backed up filestore `/playpen/ghc`. Every file in `/playpen/ghc` is a symlink to the corresponding source file in `/home/simonpj/ghc`. But when I do {{{ $ cd /playpen/ghc $ hadrian/build.sh -c -j4 --flavour=quick }}} I find a new `_build` director in `/home/simonpj/ghc`. This is bad! It should be in `/playpen/ghc`. Otherwise (a) it's slower, (b) it creates backup churn and (c) as it happens, I ran out of space on the backed-up filestore. Can this be fixed? It's a bad Hadrian shortcoming. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15576 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15576: Hadrian puts its build tree in the wrong place -------------------------------------+------------------------------------- Reporter: simonpj | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by snowleopard): You can change the build directory via the command line using `--build- root=PATH` (or shorter `-oPATH`). Can you try this? {{{ hadrian/build.sh -c -j4 --flavour=quick -o/playpen/ghc }}} This is documented here: https://github.com/snowleopard/hadrian#command- line-flags. We can also add support for specifying the build directory in `UserSettings` so that you don't need to specify it each time via the command line. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15576#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15576: Hadrian puts its build tree in the wrong place -------------------------------------+------------------------------------- Reporter: simonpj | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): I don't particularly want to ''change'' the build directory; I just want it to be right here at the root of my build tree, where it has always been. (That is `-o$PWD`, I suppose.) Is that not a good default place for it to be? I suppose I can get used to doing something in UserSettings in every build tree; but I'd prefer the old behaviour. Is it difficult to achieve for some reason? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15576#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15576: Hadrian puts its build tree in the wrong place -------------------------------------+------------------------------------- Reporter: simonpj | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by snowleopard): Ah, now I'm confused. If you run Hadrian from `/playpen/ghc` I don't understand why build results appear in `/home/simonpj/ghc`.
Is it difficult to achieve for some reason?
Actually, I would say achieving the current behaviour seems difficult! I think the magic symlink-traversal code in `build.sh` and `build.cabal.sh` has something to do with it. I believe it was added by Herbert, but I don't recall why. Can you try building and running Hadrian directly using Cabal? {{{ cd hadrian cabal new-build --disable-profiling --disable-documentation -j exe:hadrian cabal new-run hadrian -- -c -j4 --flavour=quick --directory=".." }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15576#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15576: Hadrian puts its build tree in the wrong place -------------------------------------+------------------------------------- Reporter: simonpj | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): I double-checked. * I removed `/home/simonpj/ghc/_build` entirely * I removed `/playpen/ghc/_build` entirely * I did `cd /playpen/ghc` * And then `hadrian/build.sh -c -j20 --flavour=quick` Result: * a `_build` directory immediately appeared in `/home/simonpj/ghc`, populated with `generated`, `hadrian`, `stage0`, `stage1`. * The `hadrian/` directory contains `.shake.database`. * No `_build` directory appeared in `/playpen/ghc` -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15576#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15576: Hadrian puts its build tree in the wrong place -------------------------------------+------------------------------------- Reporter: simonpj | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): OK I did what you ask in comment:3. The first `cabal new-build` completed fine; no `_build` directories anywhere. The second command `cabal new-run` command is still running. The start of its output is below. (NB: `~/5builds/HEAD-3` is the build tree that I was previously calling `/playpen/ghc`; but it really IS the build tree.) Results so far: `_build` is created in `/playpen/ghc` (or in fact `~/5builds/HEAD-3`); there is no `_build` in `/users/simonpj/ghc`. Note the messages from configure about `.git`... {{{ simonpj@cam-05-unx:~/5builds/HEAD-3/hadrian$ cabal new-run hadrian -- -c -j4 --flavour=quick --directory=".." Up to date | Running boot... | Run Configure ".": hadrian/cfg/system.config.in (and 4 more) => hadrian/cfg/system.config (and 4 more) configure: WARNING: cannot determine snapshot version: no .git directory and no VERSION file configure: WARNING: cannot determine snapshot revision: no .git directory and no 'GIT_COMMIT_ID' file Warning: libraries/text/text.cabal:4:1: The field "bug-reports" is specified more than once at positions 4:1, 42:1 | Copy file: settings => _build/stage1/lib/settings | Copy file: utils/hsc2hs/template-hsc.h => _build/stage1/lib/template- hsc.h | Copy file: driver/ghci-usage.txt => _build/stage1/lib/ghci-usage.txt | Copy file: llvm-passes => _build/stage1/lib/llvm-passes | Copy file: llvm-targets => _build/stage1/lib/llvm-targets Warning: utils/hpc/hpc-bin.cabal:11:2: Tabs used as indentation at 11:2 | Copy file: driver/ghc-usage.txt => _build/stage1/lib/ghc-usage.txt | Copy file: driver/ghc-usage.txt => _build/stage0/lib/ghc-usage.txt | Copy file: llvm-passes => _build/stage0/lib/llvm-passes | Copy file: llvm-targets => _build/stage0/lib/llvm-targets | Copy file: driver/ghci-usage.txt => _build/stage0/lib/ghci-usage.txt | Copy file: settings => _build/stage0/lib/settings | Successfully generated _build/stage1/bin/ghc-split. | Make '_build/stage1/bin/ghc-split' executable. ...more }}} It appears that my build-tree link script does not add a symlink to `.git`. I have no idea why; there is a symlink to other dot-files like `.gitignore`. Here's the script {{{ #!/usr/bin/perl if (-l "Makefile") { $source_tree = `ls -l Makefile`; $source_tree =~ s/^.*->\s*(.*)\/Makefile\s*$/$1/; print "Linking to source dir: $source_tree\n"; system("lndir $source_tree"); if ($? == 0) { system("kill-dangling-symlinks"); } else { print "errors from lndir; NOT removing dangling symlinks\n"; } } else { print "Linked Makefile not found."; exit(1); } }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15576#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15576: Hadrian puts its build tree in the wrong place -------------------------------------+------------------------------------- Reporter: simonpj | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by snowleopard): Aha, it looks like it works. Presumably missing `.git` files are only important for making releases etc. where we want to record the build commit. Did the build eventually complete with GHC in `/playpen/ghc/_build/stage1/bin`? I'll investigate the issue with `build.sh` logic that negates your symlink-based setup. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15576#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15576: Hadrian puts its build tree in the wrong place -------------------------------------+------------------------------------- Reporter: simonpj | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj):
Did the build eventually complete with GHC in /playpen/ghc/_build/stage1/bin?
Yes, it did. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15576#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15576: Hadrian puts its build tree in the wrong place -------------------------------------+------------------------------------- Reporter: simonpj | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): But how can I build stage2? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15576#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15576: Hadrian puts its build tree in the wrong place -------------------------------------+------------------------------------- Reporter: simonpj | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by snowleopard): It is Stage2 GHC. Here is how Hadrian build tree is organised: * `stage0` contains everything built by the Stage0 (bootstrapping) GHC, including the Stage1 GHC, which lives in `_build/stage0/bin/ghc.exe`. * `stage1` contains everything built by the Stage1 GHC, including the Stage2 GHC in `_build/stage1/bin/ghc.exe`. * `stage2` contains everything built by the Stage2 GHC, for example Haddock. Stage3 GHC will also be here. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15576#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15576: Hadrian puts its build tree in the wrong place -------------------------------------+------------------------------------- Reporter: simonpj | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by snowleopard): We have now removed the symlink traversal logic from Hadrian build scripts, so if you run `hadrian/build.sh` from `/playpen/ghc` the results will appear in `/playpen/ghc/_build`. Could you please check? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15576#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15576: Hadrian puts its build tree in the wrong place -------------------------------------+------------------------------------- Reporter: simonpj | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj):
Could you please check?
How do I check? I assume I have to update Hadrian somehow? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15576#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15576: Hadrian puts its build tree in the wrong place -------------------------------------+------------------------------------- Reporter: simonpj | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by snowleopard): Here is how to update Hadrian: https://ghc.haskell.org/trac/ghc/wiki/Building/Hadrian/QuickStart#UpdatingHa... Hopefully, we will soon merge Hadrian into the GHC tree and there will be no need for fiddling with submodules. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15576#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC