#11501: Building nofib/fibon returns permission denied -------------------------------------+------------------------------------- Reporter: rem | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: NoFib benchmark | Version: 7.10.3 suite | Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 | (amd64) Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by gracjan): After some research I have some insights to share. Looks like nofib is used for two distinct purposes: 1. To test speed of generated code. 2. To test compilation speed. `nofib` setup is geared towards interactive use, that is to be able to run speed experiments and compare them pair-wise. Running `make` runs both compilation speed and execution speed tests at the same time. There is also a way to run `valgrind`s `cachegrind` at the same time. Major problems with the test suite I see now are: 1. Only a handful of speed tests are enabled, everything else bitrots. 2. Build system compiles objects and links them, does not use `--make`. 3. Build system is ill suited for tests extracted from cabal projects. 4. There is a confusion between compilation speed tests and execution speed tests. The way forward as I see: 1. Enable all tests by default, but make it easy to select a subset for interactive work. 2. Use `ghc --make` for builds of simpler cases (no dependencies besiades what is shipped with ghc). 3. Use `cabal install` for more complicated cases (with more dependencies). 4. Clearly separate compilation speed tests from execution speed tests. And in distant future: 1. Run execution speed tests on TravisCI in multiple combinations of os/ghc/flags/test. 2. Run compilation speed tests on TravisCI in multiple combinations of os/ghc/flags/test. 3. Run incremental compilation speed tests on TravisCI in multiple combinations of os/ghc/flags/test. 4. Publish metrics to some kind of timeseries database. 5. Let Grafana show some nice graphs. Graphana is here: http://grafana.org/ I'd like to hear from people that use nofib in daily work how do they do this so that we do not skip any important use cases. Can you guys chime in and describe your workflow? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11501#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler