#14781: Incorrect CPU core counts in virtualized environments -------------------------------------+------------------------------------- Reporter: rtfeldman | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 8.2.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Description changed by rtfeldman: Old description:
To reproduce this bug:
1. Write a program that prints `numCapabilities` (the # of CPU cores the program thinks are available for parallelization) to stdout 2. Run this program in a virtualized environment such as Travis CI to see what it prints
Expected output: it prints the number of CPU cores available to the process, which in the case of Travis's virtualized environment is 2.
Actual output: it prints the number of *physical* cores on the machine, which in the case of Travis's servers is 32.
The real-world consequences of this bug manifest for the compiler for the Elm programming language, which is written in Haskell and compiled with GHC. By default, `elm-make` runs extremely slowly on Travis and Circle CI because it's trying to parallelize across 32 cores when only 2 are actually available to it.
For a proof-of-concept for how to correctly detect the number of available cores (as opposed to physical), here is the source code to a Rust library which does so correctly: https://github.com/seanmonstar/num_cpus/blob/master/src/lib.rs - the library distinguishes between "number of CPUs" and "number of physical CPUs," and on Travis it correctly reports 2 for CPUs and 32 for physical.
New description: To reproduce this bug, use the following `main.hs`: {{{#!haskell import Control.Concurrent main = print =<< getNumCapabilities }}} Compile it with `ghc -O2 -threaded --make main.hs -o reproduce` and run `./reproduce` in a virtualized environment such as Travis CI. **Expected output:** it prints the number of CPU cores available to the process, which in the case of Travis's virtualized environment is 2. **Actual output:** it prints the number of *physical* cores on the machine, which in the case of Travis's servers is 32. == Example of Failure on Travis Here is an example build on Travis which demonstrates this failing: (scroll to the end of the console log to see the failure) https://travis- ci.org/rtfeldman/repro-ghc-bug/builds/340268785 Here is the repository which was used to reproduce this failure on Travis: https://github.com/rtfeldman/repro-ghc-bug == Real-World Consequences The real-world consequences of this bug manifest for the compiler for the Elm programming language, which is written in Haskell and compiled with GHC. By default, `elm-make` runs extremely slowly on Travis and Circle CI because it's trying to parallelize across 32 cores when only 2 are actually available to it. == How to Fix For an example implementation which correctly detects the number of available cores (as opposed to physical), here is the source code to a Rust library which does so correctly: https://github.com/seanmonstar/num_cpus/blob/master/src/lib.rs - the library distinguishes between "number of CPUs" and "number of physical CPUs," and on Travis it correctly reports 2 for CPUs and 32 for physical. -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14781#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler