[GHC] #8594: sysctl name "hw.ncpu" (HW_NCPU) is deprecated in Mac OS X
#8594: sysctl name "hw.ncpu" (HW_NCPU) is deprecated in Mac OS X ------------------------------------+--------------------------------- Reporter: kseo | Owner: simonmar Type: task | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 7.6.3 Keywords: | Operating System: MacOS X Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+--------------------------------- According to the OS X man page for sysctl(https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPa...), HW_NCPU used by the current implementation of getNumberOfProcessors() is deprecated. Instead, Mac OS X provides the following 4 options: * hw.physicalcpu: The number of physical processors available in the current power management mode. * hw.physicalcpu_max: The maximum number of physical processors that could be available this boot. * hw.logicalcpu: The number of logical processors available in the current power management mode. * hw.logicalcpu_max: The maximum number of logical processors that could be available this boot. I am not sure which one fits best to GHC runtime. Any idea? sysconf implementation of getNumberOfProcessors() seems to prefer _SC_NPROCESSORS_ONLN to _SC_NPROCESSORS_CONF when both are defined. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8594 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8594: sysctl name "hw.ncpu" (HW_NCPU) is deprecated in Mac OS X -----------------------------------+------------------------------------ Reporter: kseo | Owner: simonmar Type: task | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 7.6.3 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: None/Unknown | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -----------------------------------+------------------------------------ Comment (by carter): I think logicalcpu counts the number of hyperthreaded cores, and physicalcpu is the number of cores (ignoring hyperthreading), which should be half that number. I checked this on my macbook air (which has 2 cores, but 4 via hyper threading) {{{ carter repoScratcher/ghc ‹master› » sysctl hw.physicalcpu hw.physicalcpu: 2 carter repoScratcher/ghc ‹master› » sysctl hw.logicalcpu hw.logicalcpu: 4 }}} I think the _max variants is because using the Instruments App, you can tell OS X to disable hardware multithreading/ adjust the number of cores the OS scheduler sees. I just disabled the hyperthreading using Instruments, and now i get {{{ carter repoScratcher/ghc ‹master› » sysctl hw.logicalcpu hw.logicalcpu: 2 }}} So i think the not max variants make sense, namely hw.logicalcpu , because that actually reflects the number of logical cores the OS sees. for general use, using the hyperthreaded cores is probably the right default, rather than not. Admittedly, on parallel computations that are memory bound, hyperthreading can either boon or a curse, and perhaps this is worth benchmarking! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8594#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8594: sysctl name "hw.ncpu" (HW_NCPU) is deprecated in Mac OS X -----------------------------------+------------------------------------ Reporter: kseo | Owner: simonmar Type: task | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 7.6.3 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: None/Unknown | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -----------------------------------+------------------------------------ Comment (by kseo): It seems hw.ncpu is equivalent to hw.logicalcpu. {{{ kseo@com100ui-MacBook-Pro:~> sysctl hw.ncpu hw.ncpu: 4 }}} {{{ kseo@com100ui-MacBook-Pro:~> sysctl hw.logicalcpu hw.logicalcpu: 4 }}} {{{ kseo@com100ui-MacBook-Pro:~> sysctl hw.physicalcpu hw.physicalcpu: 2 }}} Does GHC have a benchmark suite to measure the impact of this change? If not, we can fallback to hw.logicalcpu if hw.ncpu is not available. This at least preserves the current behavior. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8594#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8594: sysctl name "hw.ncpu" (HW_NCPU) is deprecated in Mac OS X -----------------------------------+------------------------------------ Reporter: kseo | Owner: simonmar Type: task | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 7.6.3 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: None/Unknown | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -----------------------------------+------------------------------------ Comment (by carter): benchmark suite for which? using the physical cpu count? thats be like using -N2 rather than -N4 (as our default value of -N). In performance sensitive applications, using the physical core count can sometimes be a win because of better cache behavior, but defualting to using the logical cpu count is the right default for general purpose applications. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8594#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8594: sysctl name "hw.ncpu" (HW_NCPU) is deprecated in Mac OS X -----------------------------------+------------------------------------ Reporter: kseo | Owner: simonmar Type: task | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 7.6.3 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: None/Unknown | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -----------------------------------+------------------------------------ Comment (by carter): feel free to benchmark though, but I don't think it likely matters -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8594#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8594: sysctl name "hw.ncpu" (HW_NCPU) is deprecated in Mac OS X -----------------------------------+------------------------------------ Reporter: kseo | Owner: simonmar Type: task | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 7.6.3 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: None/Unknown | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -----------------------------------+------------------------------------ Comment (by kseo): I've attached a patch to use hw.logicalcpu in OS X. FreeBSD still uses "hw.ncpu", so I split the guard. This patch does not change the behavior because hw.ncpu also returns the logical cpu count. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8594#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8594: sysctl name "hw.ncpu" (HW_NCPU) is deprecated in Mac OS X -----------------------------------+------------------------------------ Reporter: kseo | Owner: simonmar Type: task | Status: patch Priority: normal | Milestone: Component: Runtime System | Version: 7.6.3 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: None/Unknown | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -----------------------------------+------------------------------------ Changes (by thomie): * status: new => patch -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8594#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8594: sysctl name "hw.ncpu" (HW_NCPU) is deprecated in Mac OS X -------------------------------------+------------------------------------- Reporter: kseo | Owner: simonmar Type: task | Status: infoneeded Priority: normal | Milestone: 7.10.1 Component: Runtime | Version: 7.6.3 System | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: MacOS X | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Changes (by thoughtpolice): * status: patch => infoneeded * milestone: => 7.10.1 Comment: I think this patch is OK, but what minimum version is `hw.logicalcpu` supported in? Perhaps instead it should simply try both: fall back to `hw.ncpu` if `hw.logicalcpu` fails. Could you make this change @kseo? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8594#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8594: sysctl name "hw.ncpu" (HW_NCPU) is deprecated in Mac OS X -------------------------------------+------------------------------------- Reporter: kseo | Owner: simonmar Type: task | Status: infoneeded Priority: normal | Milestone: 7.10.1 Component: Runtime | Version: 7.6.3 System | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: MacOS X | Difficulty: Unknown Type of failure: | Blocked By: None/Unknown | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by thomie): [https://hg.python.org/cpython/file/1855b5c3da61/Modules/posixmodule.c#l16074 Python] (warning: big file) uses [https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPa... _SC_NPROCESSORS_ONLN]. "The number of processors currently online". Whatever that means. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8594#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8594: sysctl name "hw.ncpu" (HW_NCPU) is deprecated in Mac OS X -------------------------------------+------------------------------------- Reporter: kseo | Owner: ak3n Type: task | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 7.6.3 Resolution: | Keywords: Operating System: MacOS X | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2082 Wiki Page: | -------------------------------------+------------------------------------- Changes (by ak3n): * owner: simonmar => ak3n * differential: => Phab:D2082 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8594#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8594: sysctl name "hw.ncpu" (HW_NCPU) is deprecated in Mac OS X
-------------------------------------+-------------------------------------
Reporter: kseo | Owner: ak3n
Type: task | Status: new
Priority: normal | Milestone:
Component: Runtime System | Version: 7.6.3
Resolution: | Keywords:
Operating System: MacOS X | Architecture:
| Unknown/Multiple
Type of failure: None/Unknown | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s): Phab:D2082
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Austin Seipp
#8594: sysctl name "hw.ncpu" (HW_NCPU) is deprecated in Mac OS X -------------------------------------+------------------------------------- Reporter: kseo | Owner: ak3n Type: task | Status: closed Priority: normal | Milestone: 8.2.1 Component: Runtime System | Version: 7.6.3 Resolution: fixed | Keywords: Operating System: MacOS X | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2082 Wiki Page: | -------------------------------------+------------------------------------- Changes (by thoughtpolice): * status: new => closed * resolution: => fixed * milestone: => 8.2.1 Comment: Finally fixed. Thank you for updating the patch with my requests, Eugene! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8594#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC