[GHC] #15768: Using oneshot kqueue() on macOS
#15768: Using oneshot kqueue() on macOS -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Core | Version: 8.6.1 Libraries | Keywords: KQueue, | Operating System: MacOS X oneshot, parallel build | Architecture: | Type of failure: Other Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Oneshot kqueue() is disabled on macOS due to https://ghc.haskell.org/trac/ghc/ticket/7651 Recall that the paper "Mio: A High-Performance Multicore IO Manager for GHC" says: "Mio also uses kqueue on OS X, since Darwin, the foundation for Apple’s OS X, is a variant of BSD. However, we encountered problems running parallel builds of the GHC compiler using Mio as described above. We have been unable to uncover the underlying source of the problem to our satisfaction. However, several Internet discussions suggest that the implementation of kqueue and pipe on OS X are unstable. We were able to resolve the observed prob- lems on OS X by avoiding the use of one-shot mode on OS X, instead unregistering events after they are delivered, and by send- ing wakeup writes to a pipe monitored by the dispatcher threads’ kqueue instances on every event registration. With these changes, the behavior on OS X, including parallel builds of GHC, has been stable." Now the serious bug on KQueue has been fixed: - https://ghc.haskell.org/trac/ghc/ticket/13903 - https://phabricator.haskell.org/D3692 So, I guess it's time to use oneshot on macOS. A patch is attached. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15768 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15768: Using oneshot kqueue() on macOS -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 8.6.1 Resolution: | Keywords: KQueue, | oneshot, parallel build Operating System: MacOS X | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by kazu-yamamoto): * Attachment "macos-oneshot-kqueue.diff" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15768 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15768: Using oneshot kqueue() on macOS -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: (none) Type: bug | Status: patch Priority: highest | Milestone: 8.8.1 Component: Core Libraries | Version: 8.6.1 Resolution: | Keywords: KQueue, | oneshot, parallel build Operating System: MacOS X | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => patch * priority: normal => highest * milestone: => 8.8.1 Comment: Thanks Kazu! Let's try to get this in for 8.8. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15768#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15768: Using oneshot kqueue() on macOS -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: (none) Type: bug | Status: patch Priority: highest | Milestone: 8.8.1 Component: Core Libraries | Version: 8.6.1 Resolution: | Keywords: KQueue, | oneshot, parallel build Operating System: MacOS X | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by George): just curious, 13903 was on many platforms, right? So why did it only affect oneshot kqueue() on only the Mac? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15768#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15768: Using oneshot kqueue() on macOS -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: (none) Type: bug | Status: patch Priority: highest | Milestone: 8.8.1 Component: Core Libraries | Version: 8.6.1 Resolution: | Keywords: KQueue, | oneshot, parallel build Operating System: MacOS X | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by kazu-yamamoto): kqueue() is used on MacOS and BSD variants. But when we implemented Mio, we only tested it on MacOS and Linux. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15768#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15768: Using oneshot kqueue() on macOS -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: (none) Type: bug | Status: patch Priority: highest | Milestone: 8.8.1 Component: Core Libraries | Version: 8.6.1 Resolution: | Keywords: KQueue, | oneshot, parallel build Operating System: MacOS X | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by George): Ah ok, I was wondering why we didn't see this on Linux but the papers says we weren't using it there. We used epoll. From the paper: "The registerFd func- tion takes the callback table lock, and then inserts (or modifies) an entry in the callback table and registers the event subscription us- ing the underlying OS event notification mechanism, for example epoll [8] on Linux, kqueue [7] on BSD variants, and poll on other OSes" -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15768#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15768: Using oneshot kqueue() on macOS -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: (none) Type: bug | Status: patch Priority: highest | Milestone: 8.8.1 Component: Core Libraries | Version: 8.6.1 Resolution: | Keywords: KQueue, | oneshot, parallel build Operating System: MacOS X | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by lelf): * cc: lelf (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15768#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC