#11041: EventLog write fails if entire buffer is not written -------------------------------------+------------------------------------- Reporter: sseverance | Owner: Type: bug | Status: infoneeded Priority: high | Milestone: 8.0.1 Component: Runtime System | Version: 7.10.2 Resolution: | Keywords: eventlog Operating System: Unknown/Multiple | Architecture: x86_64 Type of failure: Incorrect result | (amd64) at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => infoneeded * failure: None/Unknown => Incorrect result at runtime * priority: normal => high * milestone: => 8.0.1 Old description:

Recently I was trying to use ThreadScope to run down a performance issue. I ran into what I believe is a corner case with flushing the eventlog data to disk. The program was generating hundreds of millions of events every few minutes.

The function printAndClearEventBuf uses fwrite to flush the buffer to disk. What I observed is that with larger buffers it would only succeed in writing the first 4096 bytes. After it would print out its debugBelch message a number of times the program would crash. It does not crash if not run with the eventlog -l flag.

My understanding is that fwrite might need to be called in a loop to ensure that all data is written. fwrite is not guaranteed to always write the entire requested buffer.

I don't have a good test case for this, but if there is agreement about potential issues with flushing the buffer I am more than happy to provide a patch.

New description: Recently I was trying to use ThreadScope to run down a performance issue. I ran into what I believe is a corner case with flushing the eventlog data to disk. The program was generating hundreds of millions of events every few minutes. The function `printAndClearEventBuf` uses `fwrite` to flush the buffer to disk. What I observed is that with larger buffers it would only succeed in writing the first 4096 bytes. After it would print out its `debugBelch` message a number of times the program would crash. It does not crash if not run with the eventlog `-l` flag. My understanding is that `fwrite` might need to be called in a loop to ensure that all data is written. `fwrite` is not guaranteed to always write the entire requested buffer. I don't have a good test case for this, but if there is agreement about potential issues with flushing the buffer I am more than happy to provide a patch. -- Comment: For the sake of completeness of the ticket, what operating system did you observe this on? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11041#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11041: EventLog write fails if entire buffer is not written -------------------------------------+------------------------------------- Reporter: sseverance | Owner: Type: bug | Status: patch Priority: high | Milestone: 8.0.1 Component: Runtime System | Version: 7.10.2 Resolution: | Keywords: eventlog Operating System: Unknown/Multiple | Architecture: x86_64 Type of failure: Incorrect result | (amd64) at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D11041 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: infoneeded => patch * differential: => Phab:D11041 Comment: Opened Phab:D11041 which ought to fix the eventlog flush logic. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11041#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11041: EventLog write fails if entire buffer is not written -------------------------------------+------------------------------------- Reporter: sseverance | Owner: Type: bug | Status: patch Priority: high | Milestone: 8.0.1 Component: Runtime System | Version: 7.10.2 Resolution: | Keywords: eventlog Operating System: Unknown/Multiple | Architecture: x86_64 Type of failure: Incorrect result | (amd64) at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D1415 Wiki Page: | -------------------------------------+------------------------------------- Comment (by Ben Gamari ): In [changeset:"f46f32b922d7ecbee2829937295521f5db1d7997/ghc" f46f32b/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="f46f32b922d7ecbee2829937295521f5db1d7997" EventLog: Loop fwrite if necessary during flush Previously the eventlog flush code would fail if `fwrite` wrote less than the requested amount. Like all Unix stream I/O operations, however, `fwrite` isn't guaranteed to write the entire buffer. Here we loop as long as `fwrite` succeeds in writing anything. Fixes #11041. Test Plan: Validate with eventlog Reviewers: austin, simonmar Reviewed By: simonmar Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D1415 GHC Trac Issues: #11041 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11041#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11041: EventLog write fails if entire buffer is not written -------------------------------------+------------------------------------- Reporter: sseverance | Owner: Type: bug | Status: closed Priority: high | Milestone: 8.0.1 Component: Runtime System | Version: 7.10.2 Resolution: fixed | Keywords: eventlog Operating System: Unknown/Multiple | Architecture: x86_64 Type of failure: Incorrect result | (amd64) at runtime | Test Case: none Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D1415 Wiki Page: | -------------------------------------+------------------------------------- Changes (by thomie): * status: patch => closed * testcase: => none * resolution: => fixed -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11041#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler