#9696: readRawBufferPtr and writeRawBufferPtr allocate memory -------------------------------------+------------------------------------- Reporter: | Owner: mergeconflict | Status: new Type: bug | Milestone: Priority: normal | Version: 7.8.3 Component: Compiler | Keywords: Resolution: | Architecture: x86_64 (amd64) Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: Runtime | Related Tickets: performance bug | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Description changed by mergeconflict: Old description:

I initially filed this as a [http://stackoverflow.com/questions/26333815 /why-do-hgetbuf-hputbuf-etc-allocate-memory question on StackOverflow], assuming that the behavior I'm seeing was intentional, or that I was misinterpreting the profiler results... but Kazu Yamamoto [https://twitter.com/kazu_yamamoto/status/522583112249663488 suggested] I should file this as an issue here. It's my first GHC ticket, so here goes:

{{{#!hs main :: IO () main = do buf <- mallocArray 1 echo buf

echo :: Ptr Word8 -> IO () echo buf = forever $ do len <- readRawBufferPtr "read" stdin buf 0 1 writeRawBufferPtr "write" stdout buf 0 (fromIntegral len) }}}

I expect the only heap allocation here should be my explicit `mallocArray`, but profiling with `+RTS -P` indicates this isn't the case: both the read and write operations do appear to allocate some short-lived heap objects. In contrast:

{{{#!hs echo :: Ptr Word8 -> IO () echo buf = forever $ do threadWaitRead $ Fd 0 len <- c_read 0 buf 1 c_write 1 buf (fromIntegral len) yield }}}

does not appear to allocate.

I did a bit of digging (copying bits of source code from `base` into my own project, to get cost center annotations) and it seems like the allocation might be happening in `throwErrnoIfRetryMayBlock`. See [https://gist.github.com/anonymous/3ba3cfa118e1d05870d4 this gist] for more detail, including core output from `-ddump-simpl`.

In all honesty, I don't know whether this is a bug. I was just surprised by it, when I first encountered it using `hGetBuf` / `hPutBuf`, so I'm trying to understand whether this is expected behavior and why.

New description: I initially filed this as a [http://stackoverflow.com/questions/26333815 /why-do-hgetbuf-hputbuf-etc-allocate-memory question on StackOverflow], assuming that the behavior I'm seeing was intentional, or that I was misinterpreting the profiler results... but Kazu Yamamoto [https://twitter.com/kazu_yamamoto/status/522583112249663488 suggested] I should file this as an issue here. It's my first GHC ticket, so here goes: {{{#!hs main :: IO () main = do buf <- mallocArray 1 echo buf echo :: Ptr Word8 -> IO () echo buf = forever $ do len <- readRawBufferPtr "read" stdin buf 0 1 writeRawBufferPtr "write" stdout buf 0 (fromIntegral len) }}} I expect the only heap allocation here should be my explicit `mallocArray`, but profiling with `+RTS -P` indicates this isn't the case: both the read and write operations do appear to allocate some short-lived heap objects each time through the loop. (Note: this isn't a space leak, the allocated objects appear to be GC'ed quickly, whatever they are). In contrast: {{{#!hs echo :: Ptr Word8 -> IO () echo buf = forever $ do threadWaitRead $ Fd 0 len <- c_read 0 buf 1 c_write 1 buf (fromIntegral len) yield }}} does not appear to allocate. I did a bit of digging (copying bits of source code from `base` into my own project, to get cost center annotations) and it seems like the allocation might be happening in `throwErrnoIfRetryMayBlock`. See [https://gist.github.com/anonymous/3ba3cfa118e1d05870d4 this gist] for more detail, including core output from `-ddump-simpl`. In all honesty, I don't know whether this is a bug. I was just surprised by it, when I first encountered it using `hGetBuf` / `hPutBuf`, so I'm trying to understand whether this is expected behavior and why. -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9696#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9696: readRawBufferPtr and writeRawBufferPtr allocate memory -------------------------------------+------------------------------------- Reporter: mergeconflict | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: newcomer Operating System: Unknown/Multiple | Architecture: x86_64 Type of failure: Runtime | (amd64) performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by thomie): * keywords: => newcomer Comment:

Also, there is a safe foreign import of `rtsSupportsBoundThreads` in GHC.IO.FD (and also some other module(s) in base), which seems quite unnecessary considering `rtsSupportsBoundThreads` simply returns a constant.

For a newcomer, to get acquainted with the patch submission process. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9696#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9696: readRawBufferPtr and writeRawBufferPtr allocate memory -------------------------------------+------------------------------------- Reporter: mergeconflict | Owner: Type: bug | Status: new Priority: low | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: newcomer Operating System: Unknown/Multiple | Architecture: x86_64 Type of failure: Runtime | (amd64) performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by mjmrotek): Replying to [comment:3 thomie]:

...

Also, there is a safe foreign import of `rtsSupportsBoundThreads` in GHC.IO.FD (and also some other module(s) in base), which seems quite unnecessary considering `rtsSupportsBoundThreads` simply returns a constant.

For a newcomer, to get acquainted with the patch submission process.

I did this part in https://phabricator.haskell.org/D1964 , grepping through HEAD didn't show any more safe imports of `rtsSupportsBoundThreads`. I didn't touch anything related to `readRawBufferPtr`, `writeRawBufferPtr`, or `throwErrnoIfRetryMayBlock`, though. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9696#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9696: readRawBufferPtr and writeRawBufferPtr allocate memory -------------------------------------+------------------------------------- Reporter: mergeconflict | Owner: Type: bug | Status: new Priority: low | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: newcomer Operating System: Unknown/Multiple | Architecture: x86_64 Type of failure: Runtime | (amd64) performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Ben Gamari ): In [changeset:"1d6177b133f3a6ac28cc8a679807563cfca3c56a/ghc" 1d6177b1/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="1d6177b133f3a6ac28cc8a679807563cfca3c56a" Using unsafe foreign import for rtsSupportsBoundThreads (part of #9696) A safe import is unnecessary considering rtsSupportsBoundThreads simply returns a constant. This commit doesn't fix the main issue of ticket #9696 that "readRawBufferPtr and writeRawBufferPtr allocate memory". Reviewers: bgamari, austin, hvr Reviewed By: hvr Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D1964 GHC Trac Issues: #9696 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9696#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9696: readRawBufferPtr and writeRawBufferPtr allocate memory -------------------------------------+------------------------------------- Reporter: mergeconflict | Owner: Type: bug | Status: patch Priority: low | Milestone: 8.2.1 Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: x86_64 Type of failure: Runtime | (amd64) performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2813 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => patch * differential: => Phab:D2813 * milestone: => 8.2.1 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9696#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9696: readRawBufferPtr and writeRawBufferPtr allocate memory -------------------------------------+------------------------------------- Reporter: mergeconflict | Owner: Type: bug | Status: closed Priority: low | Milestone: 8.2.1 Component: Compiler | Version: 7.8.3 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: x86_64 Type of failure: Runtime | (amd64) performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2813 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: patch => closed * resolution: => fixed -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9696#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler