[GHC] #13527: ghc has a stack space leak when prints warnings
#13527: ghc has a stack space leak when prints warnings -------------------------------------+------------------------------------- Reporter: slyfox | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Found on a crash when ghc tried to print a warning in a huge assembly file. I've simplified it to autogenerated C file with an error: {{{ # generate 10 million lines file # and put an error into the very end of file $ { for i in `seq 1 $((10 * 1000 * 1000))`; do echo "// a comment number $i"; done; echo 'static int foo = ;'; } > a.c }}} {{{ $ LANG=C powerpc64le-unknown-linux-gnu-ghc -c a.c +RTS -K128M stack overflow: use +RTS -K<size> to increase it }}} C compiler tried to output something like the following to stderr: {{{ a.c:10000001:0: error: error: expected expression before ';' token static int foo = ; }}} GHC finds 'a.c:10000001' location and tries to pretty-print the line. We get a space leak somewhere. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13527 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#13527: ghc has a stack space leak when prints warnings -------------------------------------+------------------------------------- Reporter: slyfox | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by slyfox): I've built ghc-prof and found out GHC is most stressed in 'ErrUtils.getCaretDiagnostic' where source file is read from disk and then chunked by newlines. lexemeToString generates a lot of heap traffic as it decodes whole file into String. This example is enough to take more that 100M of stack: {{{#!hs {-# LANGUAGE PackageImports #-} module Main (main) where import qualified StringBuffer as SB main = do b <- SB.hGetStringBuffer "a.c" let s = SB.lexemeToString b (SB.len b) print (length $ lines s) }}} {{{ $ ghc --make a.hs -o a -package=ghc -debug -rtsopts $ ./a +RTS -K100M a: Stack space overflow: current size 33624 bytes. a: Use `+RTS -Ksize -RTS' to increase it. }}} Looks like there is 2 bugs here: - '''lexemeToString''' takes a lot of stack to decode StringBuffer into String - '''getCaretDiagnostic ''' could use more efficient mechanism to split file into lines before converting everything to String. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13527#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#13527: ghc has a stack space leak when prints warnings -------------------------------------+------------------------------------- Reporter: slyfox | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * cc: Rufflewind (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13527#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#13527: ghc has a stack space leak when prints warnings -------------------------------------+------------------------------------- Reporter: slyfox | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by slyfox): Slightly more context. '''utf8DecodeString''' overflows stack: http://git.haskell.org/ghc.git/blob/HEAD:/compiler/utils/Encoding.hs#l118 {{{#!hs utf8DecodeString :: Ptr Word8 -> Int -> IO [Char] utf8DecodeString ptr len = unpack ptr where !end = ptr `plusPtr` len unpack p | p >= end = return [] | otherwise = case utf8DecodeChar# (unPtr p) of (# c#, nBytes# #) -> do chs <- unpack (p `plusPtr#` nBytes#) -- deep recursion return (C# c# : chs) }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13527#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#13527: ghc has a stack space leak when prints warnings -------------------------------------+------------------------------------- Reporter: slyfox | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Rufflewind): Strange that `utfDecodeString` returns `IO` but doesn’t call any `IO` functions besides `return`. As for `getCaretDiagnostic`, could perhaps revive this diff? https://phabricator.haskell.org/D2875 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13527#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#13527: ghc has a stack space leak when prints warnings -------------------------------------+------------------------------------- Reporter: slyfox | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): Looks like a good diagnosis, thank you. Both problems are surely fixiable if someone cares to try. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13527#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#13527: ghc has a stack space leak when prints warnings -------------------------------------+------------------------------------- Reporter: slyfox | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): The issue with `utf8DecodeString` is that we need to decode all `len` characters before we can return; we cannot do the decoding lazily as the list is demanded since the `Ptr Word8` may be freed. Phab:D3442 adds such a variant.. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13527#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#13527: ghc has a stack space leak when prints warnings
-------------------------------------+-------------------------------------
Reporter: slyfox | Owner: (none)
Type: bug | Status: new
Priority: normal | Milestone:
Component: Compiler | Version: 8.1
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture:
| Unknown/Multiple
Type of failure: None/Unknown | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s):
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#13527: ghc has a stack space leak when prints warnings -------------------------------------+------------------------------------- Reporter: slyfox | Owner: (none) Type: bug | Status: merge Priority: normal | Milestone: Component: Compiler | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * status: new => merge Comment: I'd say this is worth merging to 8.2, no? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13527#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#13527: ghc has a stack space leak when prints warnings -------------------------------------+------------------------------------- Reporter: slyfox | Owner: (none) Type: bug | Status: merge Priority: normal | Milestone: Component: Compiler | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): I suppose; I've been trying to be a bit more strict about merging to stable, but given that this is a new feature we should probably make it work correctly out of the box. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13527#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#13527: ghc has a stack space leak when prints warnings -------------------------------------+------------------------------------- Reporter: slyfox | Owner: (none) Type: bug | Status: merge Priority: normal | Milestone: Component: Compiler | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): comment:7 addresses comment:6. With this patch and 065be6e9eb5114c5f0e3a20626ec93042ce47f13 the test in comment:1 runs to completion without a stack overflow. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13527#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#13527: ghc has a stack space leak when prints warnings -------------------------------------+------------------------------------- Reporter: slyfox | Owner: (none) Type: bug | Status: closed Priority: normal | Milestone: 8.2.1 Component: Compiler | Version: 8.1 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: merge => closed * resolution: => fixed * milestone: => 8.2.1 Comment: Merged as ec5a49fe34180da0adcd7956ad60a9f8ba04c775 and 0eb5004ae3d58032bb48d77a19bed556af7c4f72. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13527#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC