[GHC] #13857: compactAdd doesn't work with SmallArray#
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: Component: Runtime | Version: 8.2.1-rc2 System | Keywords: compact | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- I get this when I try to compact a small array: {{{ internal error: stg_compactAddWorkerzh: TODO: SMALL_MUT_ARR_PTRS (GHC version 8.2.0.20170404 for x86_64_unknown_linux) Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug }}} If I look in `rts/Compact.cmm`, I can see that this hasn't been implemented. I haven't seen this in another issue, so I just wanted to make sure it was not overlooked before the 8.2 release. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by andrewthad): Just wanted to make sure this was seen by the appropriate people. The unordered-containers package just merged a PR to switch to using `SmallArray#` internally. I would like to still be able to put hashmaps on the compact heap. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:1> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by andrewthad): * cc: ezyang, simonmar (added) -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:2> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): Ugh! If this doesn't happen for 8.2.2, I may have to revert the long- awaited `unordered-containers` upgrade. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:3> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by dfeuer): * cc: dfeuer (added) -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:4> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): AFAICT it shouldn't be too difficult to implement this. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:5> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: patch Priority: high | Milestone: 8.2.2 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => patch * differential: => Phab:D3888 * milestone: => 8.2.2 Comment: Actually, I just remembered I already have a WIP patch. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:6> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: patch Priority: high | Milestone: 8.2.2 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888 Wiki Page: | -------------------------------------+------------------------------------- Comment (by David Feuer <David.Feuer@…>): In [changeset:"5f3d2d3be034e04ba872f2695ab6d7b75de66663/ghc" 5f3d2d3/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="5f3d2d3be034e04ba872f2695ab6d7b75de66663" CNF: Implement compaction for small pointer arrays Test Plan: Validate Reviewers: austin, erikd, simonmar, dfeuer Reviewed By: dfeuer Subscribers: rwbarton, andrewthad, thomie, dfeuer GHC Trac Issues: #13860, #13857 Differential Revision: https://phabricator.haskell.org/D3888 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:7> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: merge Priority: high | Milestone: 8.2.2 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888 Wiki Page: | -------------------------------------+------------------------------------- Changes (by dfeuer): * status: patch => merge -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:8> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: closed Priority: high | Milestone: 8.2.2 Component: Runtime System | Version: 8.2.1-rc2 Resolution: fixed | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: merge => closed * resolution: => fixed Comment: Merged to `ghc-8.2` with 6712904886070df03887a47448100593c55fc1ff. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:9> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888 Wiki Page: | -------------------------------------+------------------------------------- Changes (by andrewthad): * status: closed => new * resolution: fixed => Comment: This still doesn't work correctly as noted by rfbacon on a reddit thread: https://www.reddit.com/r/haskell/comments/8juajl/attempt_to_compact_hashmap_... Consider the following example: {{{ import GHC.Compact import Data.Primitive.SmallArray import Control.Monad.ST (runST) main :: IO () main = do _ <- compact (runST (newSmallArray 12 (55 :: Integer) >>= unsafeFreezeSmallArray)) putStrLn "done" }}} This causes the program to crash at runtime with an out of memory error. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:10> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888 Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): Can you verify that the problem is with `SmallArray` and not `Integer`? -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:11> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888 Wiki Page: | -------------------------------------+------------------------------------- Comment (by andrewthad): The reason why this wasn't caught earlier is that the test for compacting `SmallArray#` doesn't actually work. From D3888, we have [this test https://phabricator.haskell.org/differential/changeset/?ref=174725&whitespac... =ignore-most]: {{{ import GHC.Compact import Data.Primitive.SmallArray main :: IO () main = do arr <- newSmallArray 5 (Just 'a') arr' <- compact arr print $ getCompact arr' }}} However, this fails at runtime with {{{ compact_hash_map: compaction failed: cannot compact mutable objects }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:12> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888 Wiki Page: | -------------------------------------+------------------------------------- Comment (by andrewthad): @dfeuer Yes, I get the same problem if I try this with `Int` instead of `Integer`. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:13> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888 Wiki Page: | -------------------------------------+------------------------------------- Comment (by andrewthad): In fact, if we correct the test in GHC's test suite, we can see that it fails as well: {{{ import GHC.Compact import Data.Primitive.SmallArray main :: IO () main = do arr <- newSmallArray 5 (Just 'a') >>= unsafeFreezeSmallArray arr' <- compact arr putStrLn "done" }}} This fails with: {{{ compact_hash_map: out of memory }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:14> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888 Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Whoops, I believe I see the issue. Patch coming. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:15> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888 Wiki Page: | -------------------------------------+------------------------------------- Comment (by andrewthad): Thanks! Any chance this will make 8.4.3? -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:16> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888 Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Yes, I think we can sneak it in. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:17> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888, Wiki Page: | Phab:D4702 -------------------------------------+------------------------------------- Changes (by bgamari): * differential: Phab:D3888 => Phab:D3888, Phab:D4702 Comment: Fix in Phab:D4702. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:18> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.4.3 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888, Wiki Page: | Phab:D4702 -------------------------------------+------------------------------------- Changes (by bgamari): * milestone: 8.2.2 => 8.4.3 -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:19> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.4.3 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888, Wiki Page: | Phab:D4702 -------------------------------------+------------------------------------- Comment (by Ben Gamari <ben@…>): In [changeset:"12deb9a97c05ad462ef04e8d2062c3d11c52c6ff/ghc" 12deb9a/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="12deb9a97c05ad462ef04e8d2062c3d11c52c6ff" rts: Fix compaction of SmallMutArrPtrs This was blatantly wrong due to copy-paste blindness: * labels were shadowed, which GHC doesn't warn about(!), resulting in plainly wrong behavior * the sharing check was omitted * the wrong closure layout was being used Moreover, the test wasn't being run due to its primitive dependency, so I didn't even notice. Sillyness. Test Plan: install `primitive`, `make test TEST=compact_small_array` Reviewers: simonmar, erikd Reviewed By: simonmar Subscribers: rwbarton, thomie, carter GHC Trac Issues: #13857. Differential Revision: https://phabricator.haskell.org/D4702 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:20> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: merge Priority: high | Milestone: 8.4.3 Component: Runtime System | Version: 8.2.1-rc2 Resolution: | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888, Wiki Page: | Phab:D4702 -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => merge -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:21> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13857: compactAdd doesn't work with SmallArray# -------------------------------------+------------------------------------- Reporter: andrewthad | Owner: (none) Type: bug | Status: closed Priority: high | Milestone: 8.4.3 Component: Runtime System | Version: 8.2.1-rc2 Resolution: fixed | Keywords: compact Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3888, Wiki Page: | Phab:D4702 -------------------------------------+------------------------------------- Changes (by bgamari): * status: merge => closed * resolution: => fixed Comment: Merged to 8.4.3. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13857#comment:22> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
participants (1)
-
GHC