November 2016 - ghc-tickets

[GHC] #12665: Make Read instances faster, and make them fail fast
by GHC 01 Feb '18

01 Feb '18

#12665: Make Read instances faster, and make them fail fast -------------------------------------+------------------------------------- Reporter: dfeuer | Owner: Type: feature | Status: new request | Priority: high | Milestone: 8.2.1 Component: Core | Version: 8.0.1 Libraries | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Runtime Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- At present, the `Read` instances for standard types are generally written ''as though'' `Read` were a typical parser for a programming language. However, it most assuredly is not. Specifically, we currently follow the pattern of lexing and then parsing. Lexing looks at the string and attempts to identify the next token, whatever it may be. But for `Read`, we don't need that. Thanks to the types, we have a very clear sense of what characters we expect to encounter. I've started to sketch out an improvement, which reads `Int` and `Word` around seven times as fast (9.5 times as fast with parentheses and negation), and will fail immediately on something like `read (fix ('a':)) :: Int` rather than going into an infinite loop. To begin with, modify the definition of `paren` thus: {{{#!hs expectP' :: Char -> ReadPrec () expectP' c = lift (expect c) {-# INLINE expectP' #-} paren :: ReadPrec a -> ReadPrec a -- ^ @(paren p)@ parses \"(P0)\" -- where @p@ parses \"P0\" in precedence context zero paren p = do expectP' '(' x <- reset p expectP' ')' return x }}} This allows fast failure when looking for parentheses, so we don't have to scan to the end of the first token (whatever it may be) before concluding that it is not `'('`. Now we can parse `Word` and `Int` very efficiently. I had to specialize earlier than I wanted to convince GHC that I don't want to convert through `Integer`. I'm not sure why the `fromIntegral` rule doesn't fire reliably around here. The code below (temporarily) uses the current definition for base 16, because that's a bit fussy; I'll rewrite it soon. {{{#!hs charDiff :: Char -> Char -> Word charDiff c1 c2 = fromIntegral (ord c1 - ord c2) {-# INLINE charDiff #-} readHexOct :: ReadP Word readHexOct = do _ <- ReadP.char '0' baseId <- lexBaseChar case baseId of Oct -> readBaseP 8 Hex -> L.readHexP {-# INLINE readHexOct #-} data BaseId = Oct | Hex lexBaseChar :: ReadP BaseId lexBaseChar = do { c <- ReadP.get; case c of 'o' -> pure Oct 'O' -> pure Oct 'x' -> pure Hex 'X' -> pure Hex _ -> ReadP.pfail } readWord :: ReadP Word readWord = readNumber (readHexOct ReadP.<++ readBaseP 10) readInt :: ReadP Int readInt = fromIntegral <$> readWord readBaseP :: Integral a => Word -> ReadP a readBaseP !base = do c <- ReadP.get let diff = charDiff c '0' if diff < base then readBaseP' base (fromIntegral diff) else ReadP.pfail {-# INLINE readBaseP #-} readBaseP' :: Integral a => Word -> a -> ReadP a readBaseP' !base !acc0 = ReadP.look >>= go acc0 where go !acc (c:cs) | diff < base = ReadP.get *> go (fromIntegral base * acc + fromIntegral diff) cs where diff = charDiff c '0' go !acc _ = pure acc {-# INLINE readBaseP' #-} readNumber :: Num a => ReadP.ReadP a -> ReadPrec a readNumber p = parens $ do cs <- lift skipSpaces *> look case cs of ('-': _) -> get *> lift (skipSpaces *> (negate <$> p)) _ -> lift $ skipSpaces *> p {-# INLINE readNumber #-} }}} Other `Word` and `Int`-like types can be built on top of that foundation. I haven't yet attempted to deal with other instances, but I think there are probably a lot of opportunities for similar improvements. WARNING: I know very little about parsing, and less about `ReadPrec`. It's conceivable that I've made some semantic errors here, although I don't think I have. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/12665> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 21

[GHC] #10855: GHC rejects code that Haskell 2010 report accepts
by GHC 31 Jan '18

31 Jan '18

#10855: GHC rejects code that Haskell 2010 report accepts -------------------------------------+------------------------------------- Reporter: goldfire | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Revisions: | -------------------------------------+------------------------------------- According to [https://www.haskell.org/onlinereport/haskell2010/haskellch3.html#x8-220003 The Haskell 2010 Report], the following two modules should compile. But they don't, failing with parse errors: {{{ {-# LANGUAGE Haskell2010 #-} module Bug where bool = - case b of False -> 0; True -> (-1) }}} {{{ {-# LANGUAGE Haskell2010 #-} instance Num (IO a) where negate = id main = - do putStrLn "hi!" }}} Note the unary `-` signs. I'm not sure whether we should complicate the parser to fix these, but we should document the compliance-failure in the manual. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10855> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 5

[GHC] #10074: Implement the 'Improved LLVM Backend' proposal
by GHC 31 Jan '18

31 Jan '18

#10074: Implement the 'Improved LLVM Backend' proposal -------------------------------------+------------------------------------- Reporter: | Owner: thoughtpolice thoughtpolice | Status: new Type: task | Milestone: 7.12.1 Priority: high | Version: Component: Compiler | Operating System: Unknown/Multiple (LLVM) | Type of failure: None/Unknown Keywords: llvm, | Blocked By: codegen | Related Tickets: Architecture: | Unknown/Multiple | Test Case: | Blocking: | Differential Revisions: Phab:D530 | -------------------------------------+------------------------------------- This is a meta ticket designed to reflect the current implementation status of the 'Improved LLVM Backend' proposal, documented here: https://ghc.haskell.org/trac/ghc/wiki/ImprovedLLVMBackend -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10074> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 34

[GHC] #12245: Deriving Data at higher kinds
by GHC 31 Jan '18

31 Jan '18

#12245: Deriving Data at higher kinds -------------------------------------+------------------------------------- Reporter: simonpj | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Here's what [https://mail.haskell.org/pipermail/generics/2016-June/000564.html Lennart Spitzner wanted to do]: {{{ > {-# LANGUAGE StandaloneDeriving #-} > {-# LANGUAGE DeriveDataTypeable #-} > {-# LANGUAGE FlexibleInstances #-} > > import Data.Data ( Data ) > > data Foo f = Foo (f Bool) (f Int) > > deriving instance Data (Foo []) > deriving instance Data (Foo Maybe) }}} Of course you can't derive `Data` for `Foo` because we don't know what `f` is, so Lennart is making multiple instances, one for each instance of `f`. It's a bit clumsy. What we would really like is {{{ deriving instance (forall a. Data => Data (f a)) => Data (Foo f) }}} but we don't have higher order instances yet! So Lennart is manually making two instances. This should work, but he gets {{{ > Main.hs: line 45, column 1: > Multiple declarations of ‘$cr2C’ > Declared at: Main.hs:44:1 > Main.hs:45:1 > Main.hs: line 45, column 1: > Multiple declarations of ‘$tr2D’ > Declared at: Main.hs:44:1 > Main.hs:45:1 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/12245> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 7

[GHC] #10754: truncate /= double2Int
by GHC 28 Jan '18

28 Jan '18

#10754: truncate /= double2Int -------------------------------------+------------------------------------- Reporter: cblp | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: | Version: 7.10.2 libraries/base | Keywords: truncate, | Operating System: Unknown/Multiple double2Int, rewrite, rule | Architecture: | Type of failure: Incorrect result Unknown/Multiple | at runtime Test Case: | Blocked By: Blocking: | Related Tickets: Differential Revisions: | -------------------------------------+------------------------------------- There is a rule in libraries/base/GHC/Float.hs: {{{#!hs "truncate/Double->Int" truncate = double2Int }}} But it is not true for some values. Particularly, {{{#!hs let infinity = 1/0 :: Double truncate infinity :: Int == 0 double2Int infinity == -9223372036854775808 -- minBound }}} As a result, the value of {{{truncate (1/0 :: Double) :: Int}}} depends on optimization level. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10754> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 7

[GHC] #12690: Segmentation fault in GHC runtime system under low memory with USE_LARGE_ADDRESS_SPACE
by GHC 27 Jan '18

27 Jan '18

#12690: Segmentation fault in GHC runtime system under low memory with USE_LARGE_ADDRESS_SPACE --------------------------------------+---------------------------------- Reporter: pggiarrusso | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Runtime System | Version: 8.0.1 Keywords: | Operating System: Linux Architecture: x86_64 (amd64) | Type of failure: Runtime crash Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: --------------------------------------+---------------------------------- I have here a ~600MB core dump on GHC 8.0.1 x86_64 linux triggered under low memory (1GB RAM total, no swap, trying to build aeson with optimizations), with a stacktrace blaming the GHC runtime system. Based on code review, it appears that when requesting memory from the OS fails, the error is ignored and GHC segfaults as soon as it writes to the newly "allocated" memory. == Diagnosis == My diagnosis is that osCommitMemory calls the potentially failing `my_mmap` but returns void (hence not propagating failures); it should instead probably use `my_mmap_or_barf` or handle the failure anyhow: https://github.com/ghc/ghc/blob/a6111b8cc14a5dc019e2613f6f634dec4eb57a8a/rt… However, while I'm very convinced that's a bug, I'm only mostly confident that's the culprit, since the segfault happens later; I've just looked for bugs on allocation failures in the code ran shortly before the crash. == Raw data == The core dump is on a VM from the reporter of https://github.com/commercialhaskell/stack/issues/2575, https://github.com/jiakai0419, which graciously gave me access to the VM and bothered reporting in the first place—I'd like to thank her. I'd like to avoid copying the dump here unless strictly needed. == Reproduction == The bug is deterministic on that machine, but it requires a failure in mmap without triggering an OOM, so I'm not 100% sure how easy it is to reproduce but it should be easy on a VM with 1GB RAM and 1 core. Here's the output of `free`: {{{ $ free total used free shared buff/cache available Mem: 1015352 95864 71396 51244 848092 733132 Swap: 0 0 0 }}} Instructions to produce a dump. First, as root: {{{ echo /tmp/core > /proc/sys/kernel/core_pattern }}} that's needed because otherwise the core file is produced in a temporary folder that stack removes. Then (as non-root, root has privileged access to memory), install stack, and run: {{{ ulimit -c unlimited git clone https://github.com/jiakai0419/snowflake.git stack build --verbose }}} Below is some raw data from the core dump. In short, an initializer for newly allocated data segfaults, and gdb tells that the address it's trying to write (bd->start) to can't be accessed, even though it's lower than mblock_high_watermark so it should have been committed (mmap'ed) by getFreshMBlocks (I don't get getReusableMBlocks in detail, but it also calls `osCommitMemory`, so I guess there's the same invariant). The existence of mblock_high_watermark (beyond other reasons) should confirm this is using USE_LARGE_ADDRESS_SPACE. If this diagnosis is correct, the bug exists since USE_LARGE_ADDRESS_SPACE was introduced in https://github.com/ghc/ghc/commit/0d1a8d09f452977aadef7897aa12a8d41c7a4af0, so it's a regression in 8.0.1. {{{ (gdb) bt #0 initMBlock (mblock=0x225c00000) at rts/sm/BlockAlloc.c:676 #1 alloc_mega_group (mblocks=mblocks@entry=1) at rts/sm/BlockAlloc.c:328 #2 0x00007fe650438d98 in allocGroup (n=1) at rts/sm/BlockAlloc.c:379 #3 allocGroup (n=1) at rts/sm/BlockAlloc.c:336 #4 0x00007fe65043ed55 in allocBlock_sync () at rts/sm/GCUtils.c:38 #5 0x00007fe65043ef7d in alloc_todo_block (ws=ws@entry=0x13196b0, size=size@entry=3) at rts/sm/GCUtils.c:330 #6 0x00007fe65043f13a in todo_block_full (size=size@entry=3, ws=0x13196b0) at rts/sm/GCUtils.c:292 #7 0x00007fe65041baed in alloc_for_copy (gen_no=<optimized out>, size=3) at rts/sm/Evac.c:81 #8 copy_tag (tag=2, gen_no=<optimized out>, size=3, src=0x21b353868, info=<optimized out>, p=0x225b13f10) at rts/sm/Evac.c:99 #9 evacuate1 (p=p@entry=0x225b13f10) at rts/sm/Evac.c:596 #10 0x00007fe65041d65c in scavenge_block1 (bd=0x225b004c0) at rts/sm/Scav.c:570 #11 0x00007fe6504415dc in scavenge_find_work () at rts/sm/Scav.c:2040 #12 scavenge_loop1 () at rts/sm/Scav.c:2103 #13 0x00007fe65043cee2 in scavenge_until_all_done () at rts/sm/GC.c:968 #14 0x00007fe65043d75a in GarbageCollect (collect_gen=collect_gen@entry=1, do_heap_census=do_heap_census@entry=rtsFalse, gc_type=gc_type@entry=2, cap=cap@entry=0x7fe650669a80 <MainCapability>) at rts/sm/GC.c:403 #15 0x00007fe65042ff81 in scheduleDoGC (force_major=rtsFalse, task=0x13228a0, pcap=<optimized out>) at rts/Schedule.c:1652 #16 scheduleDoGC (pcap=<optimized out>, task=0x13228a0, force_major=rtsFalse) at rts/Schedule.c:1433 #17 0x00007fe650430c1a in schedule (initialCapability=initialCapability@entry=0x7fe650669a80 <MainCapability>, task=task@entry=0x13228a0) at rts/Schedule.c:551 #18 0x00007fe650431cf8 in scheduleWaitThread (tso=0x20000b388, ret=ret@entry=0x0, pcap=pcap@entry=0x7ffc4ee22c08) at rts/Schedule.c:2361 #19 0x00007fe65042c528 in rts_evalLazyIO (cap=cap@entry=0x7ffc4ee22c08, p=p@entry=0x7895c0, ret=ret@entry=0x0) at rts/RtsAPI.c:500 #20 0x00007fe65042e3a7 in hs_main (argc=96, argv=0x7ffc4ee22dc8, main_closure=0x7895c0, rts_config=...) at rts/RtsMain.c:64 #21 0x0000000000428414 in main () (gdb) info locals bd = 0x225c00100 block = 0x225c04000 <Address 0x225c04000 out of bounds> (gdb) print/x mblock_high_watermark $11 = 0x225d00000 (gdb) print bd->start Cannot access memory at address 0x225c00100 (gdb) print *bd Cannot access memory at address 0x225c00100 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/12690> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 4

[GHC] #11138: Kill the terrible LLVM Mangler
by GHC 23 Jan '18

23 Jan '18

#11138: Kill the terrible LLVM Mangler -------------------------------------+------------------------------------- Reporter: bgamari | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Other Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Since its introduction the LLVM backend has had a mangler which transforms assembler produced by LLVM. In the past this mangler was responsible for shuffling around code to make tables-next-to-code work, but the need for this has past with the resolution of #4213. At this point the only reason it exists is to mangle AVX instructions to 1. rewriting all symbols of type `@function` to `@object` to prevent symbol references from going through the PLT, which would be bad due to tables-next-to-code. 2. On x86_64, rewrite AVX instructions that require alignment to their unaligned counterparts, since the stack is only 16-byte aligned but these instructions require 32-byte alignment. While it's unclear that we will ever be able to discharge (1), it would be nice to investigate whether we can at least do something about (2) (there may even be performance to be gained!). -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/11138> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 9

[GHC] #12470: Move LLVM code generator to LLVM bitcode format
by GHC 23 Jan '18

23 Jan '18

#12470: Move LLVM code generator to LLVM bitcode format -------------------------------------+------------------------------------- Reporter: bgamari | Owner: Type: feature | Status: new request | Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 (LLVM) | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Currently GHC produces LLVM's textual intermediate representation. While this is easy to produce, it's not the path recommended by LLVM's developers. Instead, they [[http://llvm.org/docs/DeveloperPolicy.html#ir- backwards-compatibility|recommend]] that compilers produce LLVM bitcode, which they maintain much stronger backwards compatibility guarantees on. It seems sensible to move the backend over to this format. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/12470> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 3

[GHC] #9885: ghc-pkg parser eats too much memory
by GHC 21 Jan '18

21 Jan '18

#9885: ghc-pkg parser eats too much memory -------------------------------------+------------------------------------- Reporter: gnezdo | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: ghc-pkg | Version: 7.8.3 Keywords: | Operating System: Linux Architecture: Unknown/Multiple | Type of failure: Runtime Difficulty: Moderate (less | performance bug than a day) | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Parsing of spec files in ghc-pkg scales very poorly. The following script demonstrates memory consumption growth as a function of the number of tokens in ld-options (16K leads to ~6G) #!/bin/bash # Demonstrates memory consumption behavior of ghc-pkg as a function of # the number of ld-options arguments. for i in {10..14}; do size=$((1 << $i)) echo $size rm -fr a.packages /usr/bin/ghc-pkg init a.packages cat > a.spec <<EOF name: project id: project license: AllRightsReserved version: 1.0 EOF echo -n ld-options: >> a.spec for i in $(seq 1 $size); do echo -n "x "; done >> a.spec /usr/bin/time /usr/bin/ghc-pkg --global-package-db a.packages register --force a.spec done exit 0 # This was collected with ghc-7.6.3. 7.8.3 fares as badly. % bash a.sh 1024 Reading package info from "a.spec" ... done. 0.03user 0.00system 0:00.03elapsed 97%CPU (0avgtext+0avgdata 17848maxresident)k 0inputs+32outputs (0major+4973minor)pagefaults 0swaps 2048 Reading package info from "a.spec" ... done. 0.09user 0.01system 0:00.10elapsed 99%CPU (0avgtext+0avgdata 60872maxresident)k 0inputs+56outputs (0major+15723minor)pagefaults 0swaps 4096 Reading package info from "a.spec" ... done. 0.41user 0.07system 0:00.49elapsed 99%CPU (0avgtext+0avgdata 294340maxresident)k 0inputs+104outputs (0major+74090minor)pagefaults 0swaps 8192 Reading package info from "a.spec" ... done. 1.72user 0.30system 0:02.04elapsed 99%CPU (0avgtext+0avgdata 1168836maxresident)k 0inputs+192outputs (0major+292716minor)pagefaults 0swaps 16384 Reading package info from "a.spec" ... done. 9.11user 1.38system 0:10.51elapsed 99%CPU (0avgtext+0avgdata 5396932maxresident)k 0inputs+376outputs (0major+1349736minor)pagefaults 0swaps -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9885> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 6

[GHC] #11645: Heap profiling - hp2ps: samples out of sequence
by GHC 21 Jan '18

21 Jan '18

#11645: Heap profiling - hp2ps: samples out of sequence -------------------------------------+------------------------------------- Reporter: thomie | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Profiling | Version: 8.0.1-rc2 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: #664 Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- `make TEST=hpc_fork WAY=prof` is failing, but it has nothing to do with `hpc`. {{{ hp2ps: hpc_fork.hp, line 9, samples out of sequence hp2ps error when processing heap profile for hpc_fork }}} hpc_fork.hs: {{{ module main where import system.posix.process import control.concurrent main = do pid1 <- forkprocess $ do threaddelay 100000 pid2 <- forkprocess $ do threaddelay 100000 print () }}} To reproduce: {{{ $ ghc-8.0.1 hpc_fork.hs -fforce-recomp -prof $ ./hpc_fork +RTS -hc $ hp2ps hpc_fork.hp }}} Example hpc_fork.hp: {{{ JOB "hpc_fork +RTS -hc" DATE "Thu Feb 25 22:37 2016" SAMPLE_UNIT "seconds" VALUE_UNIT "bytes" BEGIN_SAMPLE 0.000000 END_SAMPLE 0.000000 BEGIN_SAMPLE 0.005672 END_SAMPLE 0.005672 BEGIN_SAMPLE 0.003167 END_SAMPLE 0.003167 BEGIN_SAMPLE 0.003311 END_SAMPLE 0.003311 }}} This is a regression from 7.10.3. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/11645> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 12