Re: Memory usage exploding for complex pattern matching
Hello all, Following up on this, I have created a trac ticket https://ghc.haskell.org/trac/ghc/ticket/14987#no2 as suggested a couple days ago. In order to try to hack my way through and get my library to compile, I tried progressively adding more and more pattern synonyms to try to avoid exhaustiveness checking,yet, something really surprising happened. Big examples have type errors where small ones don't! I tried documenting the process in a repository: https://github.com/VictorCMiraldo/ghc-14987-repro-pipeline Unfortunately, however, I couldn't get the self contained repros in the repository above to throw type errors. Hence, I'm attaching my full code. Compiling the file src/Generics/MRSOP/Examples/GoAST.hs gives type errors, even though it is being generated by the same template haskell code as the other files inside Examples, all of which compile just fine. To trigger the error, just run `stack build`. It really is a shame that the combination of pattern synonyms that shows the best memory performance crashes completely on bigger examples. Any suggestions are appreciated, thank you very much! Have a great weekend! Cheers, Victor On Fri, Mar 30, 2018 at 5:23 PM, Ben Gamari <ben@smart-cactus.org> wrote:
"Victor Miraldo (UU)" <v.cacciarimiraldo@uu.nl> writes:
Hello,
I have just tried compiling my code with 8.4.2 and using -fmax-pmcheck-iterations=0, I gave GHC 12GB of ram and it still ran out (through `ulimit -v$((1024*1024*12))`).
Hmmm, I'm a bit confused. Why are our results so different? How precisely are you invoking GHC?
Here I meant my whole code, not just the repro. I could have been more clear. Nevertheless, I'm calling it through stack:
I'll admit that I am a bit lost; Minimal.hs compiles for me with a maximum residency of ~3.5 GBytes with both -O1 and the PM check enabled using GHC 8.4.1. Is this not the repro you are referring to?
I get the same behavior as you for Minimal.hs.
The "my code" above referred to the whole library that I'm developping. In fact, the Minimal.hs file contains a distilled version of that library with a template haskell splice that we are trying to use in one of our fully fledged examples.
Okay, I just wanted to be certain we were indeed seeing the same behavior. Indeed 3.5 GB is quite a lot of memory for a 700 LoC program, even one with the deep matches seen in Minimal.hs.
Cheers,
- Ben
| In order to try to hack my way through and get my library to compile, | I tried progressively adding more and more pattern synonyms to try to | avoid exhaustiveness checking,yet, something really surprising | happened. Big examples have type errors where small ones don't! That is strange. Sounds as if you want a new Trac report, just for that. Simon | -----Original Message----- | From: Victor Miraldo (UU) <v.cacciarimiraldo@uu.nl> | Sent: 07 April 2018 12:02 | To: Ben Gamari <ben@smart-cactus.org> | Cc: Simon Peyton Jones <simonpj@microsoft.com>; ghc-devs@haskell.org | Subject: Re: Memory usage exploding for complex pattern matching | | Hello all, | | Following up on this, I have created a trac ticket | https://ghc.haskell.org/trac/ghc/ticket/14987#no2 as suggested a | couple days ago. | | In order to try to hack my way through and get my library to compile, | I tried progressively adding more and more pattern synonyms to try to | avoid exhaustiveness checking,yet, something really surprising | happened. Big examples have type errors where small ones don't! | | I tried documenting the process in a repository: | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithu | b.com%2FVictorCMiraldo%2Fghc-14987-repro- | pipeline&data=02%7C01%7Csimonpj%40microsoft.com%7C59fdacad7f94412d665e | 08d59c7717f3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636586957847 | 493826&sdata=bnuBOL%2FELkmJahidX%2BmWPSDDf%2BuKlJkBAJyFVyY%2Bel8%3D&re | served=0 | | Unfortunately, however, I couldn't get the self contained repros in | the repository above to throw type errors. Hence, I'm attaching my | full code. Compiling the file src/Generics/MRSOP/Examples/GoAST.hs | gives type errors, even though it is being generated by the same | template haskell code as the other files inside Examples, all of which | compile just fine. To trigger the error, just run `stack build`. | | It really is a shame that the combination of pattern synonyms that | shows the best memory performance crashes completely on bigger | examples. | | Any suggestions are appreciated, thank you very much! | | Have a great weekend! | Cheers, | Victor | | | | On Fri, Mar 30, 2018 at 5:23 PM, Ben Gamari <ben@smart-cactus.org> | wrote: | > "Victor Miraldo (UU)" <v.cacciarimiraldo@uu.nl> writes: | > | >> Hello, | >> | >>>>>> I have just tried compiling my code with 8.4.2 and using | >>>>>> -fmax-pmcheck-iterations=0, I gave GHC 12GB of ram and it still | >>>>>> ran out (through `ulimit -v$((1024*1024*12))`). | >>>>>> | >>>>> Hmmm, I'm a bit confused. Why are our results so different? How | >>>>> precisely are you invoking GHC? | >>>> | >>>> Here I meant my whole code, not just the repro. I could have been | more clear. | >>>> Nevertheless, I'm calling it through stack: | >>>> | >>> I'll admit that I am a bit lost; Minimal.hs compiles for me with a | >>> maximum residency of ~3.5 GBytes with both -O1 and the PM check | >>> enabled using GHC 8.4.1. Is this not the repro you are referring | to? | >> | >> I get the same behavior as you for Minimal.hs. | >> | >> The "my code" above referred to the whole library that I'm | developping. | >> In fact, the Minimal.hs file contains a distilled version of that | >> library with a template haskell splice that we are trying to use in | >> one of our fully fledged examples. | >> | > Okay, I just wanted to be certain we were indeed seeing the same | > behavior. Indeed 3.5 GB is quite a lot of memory for a 700 LoC | > program, even one with the deep matches seen in Minimal.hs. | > | > Cheers, | > | > - Ben | >
participants (2)
-
Simon Peyton Jones -
Victor Miraldo (UU)