Hello all! I'm using Alex/Happy to generate a parser for PHP. This is my first go at using a parser generator
instead of parser combinators and it's been going pretty smoothly so
far, but I've hit a snag.
Compiling the
happy-generated parser is excruciatingly slow -- 20+ minutes with -O0 --
which messes up the typically very fast feedback loop haskell+ghci
allows. Running the code is great; my quickcheck test suite is very
fast. But compiling and loading into ghci are so slow that iterating on
the project further is not practical until I can figure out how to speed
it up.
Are there any folklore techniques for dealing with this problem, or any resources about how to profile/debug slow compilation?
I'm on macos using Stack with lts-13.21, which includes happy-1.19.10.
I
imagine some of this slowdown is inevitable because of the grammar's
size; it has ~275 nonterminals and the generated `Parser.hs` module is
3.8MB. Usually for a problem like this I'd break the module up into
smaller pieces that won't change as often, but in this case it's an
opaque monolith generated by happy. I also see that the generated code doesn't have type signatures on all top level functions.
Here's the relevant output from GHC with -v2 enabled:
```
*** Parser [PHP.Parse.Parser]:
!!! Parser [PHP.Parse.Parser]: finished in 1980.24 milliseconds, allocated 3155.733 megabytes
*** Renamer/typechecker [PHP.Parse.Parser]:
!!! Renamer/typechecker [PHP.Parse.Parser]: finished in 243923.86 milliseconds, allocated 178189.005 megabytes
*** Desugar [PHP.Parse.Parser]:
Result size of Desugar (before optimization)
= {terms: 1,346,664,
types: 672,082,252,
coercions: 789,
joins: 0/9,651}
Result size of Desugar (after optimization)
= {terms: 834,353,
types: 285,053,589,
coercions: 246,053,
joins: 235/735}
!!! Desugar [PHP.Parse.Parser]: finished in 103983.68 milliseconds, allocated 17597.213 megabytes
*** Simplifier [PHP.Parse.Parser]:
Result size of Simplifier iteration=1
= {terms: 1,059,012,
types: 69,942,882,
coercions: 1,934,739,
joins: 235/1,935}
Result size of Simplifier iteration=2
= {terms: 815,760,
types: 64,805,160,
coercions: 1,368,873,
joins: 233/1,929}
Result size of Simplifier
= {terms: 776,400,
types: 64,474,214,
coercions: 1,297,320,
joins: 233/1,929}
!!! Simplifier [PHP.Parse.Parser]: finished in 197842.50 milliseconds, allocated 116983.797 megabytes
*** CoreTidy [PHP.Parse.Parser]:
Result size of Tidy Core
= {terms: 776,400,
types: 64,474,214,
coercions: 1,297,320,
joins: 233/1,929}
!!! CoreTidy [PHP.Parse.Parser]: finished in 97776.92 milliseconds, allocated 11254.968 megabytes
*** CorePrep [PHP.Parse.Parser]:
Result size of CorePrep
= {terms: 795,714,
types: 66,237,082,
coercions: 1,297,320,
joins: 233/4,234}
!!! CorePrep [PHP.Parse.Parser]: finished in 7911.78 milliseconds, allocated 2564.067 megabytes
*** Stg2Stg:
*** CodeGen [PHP.Parse.Parser]:
!!! CodeGen [PHP.Parse.Parser]: finished in 7048.73 milliseconds, allocated 6407.563 megabytes
*** Assembler:
*** CorePrep [PHP.Parse.Parser]:
Result size of CorePrep
= {terms: 795,714,
types: 66,237,082,
coercions: 1,297,320,
joins: 233/4,234}
!!! CorePrep [PHP.Parse.Parser]: finished in 3069.67 milliseconds, allocated 2564.018 megabytes
*** Stg2Stg:
*** CodeGen [PHP.Parse.Parser]:
!!! CodeGen [PHP.Parse.Parser]: finished in 5970.62 milliseconds, allocated 6634.422 megabytes
*** Assembler:
*** Deleting temp files:
Warning: deleting non-existent /var/folders/td/sxyy9wl919740vddr49g8nth0000gn/T/ghc1272_0/ghc_70.s
Warning: deleting non-existent /var/folders/td/sxyy9wl919740vddr49g8nth0000gn/T/ghc1272_0/ghc_72.c
Warning: deleting non-existent /var/folders/td/sxyy9wl919740vddr49g8nth0000gn/T/ghc1272_0/ghc_74.c
```
--
Nathan Bloomfield
Systems Analyst, Team Neutron
Automattic, Inc.