[GHC] #9885: ghc-pkg parser eats too much memory

14 Dec 2014

      #9885: ghc-pkg parser eats too much memory
-------------------------------------+-------------------------------------
       Reporter:  gnezdo             |                   Owner:
           Type:  bug                |                  Status:  new
       Priority:  normal             |               Milestone:
      Component:  ghc-pkg            |                 Version:  7.8.3
       Keywords:                     |        Operating System:  Linux
   Architecture:  Unknown/Multiple   |         Type of failure:  Runtime
     Difficulty:  Moderate (less     |  performance bug
  than a day)                        |               Test Case:
     Blocked By:                     |                Blocking:
Related Tickets:                     |  Differential Revisions:
-------------------------------------+-------------------------------------
 Parsing of spec files in ghc-pkg scales very poorly. The following script
 demonstrates memory consumption growth as a function of the number of
 tokens in ld-options (16K leads to ~6G)

 #!/bin/bash
 # Demonstrates memory consumption behavior of ghc-pkg as a function of
 # the number of ld-options arguments.

 for i in {10..14}; do
   size=$((1 << $i))
   echo $size
   rm -fr a.packages
   /usr/bin/ghc-pkg init a.packages
   cat > a.spec <<EOF
 name: project
 id: project
 license: AllRightsReserved
 version: 1.0
 EOF
   echo -n ld-options: >> a.spec
   for i in $(seq 1 $size); do echo -n "x "; done >> a.spec
   /usr/bin/time /usr/bin/ghc-pkg --global-package-db a.packages register
 --force a.spec
 done
 exit 0

 # This was collected with ghc-7.6.3. 7.8.3 fares as badly.
 % bash a.sh
 1024
 Reading package info from "a.spec" ... done.
 0.03user 0.00system 0:00.03elapsed 97%CPU (0avgtext+0avgdata
 17848maxresident)k
 0inputs+32outputs (0major+4973minor)pagefaults 0swaps
 2048
 Reading package info from "a.spec" ... done.
 0.09user 0.01system 0:00.10elapsed 99%CPU (0avgtext+0avgdata
 60872maxresident)k
 0inputs+56outputs (0major+15723minor)pagefaults 0swaps
 4096
 Reading package info from "a.spec" ... done.
 0.41user 0.07system 0:00.49elapsed 99%CPU (0avgtext+0avgdata
 294340maxresident)k
 0inputs+104outputs (0major+74090minor)pagefaults 0swaps
 8192
 Reading package info from "a.spec" ... done.
 1.72user 0.30system 0:02.04elapsed 99%CPU (0avgtext+0avgdata
 1168836maxresident)k
 0inputs+192outputs (0major+292716minor)pagefaults 0swaps
 16384
 Reading package info from "a.spec" ... done.
 9.11user 1.38system 0:10.51elapsed 99%CPU (0avgtext+0avgdata
 5396932maxresident)k
 0inputs+376outputs (0major+1349736minor)pagefaults 0swaps

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9885>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler