March 2007 - Glasgow-haskell-users

inter module optimizations
by Fawzi Mohamed 28 Mar '07

28 Mar '07

I had posted some data on inter-module optimizations that I had calculated when splitting my program from one computational module to many different ones. Tim Chevalier suggested that my calculation could be interesting to the people here. So I made the effort of preparing the various versions of my code and re doing the analysis better. Unfortunately I had already began renaming things without doing a darcs record, so in the split version some function names are different. I have a tar.bz archive of 21KB, but I did not know if it is considered rude to send attachments, but if someone is interested I can send him the file. Basically it mainly boils down to non-inlining of some important functions on a newtype ( type LatLocI = Word32 newtype LatLoc = LatLoc LatLocI deriving (Eq,Ord) ), because specialization should not be an issue as I had already given specific signatures to my functions. Also worth noting is that using the profiling with -O2 compilation makes one thing that inlining (or using a single module) makes the program slower, whereas the opposite is true. I think that the profiling overhead are incorrectly evaluated. I know that with -O2 one cannot expect profiling to be good, but it would be nice if it wouldn't be so misleading Here some data (obtained with a script that is also in the tar.bz archive) ******** allInOne: original program, monolithic main computational module * timings of -O2 executable 7.67user 0.00system 0:07.69elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+894minor)pagefaults 0swaps * timings of the executable with profiling total time = 15.25 secs (305 ticks @ 50 ms) total alloc = 5,888,786,120 bytes (excludes profiling overheads) ******** splitModule NoReexport NoInline directives: split computational module, no export list for split modules * timings of -O2 executable 10.14user 0.01system 0:10.17elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+901minor)pagefaults 0swaps * timings of the executable with profiling total time = 11.85 secs (237 ticks @ 50 ms) total alloc = 5,888,780,912 bytes (excludes profiling overheads) ******** splitModule Reexport NoInline directives: computational module, no export list for split modules, old module reexport using export list * timings of -O2 executable 8.88user 0.00system 0:08.90elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+901minor)pagefaults 0swaps * timings of the executable with profiling total time = 12.20 secs (244 ticks @ 50 ms) total alloc = 5,888,780,912 bytes (excludes profiling overheads) ******** splitModule NoReexport Inline directives: split computational module, no export list for split modules, explicit inline directives * timings of -O2 executable 6.44user 0.01system 0:06.46elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+895minor)pagefaults 0swaps * timings of the executable with profiling total time = 18.80 secs (376 ticks @ 50 ms) total alloc = 5,374,883,312 bytes (excludes profiling overheads) ************* Fawzi

2 2

GHC compiled program slower with -O1 than -O0
by Neil Mitchell 28 Mar '07

28 Mar '07

Hi, I've written a Haskell program which goes slower with -O1 (19 seconds) than -O0 (12 seconds). Is this considered to be a bug? How interested are you in tracking this down? The problem is that the code is not yet released, depends on Yhc (both the libraries and the binary), a few other modules and may well be a nightmare to reproduce. Even once its reproduced, its a reasonably large program (52 modules), so isn't going to be particularly fun. Are you interested in details of how to reproduce, or shall I wait until I release the program and you have a more stable set of programs to work against? Thanks Neil

3 3

Trouble trying to find packages for ubuntu linux
by Brian Hulley 21 Mar '07

21 Mar '07

Hi, I'm a total linux newbie and am trying to get ghc6.6 installed together with all the libraries. I'm using Ubuntu linux which uses Debian package format but has its own repositories which only have ghc6.4.2 at the moment. >From the ghc downloads page it says the Debian packages are in Debian testing and unstable (I assume 'testing' is the one people are supposed to use(?)). I've managed to get ghc by itself by just browsing the Debian testing repo at http://packages.debian.org/testing/devel/ but the problem arises when I try to get the library packages, since some of them eg libghc6-base-dev, libghc6-base-prof are not listed there (also not found in the unstable repo at the corresponding url above). I'm trying to find all the ghc library packages so I can download them to my own computer and make a local repository to then add to my /etc/apt/sources.list so that I can install the libs without the danger of their dependencies causing other things in the Debian repository to overwrite stuff I've already got from the Ubuntu repository (I've read about apt pinning but this is far too complicated and risky for me at my level of inexperience, and the last time I added the whole Debian repo to my apt sources I make a complete mess of my installation and had to start from scratch...). The ideal thing would be if there was a ghc repo hosted by haskell.org containing only ghc* + libghc* so that any Debian-related linux distro could use it safely by adding the repo to the software sources, so that other things the packages depend on could be obtained from the distro-specific repository as usual (eg libc etc). In any case has anyone managed to get the latest ghc + libs installed in Ubuntu without breaking their system (or their mind :-)) ? Thanks, Brian.

6 8

Re: strict bits of datatypes
by Adrian Hey 21 Mar '07

21 Mar '07

Excuse me for moving this discussion over to ghc users. (On Haskell prime) John Meacham wrote: > Although I have not looked into this much, My guess is it is an issue in > the simplifier, normally when something is examined with a case > statement, the simplification context sets its status to 'NoneOf []', > which means we know it is in WHNF, but we don't have any more info about > it. I would think that the solution would be to add the same annotation > in the simplifier to variables bound by pattern matching on strict data > types? > > Just a theory. I am not sure how to debug this in ghc without digging > into it's code. Well the latest on this is I've sent Simon some code which illustrates the problem, but it seems not quite as simple as I first speculated, in that a simple function that just overwrites tree elements takes exactly the same time whether I rely on strictness annotations or explicit seqs (something I would not expect if my original speculation was correct). But a more complex function that does insertions into the tree takes about 15% longer using strictness annotations than it does with explicit seqs. The object file seems quite a bit larger too. BTW, I suspect one (perhaps the only) reason for the apparent jump from 5% longer (as stated in my earlier post) to 15% is that I modified the test so less time would be spent on garbage collection, thereby amplifying the apparent difference in speeds. I can post the relevant code to anyone who's interested and thinks they might be able to explain this. I guess the next step would be for someone who understands core to take a look at it. I'm afraid I find core incomprehensible :-( Regards -- Adrian Hey

1 0

Re: Installing GHC
by daniel.is.fischer＠web.de 18 Mar '07

18 Mar '07

> -----Ursprüngliche Nachricht----- > Von: Ian Lynagh <igloo(a)earth.li> > Gesendet: 18.03.07 14:21:47 > An: daniel.is.fischer(a)web.de > CC: glasgow-haskell-users(a)haskell.org > Betreff: Re: Installing GHC > On Sun, Mar 18, 2007 at 12:30:15PM +0100, daniel.is.fischer(a)web.de wrote: > > > > It is a debian package, only it doesn't work here. In the sidux forum, I've heard of people having no problems installing ghc, it's just me who gets bitten. > > Here's a session: > > > > root@macduff:/home/dafis# apt-get -s install ghc6 > [...] > > Inst ghc6 (6.6-3 Debian:unstable) > > Conf ghc6 (6.6-3 Debian:unstable) > > root@macduff:/home/dafis# apt-get install ghc6 > [...] > > Entpacke ghc6 (aus .../archives/ghc6_6.6-3_i386.deb) ... > > Richte ghc6 ein (6.6-3) ... > > OK, that does look suspiciously like it's using the Debian package. > > What do > > dpkg -L ghc6 | grep /bin > dpkg -s ghc6 > ls -l /usr/bin/ghc /usr/bin/ghc6 /usr/lib/ghc-6.6/bin/ghc-6.6 > zcat /usr/share/doc/ghc6/changelog.Debian.gz | head -32 > ls -l /usr/local/bin/ghc /usr/local/lib/ghc-6.6/ghc-6.6 > > say? > > You may just also have a locally installed ghc in /usr/local Oh, damn, indeed. Although "make install" failed with an error with the generic linux binary, it did install the (non-working) binaries in /usr/local/bin, so the system didn't look in /usr/bin, where the working thing is. So for the next few months I will remember to check whether make left anything behind if it dies with an error... > > > root@macduff:/home/dafis# ghc --version > > /usr/local/lib/ghc-6.6/ghc-6.6: error while loading shared libraries: libreadline.so.4: cannot open shared object file: No such file or directory > > > Thanks > Ian > > Thanks, Daniel _______________________________________________________________________ Viren-Scan für Ihren PC! Jetzt für jeden. Sofort, online und kostenlos. Gleich testen! http://www.pc-sicherheit.web.de/freescan/?mc=022222

1 0

Re: Installing GHC
by daniel.is.fischer＠web.de 18 Mar '07

18 Mar '07

> -----Ursprüngliche Nachricht----- > Von: Ian Lynagh <igloo(a)earth.li> > Gesendet: 17.03.07 00:03:44 > An: daniel.is.fischer(a)web.de > Betreff: Re: Installing GHC > On Fri, Mar 16, 2007 at 11:55:03PM +0100, daniel.is.fischer(a)web.de wrote: > > > > > > I'm trying to install a ghc on a sidux box (something akin to debian, I believe). apt-get install seemed to work, but running ghc results in: > > > > /usr/local/lib/ghc-6.6/ghc-6.6: error while loading shared libraries: libreadline.so.4: cannot open shared object file: No such file or directory > > > > > > Do you mean you did "apt-get install ghc" and it installed it into > > > /usr/local? > > > > I did "apt-get install ghc6 ghc6-prof ghc6-doc", so I guess that's the debian-package, more so, because the sid in idux is debian's Still In Development. > > It did install in usr/local without complaints, just doesn't work without libreadline.so.4 > > The Debian package is compiled against readline5, not 4, and doesn't > install into /usr/local. This definitely sounds like a sidux package to > me. It is a debian package, only it doesn't work here. In the sidux forum, I've heard of people having no problems installing ghc, it's just me who gets bitten. Here's a session: root@macduff:/home/dafis# apt-get -s install ghc6 Paketlisten werden gelesen... Fertig Abhängigkeitsbaum wird aufgebaut... Fertig Vorgeschlagene Pakete: ghc6-prof haskell-doc Die folgenden NEUEN Pakete werden installiert: ghc6 0 aktualisiert, 1 neu installiert, 0 zu entfernen und 195 nicht aktualisiert. Inst ghc6 (6.6-3 Debian:unstable) Conf ghc6 (6.6-3 Debian:unstable) root@macduff:/home/dafis# apt-get install ghc6 Paketlisten werden gelesen... Fertig Abhängigkeitsbaum wird aufgebaut... Fertig Vorgeschlagene Pakete: ghc6-prof haskell-doc Die folgenden NEUEN Pakete werden installiert: ghc6 0 aktualisiert, 1 neu installiert, 0 zu entfernen und 195 nicht aktualisiert. Es müssen noch 0B von 22,7MB Archiven geholt werden. Nach dem Auspacken werden 108MB Plattenplatz zusätzlich benutzt. Wähle vormals abgewähltes Paket ghc6. (Lese Datenbank ... 99769 Dateien und Verzeichnisse sind derzeit installiert.) Entpacke ghc6 (aus .../archives/ghc6_6.6-3_i386.deb) ... Richte ghc6 ein (6.6-3) ... root@macduff:/home/dafis# ghc --version /usr/local/lib/ghc-6.6/ghc-6.6: error while loading shared libraries: libreadline.so.4: cannot open shared object file: No such file or directory One confusing thing is that libreadline4 isn't listed as one of the dependencies of ghc6. Any ideas anybody? > > > Thanks > Ian > Thanks, Daniel > __________________________________________________________________________ Erweitern Sie FreeMail zu einem noch leistungsstärkeren E-Mail-Postfach! Mehr Infos unter http://freemail.web.de/home/landingpad/?mc=021131

2 1

ANN: regex-base version 0.72
by Chris Kuklewicz 17 Mar '07

17 Mar '07

The version 0.71 of regex-base supplied with GHC 6.6 has a bug in Text.Regex.Base.Context: > instance (RegexLike a b) => RegexContext a b (MatchResult b) where that essentially uses 'tail' where it should not, and this may even be using tail on an empty list. This has been patched in the just announced unstable version of regex-base 0.83 and this message announces that the stable repository at http://darcs.haskell.org/packages/regex-base/ now has version 0.72 which is just 0.71 with this bug fixed. For the eventual GHC 6.6.1 release, the supplied version of regex-base should either be conservatively updated to 0.72 or updated to the slightly more convenient regex-unstable/regex-base version (currently 0.83). Cheer, Chris Kuklewicz

2 1

Re: Installing GHC on sidux/debian
by Daniel Fischer 16 Mar '07

16 Mar '07

> Hi Daniel. > I haven't used rpm based systems very much. So this might not help you > .. > Unfortunately not. If sidux were rpm-based, I might not have these problems. On my SuSE at home I'm a happy ghc-user. Now, I'm visiting my brother, he's a bleeding-edge junkie and his most recent bleeding edge is sidux (branched off kanotix with the express aim to be extra edgy). Since I'll stay for more than a weekend, I want my Haskell here, too. I "apt-get install"ed ghc, got no errors, but can't run it, because sidux has no libreadline.so.4, which ghc needs. So I tried to install a libreadline.so.4, first via the compatibility package linked from the ghc-page. No success, complained about unfulfilled dependencies like ldconfig, though all mentioned dependencies ARE installed here (at least, locate finds them). Tried apt-get install libreadline4, couldn't find a candidate for installation (probably too old :-(, I'm half astonished that sidux is not past libreadline.so.5). So, any debian-users out there, how did you get libreadline.so.4 installed? Maybe that would work for sidux, too. > Try using ldd on you ghc binary to see all dll dependencies. ldd says: not a dynamic executable > Perhaps this gives you some info why the libraries aren't found. > > In general you can try using tools like www.rmpseek.com to find missing > packages. Found plenty, but: Just can't find one which I can install here... > If libraries aren't found running ldconfig might help. This recreates > all symlinks. > > But perhaps the most easiest way is trying to make your sidux box using > debian package repositories (don't know wether this messes up your > system) as I think the ghc rpm package was compiled and tested with > them. > > You might also try installig ghc from source (you need another ghc > binary install first to bootstrap) You can find documentation about how > to install from source on the ghc site. No ghc here, yet, once I have, I WILL build from source. > > If really everythig else fails you can also create a new environment > containing another os (such as debian) eg by using user mode linux. > (Then you need some more disk space) Maybe I can persuade my brother to get a decent linux installation, like SuSE or RedHat, where you can just install rpm's and be done. > > Do you have any recent ghc running? > > Good luck > Marc > Thanks, Daniel __________________________________________________________________________ Erweitern Sie FreeMail zu einem noch leistungsstärkeren E-Mail-Postfach! Mehr Infos unter http://freemail.web.de/home/landingpad/?mc=021131

2 1

Installing GHC
by Daniel Fischer 16 Mar '07

16 Mar '07

Hi, I'm trying to install a ghc on a sidux box (something akin to debian, I believe). apt-get install seemed to work, but running ghc results in: /usr/local/lib/ghc-6.6/ghc-6.6: error while loading shared libraries: libreadline.so.4: cannot open shared object file: No such file or directory So I'm trying to get libreadline.so.4. The compatibility RPM doesn't help, kpackage complains about missing /sbin/ldconfig, libncurses.so.5, libc.so.6 althouh all are present. apt-get install libreadline4 results in Paket libreadline4 ist nicht verfügbar, wird aber von einem anderen Paket referenziert. Das kann heißen, dass das Paket fehlt, dass es veraltet ist oder nur aus einer anderen Quelle verfügbar ist. Doch die folgenden Pakete ersetzen es: readline-common E: Paket libreadline4 hat keinen Installationskandidaten What should I try next? Thanks, Daniel _____________________________________________________________________ Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! http://smartsurfer.web.de/?mc=100071&distributionid=000000000066

4 3

FreeBSD/amd64 port: more progress
by Gregory Wright 15 Mar '07

15 Mar '07

Hi Ian, I'm building ghc-6.6.20070314 using the unregisterized ghc-6.4.2. (BTW, the unregisterized 6.4.2 seems quite reliable. I was able to build happy-1.15 and alex-2.0.1 without any problem.) I configured 6.6.20070314 for debugging by putting GhcUnregisterised=YES GhcWithNativeCodeGen=NO GhcWithInterpreter=NO SplitObjs=NO GhcWithSMP=NO SRC_HC_OPTS+=-debug -L/usr/local/lib GhcRTSWays=debug GhcRtsHcOpts+=-optc-DDEBUG GhcRtsCcOpts+=-optc-g EXTRA_LD_OPTS=-L/usr/local/lib -lbfd -liberty in mk/build.mk. With debugging turned on, the new 6.6 no longer hangs when compiling rts/Linker.c, rather the new 6.6 ghc-inplace fails compiling the first file it tries, Adjustor.c. I was running "top -u" in another window and when the build failed the memory used went unprintably high (i.e., "top" couldn't print it, nor could you print what I said when I noticed this). I was able to re-run the failed compilation under gdb. I added "-v" to ghc's command line. The big slowdown seemed to occur while ghc-inplace was generating the command line for the c compiler. This is when I interrupted it. From gdb: (gdb) run -B/tmp/ghc -optc-O -optc-Wall -optc-W -optc-Wstrict- prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations - optc-Winline -optc-Waggregate-return -optc-Wbad-function-cast -optc- I../includes -optc-I. -optc-Iparallel -optc-DCOMPILING_RTS -optc- fomit-frame-pointer -optc-optc-g -optc-DNOSMP -optc-I/usr/local/ include -optc-fno-strict-aliasing -H16m -O -debug -L/usr/local/lib - optc-O2 -optc-DDEBUG -optc-DNOSMP -static -I/usr/local/include -I. - #include HCIncludes.h -fvia-C -dcmm-lint -v -c Adjustor.c -o Adjustor.o Starting program: /tmp/ghc/compiler/stage1/ghc-6.6.20070314 -B/tmp/ ghc -optc-O -optc-Wall -optc-W -optc-Wstrict-prototypes -optc- Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc- Waggregate-return -optc-Wbad-function-cast -optc-I../includes -optc- I. -optc-Iparallel -optc-DCOMPILING_RTS -optc-fomit-frame-pointer - optc-optc-g -optc-DNOSMP -optc-I/usr/local/include -optc-fno-strict- aliasing -H16m -O -debug -L/usr/local/lib -optc-O2 -optc-DDEBUG -optc- DNOSMP -static -I/usr/local/include -I. -#include HCIncludes.h -fvia- C -dcmm-lint -v -c Adjustor.c -o Adjustor.o Glasgow Haskell Compiler, Version 6.6.20070314, for Haskell 98, compiled by GHC version 6.4.2 Using package config file: /tmp/ghc/driver/package.conf.inplace wired-in package base not found. wired-in package rts mapped to rts-1.0 wired-in package haskell98 not found. wired-in package template-haskell not found. Hsc static flags: -funregisterised -static -static Created temporary directory: /tmp/ghc27577_0 *** C Compiler: gcc -x c Adjustor.c -o /tmp/ghc27577_0/ghc27577_0.s -v -S -Wimplicit - O -D__GLASGOW_HASKELL__=606 -DNO_REGS -DUSE_MINIINTERPRETER -O -Wall - W -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations - Winline -Waggregate-return -Wbad-function-cast -I../includes -I. - Iparallel -DCOMPILING_RTS -fomit-frame-pointer -optc-g -DNOSMP -I/usr/ local/include -fno-strict-aliasing -O2 -DDEBUG -DNOSMP -I /usr/local/ include -I . -I /tmp/ghc/includes -fwrapv Using built-in specs. Configured with: FreeBSD/amd64 system compiler Thread model: posix gcc version 3.4.6 [FreeBSD] 20060305 /usr/libexec/cc1 -quiet -v -I../includes -I. -^C Program received signal SIGINT, Interrupt. 0x00000000015b56f3 in findMBlockMap (p=0x3a242700000) at MBlock.c:68 68 for( i = 0; i < mblock_map_count; i++ ) (gdb) bt #0 0x00000000015b56f3 in findMBlockMap (p=0x3a242700000) at MBlock.c:68 #1 0x00000000015b574b in markHeapAlloced (p=0x3a242700000) at MBlock.c:98 #2 0x00000000015b59df in getMBlocks (n=262145) at MBlock.c:280 #3 0x00000000015ac617 in allocMegaGroup (n=262145) at BlockAlloc.c:174 #4 0x00000000015ac3e0 in allocGroup (n=67108865) at BlockAlloc.c:72 #5 0x000000000159e24a in allocate (n=34359738372) at Storage.c:504 #6 0x000000000159e439 in allocatePinned (n=34359738372) at Storage.c: 593 #7 0x00000000015a1376 in newPinnedByteArrayzh_fast () #8 0x000000000159d3e2 in StgRun (f=0x15a1330 <newPinnedByteArrayzh_fast>, basereg=0x3a2) at StgCRun.c:93 #9 0x00000000015990f3 in schedule (mainThread=0x2164080, initialCapability=0x3a2) at Schedule.c:932 #10 0x0000000001599edd in waitThread_ (m=0x2164080, initialCapability=0x0) at Schedule.c:2156 #11 0x0000000001599dd3 in scheduleWaitThread (tso=0x801dc0000, ret=0x0, initialCapability=0x0) at Schedule.c:2050 #12 0x00000000015959b1 in rts_evalLazyIO (p=0x1931270, ret=0x0) at RtsAPI.c:459 #13 0x000000000159521f in main (argc=1114636288, argv=0x3a2) at Main.c:104 (gdb) f 0 #0 0x00000000015b56f3 in findMBlockMap (p=0x3a242700000) at MBlock.c:68 68 for( i = 0; i < mblock_map_count; i++ ) Looks like someone is asking for too much memory (n=34359738372)! I've taken a cursory look at this, but I wanted to send a note in case you know what is wrong off the top of your head. I'll be away next week so I won't be able to easily test things on my amd64 box. I will be able to look at code, if you can point me to the right places. (Should I be looking at 6.4.2/6.6 differences in Storage.c or Schedule.c?) Best Wishes, Greg

2 4