March 2006 - Glasgow-haskell-users

Re: registering packages in ghc using a directory of package files
by Duncan Coutts 13 Mar '06

13 Mar '06

On Mon, 2006-03-13 at 13:17 +0000, Simon Marlow wrote: > Duncan Coutts wrote: > > > Attached is the current patch I'm testing. > > I think I've said in the past that in principle I like the idea. I'll > commit your patch. Great. > In terms of making this a feature we advertise in the documentation, I > think we could tweak the details a bit. Sure. > I had in mind allowing -package-conf to specify either a directory or > a file, with the obvious interpretation. I think that's how I did a version of this patch a year or so ago. > Also, with your patch, each file in package.conf.d contains a list of > packages. Yes. > Having just a single package per file seems cleaner, Yes it does. > unless you have a good reason to want the possibility for multiples. > I realise this means wider-ranging changes. It was just that this was the simplest change that reused as much code as possible. Coincidentally it is also how we keep the per-package .conf files in gentoo, ie they're installed package.conf databases rather than indivdual package files. There are reasons we did that in gentoo but they're not essential. For ghc I'd recommend a dir of individual package files. The other problem with my patch is that each .conf file that is loaded up is it's own package db so ghc-pkg reports: /usr/lib64/ghc-6.4.1/package.conf.d/hsql-postgresql-1.7.conf: hsql-postgresql-1.7 /usr/lib64/ghc-6.4.1/package.conf.d/hsql-sqlite-1.7.conf: hsql-sqlite3-1.1 /usr/lib64/ghc-6.4.1/package.conf: rts-1.0, base-1.0, haskell98-1.0, template-haskell-1.0, unix-1.0, parsec-1.0, haskell-src-1.0, network-1.0, QuickCheck-1.0, HUnit-1.1, mtl-1.0, fgl-5.2, stm-1.0, readline-1.0, (lang-1.0), (concurrent-1.0), (posix-1.0), (util-1.0), (data-1.0), (text-1.0), (net-1.0), (hssource-1.0) But this is just because I was taking a very simple approach in my patch. It would obviously be better to consider a single directory of individual package files to be one package database. That will of course need more code changes. Then to do this fully there would need to be further changes to use a dir as ghc's main package db rather than the current file, and to get ghc-pkg to understand directory package dbs so it could register packages in them. Duncan

2 1

printing No. of reductions in GHCi
by RHafiz 13 Mar '06

13 Mar '06

Hello there Is there anyway to find out the number of reductions per execution in GHC? (In Hugs , :set +s prints no of reductions but in GHC, the same command prints time and memory.) Thanks a lot in advance. RHafiz -- View this message in context: http://www.nabble.com/printing-No.-of-reductions-in-GHCi-t1262464.html#a334… Sent from the Haskell - Glasgow-haskell-users forum at Nabble.com.

2 1

registering packages in ghc using a directory of package files
by Duncan Coutts 13 Mar '06

13 Mar '06

All, Yet again I've got annoyed by the difficulties with the way ghc registers packages. It's hard to get right. This time I had someone testing hdbc-odbc for me on ppc64. He uninstalled ghc and reinstalled it (to pick up a fix I had made) then he found that building hdbc-odbc failed because the hdbc package was not registered. However portage (the Gentoo package management facility) knew that it was installed. Of course what happened is that when ghc was reinstalled all the ghc packages that were already installed were not re-registered. Actually that is my fault because our ghc ebuild is supposed to do this. (Technically to do this correctly we should re-register the packages in dependency order, but we don't and just use --force) The point is that there are several bits that the packaging system has to get right for this to all work. The majority of the code in our ghc-package.eclass is for dealing with this issue. If we could just have ghc use a directory of package files rather than registering via modifying one central file then this would all become much simpler and more robust. One concern is that it would remove a point where the package database can be checked for consistency. I don't think this is necessarily so. The place that this QA check is really important is when users are manually installing packages. If an package management system finds a QA problem at registration time then it's already an unrecoverable bug. So the ghc-pkg could continue to be used in the way it is now, it will do it's carious checks and then install a file into a global package.conf.d directory rather than modifying a global package.conf file. For distro packaging systems we could use "ghc-pkg --check" or something, and then the package system would take responsibility for actually installing the file (and removing it when the package is uninstalled/upgraded). I'm somewhat tempted to patch our current ghc to do this, it might be easier to get that working than to get our current hacks to work properly. Duncan

2 2

RE: inside the GHC code generator
by Simon Peyton-Jones 12 Mar '06

12 Mar '06

| last days i studied GHC code generator, reading -ddumps of all sorts, | GHC sources, various papers and John's appeals :) | | what i carried from this investigation: | | GHC has great high-level optimization. moreover, GHC now allows to | program STG almost directly. when i look at STG code i don't see | anything that can be done better - at least for these simple loops i | tried to compile. i see just unboxed variables, primitive operations | and simple loops represented as tail recursion. fine. | | then STG compiled to the simplified C (called abstract C earlier and | quasi C-- now), what is next can be translated: | | * to real C and then compiled by gcc | * to assembler by build-in simple C-- compiler | * to assembler by some external C-- compiler (at least it is | theoretically possible) Good stuff Bulat. There's plenty of interesting stuff to be done here. However, let me strongly urge you *not* to focus attention primarily on the gcc route. Compiling via C has received a lot of attention over the years, and there are many papers describing cool hacks for doing so. GHC does not do as well as it could. But there are serious obstacles. That's not gcc's fault -- it wasn't designed for this. Accurate GC is one of them, tail calls is another, and there are plenty more smaller things that bite you only after you've invested a lot of time. This way lies madness. C-- was *designed* for this purpose. GHC uses C-- as its intermediate language (just before emitting C). So a good route is this: * Write C-- to C-- optimisations * Then, if you like, translate that code to C. Already you will be doing better than GHC does today, because the C-- to C-- optimiser will let you generate better C * But you can also emit C--, or native code; both of these paths will directly benefit from your C-- optimisations. The point is that none of this relies on Quick C--; you can always use an alternative back end. You can almost do this today. GHC uses C-- as an intermediate language. But alas, GHC's code generator does not take advantage of C--'s native calls or parameter passing. Instead, it pushes things on an auxiliary stack etc. (Those Sp memory operations you see.) This isn't necessary. We are planning to split GHC's code generator into two parts A) Generate C-- with native calls, with an implicit C-- stack B) Perform CPS conversion, to eliminate all calls in favour of jumps using an explicit stack The current code gen is A followed by B. But A is a much more suitable optimisation platform, and gives more flexibility. Chris Thompson, and undergrad at Cambridge, is doing (B) as his undergrad project, although it remains to be seen whether he'll have enough complete to be usable. Another shortcoming is that the native code generator in GHC isn't capable of dealing with backward jumps to labels (because GHC hasn't needed that so far). But if you did C-- optimisation, you'd probably generate such jumps. It'd be great to beef up the native code gen to handle that. Many of the optimisations you describe (perhaps all) are readily expressible in the C-- intermediate language, and by working at that level you will be independent of with the back end is gcc, a native code generator, or Quick C--, or some other C-- compiler. Much better. Simon

6 5

RE: Re[2]: How to find out the C type signature corresponding to a Haskell function type in FFI?
by Simon Marlow 10 Mar '06

10 Mar '06

On 09 March 2006 16:39, Bulat Ziganshin wrote: > Hello Simon, > > Thursday, March 9, 2006, 5:08:09 PM, you wrote: > >>>> for small "home" projects you can even use "Int" and "int" which >>>> would work in most of Haskell implementations > >> Yes, and to give a concrete example: HsInt and int are different >> types on x86_64 (and most other 64-bit C implementations). > > yes, it's better to use "Int" and "long" types :) I hope that smiley means you're joking... Cheers, Simon

1 0

RE: GHC 6.4.1 and Win32 DLLs: Bug in shutdownHaskell?
by Simon Marlow 10 Mar '06

10 Mar '06

On 09 March 2006 18:19, Michael Marte wrote: > Simon Marlow wrote: > >> Michael Marte wrote: >> >>> Lennart Augustsson wrote: >>> >>>> I'm not implying anything, except that I've plugged the space >>>> leak of 256M every time a DLL is loaded&unloaded. >>>> >>>> -- Lennart >>>> >>>> Michael Marte wrote: >>>> >>>>> Lennart, >>>>> >>>>> do you imply that you have fixed the problem causing the crashes? >>>>> May I safely assume that DLLs produced by ghc 6.4.2 will not crash >>>>> upon being freed? >>>> >>>> >>>> >>> I wanted to test the fix for ticket #711 but I am stuck because >>> there is no current snapshot of ghc 6.5 for mingw. (The last one is >>> dated end of February, the patch was commited on March, 2nd.) >> >> >> The patch hasn't been committed at all, AFAIK. > > Ok, it am new to Darcs, so this patch is only a proposal contained in > the ticket? > >> >>> I saw that the patch is scheduled for milestone 6.4.2 but has been >>> commited only for 6.5. When will it be available for 6.4.2? >>> >>> I started learning to compile ghc and I have succeeded with the >>> current head of the 6.4 branch on Debian Linux/testing. Next step is >>> to compile the head on this system. My current problem is a missing >>> symbol when linking stage1/ghc-6.5, namely u_gencat referenced by >>> stage1/parser/Lexer.o. Am I missing some library? >>> (I think my sources are up to date, I pulled only half an hour ago.) >> >> >> When compiling with 6.4.1 or later, u_gencat should come from >> libHSbase_cbits.a in your 6.4.1 installation. Is it there? >> >> > No, I am compiling with plain 6.4 (a newer Debain package is not > available) and it does not have u_gencat in this library. > However I can compile the head of the 6.4 branch which in turn should > be able to compile the head, shouldn't it? Yes, I think a plain 6.4 can't compile the HEAD right now. We'll try to fix this, but for now you should use a 6.4.1 or later. Cheers, Simon

1 0

Memory leak in Fudgets
by Dimitry Golubovsky 09 Mar '06

09 Mar '06

I wrote a very simple Fudgets program (just copies stdin to stdout, no graphics involved) -------- leaktest.hs ------------------ module Main where import Graphics.UI.Fudgets.Fudgets main = fudlogue (stdoutF >==< stdinF) and ran it like this: yes | leaktest I found out that the program grows in memory. Its "traditional" analog: -------- leaktest2.hs ----------------- module Main where import System.IO main = do hGetLine stdin >>= hPutStrLn stdout main works in constant space. I tried to observe the GC realtime stats with +RTS -Sstderr -M600K It turns out, garbage is collected, number of allocated and collected bytes are mostly constant, and number of live bytes fluctuates, but averagely stays the same: Alloc Collect Live GC GC TOT TOT Page Flts bytes bytes bytes user elap user elap 316720 262144 9528 0.00 0.00 0.02 0.02 144 499 (Gen: 0) 262128 262144 22188 0.01 0.01 0.03 0.04 0 3 (Gen: 0) 262128 262144 8276 0.00 0.00 0.04 0.05 0 0 (Gen: 0) 262128 262144 24612 0.01 0.01 0.05 0.06 0 0 (Gen: 0) 262128 262144 10680 0.00 0.00 0.06 0.07 0 0 (Gen: 0) 262128 262144 23376 0.00 0.00 0.06 0.08 0 0 (Gen: 0) 262128 262144 9444 0.00 0.00 0.07 0.09 0 0 (Gen: 0) 262012 262144 25492 0.00 0.00 0.07 0.10 0 0 (Gen: 0) ... program grows to 10M ......... 262128 262144 1684 0.01 0.01 42.33 66.10 0 0 (Gen: 0) 262128 262144 13396 0.01 0.01 42.34 66.11 0 0 (Gen: 0) 262124 262144 25540 0.01 0.01 42.35 66.12 0 0 (Gen: 0) 262128 262144 11988 0.00 0.00 42.35 66.13 0 0 (Gen: 0) 262128 262144 1760 0.00 0.00 42.36 66.14 0 0 (Gen: 0) 262128 262144 14464 0.00 0.00 42.37 66.15 0 0 (Gen: 0) 262128 262144 1656 0.00 0.00 42.37 66.16 0 0 (Gen: 0) .... and was interrupted ........... 1,614,769,480 bytes allocated in the heap 78,801,952 bytes copied during GC 25,624 bytes maximum residency (6160 sample(s)) 6160 collections in generation 0 ( 10.35s) 1 Mb total memory in use INIT time 0.01s ( 0.00s elapsed) MUT time 32.01s ( 55.38s elapsed) GC time 10.35s ( 10.78s elapsed) EXIT time 0.00s ( 0.00s elapsed) Total time 42.37s ( 66.16s elapsed) %GC time 24.4% (16.3% elapsed) Alloc rate 50,430,027 bytes per MUT second Productivity 75.5% of total user, 48.4% of total elapsed So, what could be the reason of such memory leak? What else may grow if the heap remains constant? How can it be observed? Any ideas are welcome. -- Dimitry Golubovsky Anywhere on the Web

3 2

Allocating aligned memory?
by Li, Peng 09 Mar '06

09 Mar '06

In GHC, how can I allocate a chunk of memory aligned to some block size (say, 512 or 1024 bytes)? I tried to specify it in the "alignment" method in the Storable typeclass, but that does not seem to work. Is Storable.alignment really used in GHC? If so, is there a code example that allocates aligned memory in this way? For the moment I am using the C function memalign() like this: foreign import ccall "static stdlib.h" memalign :: CInt -> CInt -> IO (Ptr CChar) .... do ptr <- memalign alignment size fptr <- newForeignPtr finalizerFree ptr Is it safe to do so? Thanks, Peng

4 3

How to find out the C type signature corresponding to a Haskell function type in FFI?
by Brian Hulley 09 Mar '06

09 Mar '06

Hi - I've got a Haskell module with the following ffi import: foreign import ccall duma_init :: Int -> IO Int However my problem is that I've got no idea what the type signature for the corresponding C function should be, and when I compile the above module with ghc -fglasgow-exts -fffi --make it doesn't create any stubs either. I've tried looking at the wiki but that only seems to give specific examples. I'm trying to find what the mapping is between Haskell function signatures and C signatures is in general. Any ideas? Also, I really wanted to be able to use () -> IO () but () doesn't seem to be allowed in FFI... A third point is, how would I pass an arbitrary monad instead of just using IO? Thanks, Brian.

7 11

Using GHC with SMP and FFI?
by Li, Peng 09 Mar '06

09 Mar '06

[1] Extending the Haskell Foreign Function Interface with Concurrency [2] Haskell on a Shared-Memory Multiprocessor I read the above two papers [1,2] and I have been trying to write an application that uses both FFI and SMP. The first paper [1] shows how FFI is implemented on uniprocessor concurrent Haskell; the second paper [2] shows how SMP Concurrent Haskell is implemented. However, I found little documentation on using FFI with the latest SMP extension. In addition to [1], what has been changed and what should a programmer know if he wants to use FFI in a multithreaded program running on SMP machines? Best, Peng

2 1