- Glasgow-haskell-users

> 2^32 b, 6.6 for Itanium
by Serge D. Mechveliani 13 Jun '07

13 Jun '07

Dear GHC developers, I have the two questions. 1. How ghc-6.6 runs on 64-bit machines having more that 2^32 byte memory ? Can it run into "Segmentation fault" ? 2. Can you prepare a reliable ghc-6.6 binary for Itanium a64 Linux ? Or maybe, someone can contribute ? One of my colleagues says the following. ------------------------------------- He has taken such a binary from > (http://www.haskell.org/ghc/distribution_packages.html#debian) > somehow installed it and tried to make from source. It reports >/tmp/ghc12430_0/ghc12430_0.s: Assembler messages: > >/tmp/ghc12430_0/ghc12430_0.s:51:0: > Error: Operand 1 of `mov' should be a general register ------------------------------------- (though, I have not looked so far, in how all this was done). Regards, ----------------- Serge Mechveliani mechvel(a)botik.ru

2 1

unneeded laziness example
by Serge D. Mechveliani 12 Jun '07

12 Jun '07

People, I wrote recently > And now unneeded laziness bites. > [..] > In ghc-6.6, it needs at least 100 Mb of heap to perform. > If I set seq infoK' between `shows' and `p', then it suffices 14 Mb. > [..] > Now: how to program real problems in Haskell ? > Should I try to set seq everywhere and see whether it helps ? I am sorry. This was false alarm. I continued the investigation and have found that Haskell and GHC are NOT guilty in this example (uh! how good). This is the specifics of the method. Probably, one needs to think of seq only if one applies such a method with expenses depending heavily on the computation order. An interesting thing, though. The simplified and contrived example is --------------------------------------------------- main = putStr $ test 18 test :: Integer -> String test w = shows (f tM) "\n" where f = Prelude.minimum . map Prelude.minimum mM = makeMatrix w strictnessM = Prelude.minimum $ map Prelude.minimum mM tM = -- seq strictnessM $ transpose mM --------------------------------------------------- Here mM is n by n matrix of small integers, n = 385 for w = 18. The program prints f $ transpose mM. makeMatrix computes the rows of mM in a complex way, finding m(i,j) uses the table of m(i',j') for other pairs (i',j'). And the time and space costs depend heavily on the order in which m(i,j) are computed. Applying `transpose' changes (due to Haskell laziness) this order into more expensive one: 5 times more of heap needed. And adding `seq ...' before `transpose' returns it to initial order. So, everything is natural. Regards, ----------------- Serge Mechveliani mechvel(a)botik.ru

1 0

Seemingly subtle change causes large performance variation
by Matthew Danish 12 Jun '07

12 Jun '07

Hello, I've been playing with the INTEST problem on SPOJ which demonstrates the ability to write a program which processes large quantities of input data. http://www.spoj.pl/problems/INTEST/ I came across some curious behavior while cleaning up the program. The original program, which runs fast (enough), is: module Main(main) where import Control.Monad import Data.Maybe import qualified Data.ByteString.Char8 as B divisibleBy :: Int -> Int -> Bool a `divisibleBy` n = a `rem` n == 0 main :: IO () main = do [n,k] <- (map int . B.split ' ') `fmap` B.getLine :: IO [Int] let doLine :: Int -> Int -> IO Int doLine r _ = B.getLine >>= testDiv r testDiv r l | int l `divisibleBy` k = return (r + 1) | otherwise = return r foldM doLine 0 [1..n] >>= print where int :: B.ByteString -> Int int = fst . fromJust . B.readInt But when I make a slight modification, the program chews up a ton more memory and takes more time: module Main(main) where import Control.Monad import Data.Maybe import qualified Data.ByteString.Char8 as B divisibleBy :: Int -> Int -> Bool a `divisibleBy` n = a `rem` n == 0 main :: IO () main = do [n,k] <- (map int . B.split ' ') `fmap` B.getLine :: IO [Int] let doLine :: Int -> Int -> IO Int doLine r _ = B.getLine >>= return . testDiv r -- 'return' moved here ^^ testDiv r l | int l `divisibleBy` k = r + 1 | otherwise = r foldM doLine 0 [1..n] >>= print where int :: B.ByteString -> Int int = fst . fromJust . B.readInt This program will generate sample data: import System.Random import System.Environment import Control.Monad main = do [n] <- map read `fmap` getArgs :: IO [Int] k <- randomRIO (1, 100) putStrLn $ unwords [show n, show k] replicateM_ n $ randomRIO (1, 10^9) >>= print Note that the same behavior occurs even if I manually inline the local function and try: return (if .. then .. else). Some sample runs: $ ghc/compiler/ghc-inplace ghc-6.7.20070601: no input files $ ghc/compiler/ghc-inplace --make -O2 intest.hs [1 of 1] Compiling Main ( intest.hs, intest.o ) Linking intest ... $ ghc/compiler/ghc-inplace --make -O2 intest_slow.hs [1 of 1] Compiling Main ( intest_slow.hs, intest_slow.o ) Linking intest_slow ... $ time ./intest +RTS -tstderr -RTS < test1 ./intest +RTS -tstderr 8830 <<ghc: 134876896 bytes, 248 GCs, 28672/28672 avg/max bytes residency (1 samples), 2M in use, 0.00 INIT (0.00 elapsed), 0.12 MUT (0.12 elapsed), 0.00 GC (0.00 elapsed) :ghc>> real 0m0.129s user 0m0.124s sys 0m0.006s $ time ./intest_slow +RTS -tstderr -RTS < test1 ./intest_slow +RTS -tstderr 8830 <<ghc: 144278584 bytes, 269 GCs, 7030784/21843968 avg/max bytes residency (6 samples), 38M in use, 0.00 INIT (0.00 elapsed), 0.13 MUT (0.14 elapsed), 0.10 GC (0.15 elapsed) :ghc>> real 0m0.296s user 0m0.238s sys 0m0.058s -- -- Matthew Danish -- user: mrd domain: cmu.edu -- OpenPGP public key: C24B6010 on keyring.debian.org

2 1

unneeded laziness case
by Serge D. Mechveliani 11 Jun '07

11 Jun '07

People, A couple of times I suffered of unneeded strictness in Haskell. And now unneeded laziness bites. Consider the following program: ------------------------------------------------------- main = let mK = <formed in a certain complex way> :: Map.Map Partition Integer row = [0, 1, 0 ... 0] (mK', p) = decompose mK row infoK' = sum $ map sum $ mapmap (\ n -> rem n 2) $ Map.elems mK' in putStr (shows p -- seq infoK' "\n") ------------------------------------------------------- In ghc-6.6, it needs at least 100 Mb of heap to perform. If I set seq infoK' between `shows' and `p', then it suffices 14 Mb. If I set there seq mK', then again it needs 100 Mb. 7 times as large. mK is a triangular n by n matrix, n is about 900 (so, there are about 1 million of elements in mK). mK has small integers in it. p is found actually by solving the linear system X x mK = row. Now: how to program real problems in Haskell ? Should I try to set seq everywhere and see whether it helps ? Also note: seq mK' was not sufficient. I do not know, probably, as mK' has structure, seq does not evaluate all its levels. Therefore the programmer needs to invent a function that gathers much of information of mK' and puts it into one Integer (infoK' = sum ... mK'). Then, seq infoK' forces most of evaluation of mK'. Is this a generic style to program real problems ? To write all this strange code instead of plain let (_, p) = decompose mK row in putStr (shows p "\n") ? The story was as follows. My friend asked me to decompose certain large symmetric polynomial into elementary symmetric ones. This was needed for a real mathematical problem. I applied my DoCon program, which is written in Haskell. 1000 Mb RAM was not enough. I started to investigate, because I suspected that this case needs several times smaller memory. I found a fragment of the type shown above. And I never use `seq'. I have put a bit smaller example to make it 300 Mb sufficient. The effect was discovered by occasion. I have forgotten that `decompose' returns a pair with a large matrix as fst and wrote let p = decompose mK row in putStr $ show p It has computed in a relatively small memory, and fast, but before the p value, it printed several pages of the matrix value. I said "Oh! we need to improve this, we only need p, the second part": let (_, p) = decompose mK row in putStr $ show p And this takes 7 times more of memory and 2 times more of time! Then, I understood, what is happening and verified the idea by introducing seq. And now, I wonder ... Thank you in advance for your help and comments. Regards, ----------------- Serge Mechveliani mechvel(a)botik.ru

1 0

Failure building HEAD
by Gregory Wright 08 Jun '07

08 Jun '07

Hi, Today I was building HEAD using macports and had the error: configure: Using compiler: ../../compiler/ghc-inplace configure: Compiler flavor: GHC configure: Compiler version: 6.7.20070607 configure: Using package tool: ../../utils/ghc-pkg/ghc-pkg-inplace checking for gcc... gcc checking for C compiler default output file name... configure: error: C compiler cannot create executables See `config.log' for more details. (this was in libraries/base). The config.log showed the problem: configure:1680: checking for C compiler default output file name configure:1683: gcc -I/opt/local/include --configure-option=-O2 -I/ opt/local/include --configure-option=-I/opt/local/include -L/opt/ local/lib --configure-option=-L/opt/local/lib conftest.c >&5 cc1: error: unrecognized command line option "-fconfigure-option=-O2" cc1: error: unrecognized command line option "-fconfigure-option=-I/ opt/local/include"cc1: error: unrecognized command line option "- fconfigure-option=-L/opt/local/lib"configure:1686: $? = 1 It looks like cabal is messed up, passing "--configure-option=" when it should be stripping it out. Is this a problem with the HEAD branch or do I need a development version of cabal from the darcs repository to build the latest ghc? Best Wishes, Greg

2 4

GHC 6.6.1 on Debian Etch?
by Wagner Ferenc 08 Jun '07

08 Jun '07

Hi, what's the best way to install GHC 6.6.1 on a Debian Etch system? Basically: are there installable packages available somewhere, or should I recompile the Sid packages, or create a stub package from a binary .tar.gz bundle, or some other option I didn't think of? -- Thanks, Feri.

3 4

Prelude not in haskell98?
by Iavor Diatchki 08 Jun '07

08 Jun '07

Hello, I am using GHC 6.6 and I am trying to build a library using Cabal. The library is written in Haskell'98 so I made the Cabal file depend only on the package "haskell98". Unfortunately building the library fails with the following error: MyModule.hs:1:0: Failed to load interface for `Prelude': it is a member of package base, which is hidden The module "MyModule" does not have any imports (except for the implicitly imported Prelude). If I add an extra dependency on package "base", then the library compiles. All this seems to indicate that the Prelude is not a part of the "haskell98" package. Is this the case, and if so, is this intentional? It would be nice if we could create Cabal packages that explicitly indicate that the library depends only on Haskell 98 libraries. -Iavor

7 8

"GHC as a library" stalls when nobody is consuming it's output
by Mads Lindstrøm 08 Jun '07

08 Jun '07

Hi all While trying to implement a GUI for GHCi, I have run into an annoying concurrency problems. I think "GHC as a library" is at fault, as it stalls (maybe some deadlock) when nobody is consuming it's output. This message is a literate Haskell program, which illustrates the problem. This test-program starts a thread which prints an 'A' on stderr every second. Then it wait three seconds (meanwhile the 'A's are printing), and runs, via "GHC as a library", the following Haskell code "[1..]". If I start the program as: ./IsGhcBlocking >/dev/null everything works fine and the 'A' keeps coming out on stderr. However, if I do: ./IsGhcBlocking | sleep 10 I only see three 'A's. I believe that is because the sleep-command, do not consume any input. The program was tested on Debian Etch running GHC 6.6. > module Main where > Compile with: ghc -threaded -package ghc-6.6 --make IsGhcBlocking.lhs > import qualified GHC as GHC > import qualified Outputable as GHC > import qualified Packages as GHC > import qualified DynFlags as GHC > import qualified ErrUtils as GHC > > import System.IO > import Control.Concurrent > -- the path of our GHC 6.6 installation > path :: FilePath > -- path = "c:\\ghc-6.6" > path = "/usr/lib/ghc-6.6/" > > main :: IO() > main = do let printAs = do threadDelay (10^6) > hPutStrLn stderr "A" > printAs > forkOS printAs -- forkIO gives the same result > threadDelay (10^6 * 3) > session <- initializeSession > GHC.runStmt session "[1..]" > return () > > initializeSession = > do session <- GHC.newSession GHC.Interactive (Just path) > > -- initialize the default packages > dflags0 <- GHC.getSessionDynFlags session > let myLogAction _ locSpec style errMsg = hPutStrLn stderr showMsg where > showMsg = GHC.showSDoc $ GHC.withPprStyle style $ GHC.mkLocMessage locSpec errMsg > dflags1 = dflags0 { GHC.log_action = myLogAction } > (dflags2, packageIds) <- GHC.initPackages dflags1 > GHC.setSessionDynFlags session dflags2{GHC.hscTarget=GHC.HscInterpreted} > GHC.setContext session [] [] > return session The stalling becomes a problem, when one wants to interrupt "GHC as a library"'s runStmt function. One could interrupt "GHC as a library" with Panic.interruptTargetThread (see http://article.gmane.org/gmane.comp.lang.haskell.glasgow.user/12289) However, as "GHC as a library" locks up (stalls, deadlocks) there is no way of executing Panic.interruptTargetThread. Some people may suggest that I should always consume the output from "GHC as a library". But that is easier said than done. Many GUI libraries (including WxHaskell, which I am using) only allows for one active thread at a time (see http://wxhaskell.sourceforge.net/faq.html) Thus my GUI cannot simultaneously tell "GHC as a library" to interrupt it's execution and read it's output. For thus wondering how I can run both a GUI and "GHC as a library", if WxHaskell is not happy about threading, the answer is that I run the GUI and "GHC as a library" in separate processes. Greetings, Mads Lindstrøm

2 1

Indirection vs. in-place update of closures
by Stefan Holdermans 08 Jun '07

08 Jun '07

Hi, Is there any data available on how often, in GHC-generated code, closures are being updated in-place as opposed to and in comparison with being replaced by indirections to their results? How is it decided whether an in-place or an indirecting update is performed? Thanks, Stefan

2 1

debugging why we end up calling the wrapper rather than the worker
by Duncan Coutts 08 Jun '07

08 Jun '07

Hia all, I'm trying to figure out why this piece of code does not optimise the way I expect. It's binary serialisation again. Yes, again. The crux is the write function write :: Int -> (Ptr Word8 -> IO ()) -> Put () write !n body = Put $ \c buf@(Buffer fp o u l) -> if n <= l then write' c fp o u l else write' (flushOld c n fp o u) (newBuffer c n) 0 0 0 where {-# NOINLINE write' #-} write' c !fp !o !u !l = -- warning: this is a tad hardcore B.inlinePerformIO (withForeignPtr fp (\p -> body $! (p `plusPtr` (o+u)))) `seq` c () (Buffer fp o (u+n) (l-n)) In the if condition, the first path is the fast path. We do want this write' join point, we want to make a call to write' at runtime rather than inlining it into both the fast and slow paths. This is why we tag it with NOINLINE. Simon kindly recently fixed a bug where this NOINLINE was not remembered in the .hi files, so it now really does not inline write' into both branches. However, looking at the core/stg we can see that we're always calling the write' wrapper function that takes boxed arguments and calls the wrapper. It seems to me that the calls in both branches ought to be calls directly to the worker rather than the wrapper. We do have all the unboxed arguments available but they get boxed up to make the call to the write' wrapper. This isn't going to do good things for performance. Here's the STG code: So the important question is whether we have unboxed versions of all the args available. First of all lets look at the wrapper and see what args it unboxes when it calls the worker ($wwrite'_s17v in this case). write'_s17W = sat-only \r [w1_s17P w2_s17C w3_s17G w4_s17J w5_s17M] case w2_s17C of w6_s18S { GHC.ForeignPtr.ForeignPtr ww6_s17Q ww7_s17R -> case w3_s17G of w7_s18T { GHC.Base.I# ww8_s17S -> case w4_s17J of w8_s18U { GHC.Base.I# ww9_s17T -> case w5_s17M of w9_s18V { GHC.Base.I# ww10_s17U -> $wwrite'_s17v w1_s17P ww6_s17Q ww7_s17R ww8_s17S ww9_s17T ww10_s17U; }; }; }; }; So out of the args [w1_s17P w2_s17C w3_s17G w4_s17J w5_s17M] all but the first (which is a continuation function) get unboxed and the bits passed on to $wwrite'_s17v. So we should expect that if we have all those components available that we could make a direct call to the worker $wwrite'_s17v rather than going via the wrapper write'_s17W. So lets look at the STG code that we get from this bit of source: if n <= l then write' c fp o u l else write' (flushOld c n fp o u) (newBuffer c n) 0 0 0 (The STG code here is from a use when n was 3): We want to see if all the components that the worker $wwrite'_s17v needs are available: case <=# [3 ww5_s17X] of wild1_s18W { GHC.Base.False -> case Put.newBuffer w_s17Z n1_r16x of sat_s18d { __DEFAULT -> let { sat_s189 = NO_CCS GHC.Base.I#! [ww4_s187]; } in let { sat_s186 = NO_CCS GHC.Base.I#! [ww3_s184]; } in let { sat_s183 = NO_CCS GHC.ForeignPtr.ForeignPtr! [ww1_s180 ww2_s181]; } in let { sat_s18b = \u [] Put.flushOld w_s17Z n1_r16x sat_s183 sat_s186 sat_s189; } in write'_s17W sat_s18b sat_s18d lvl_r16z lvl_r16z lvl_r16z; }; GHC.Base.True -> let { sat_s18l = NO_CCS GHC.Base.I#! [ww5_s17X]; } in let { sat_s18j = NO_CCS GHC.Base.I#! [ww4_s187]; } in let { sat_s18h = NO_CCS GHC.Base.I#! [ww3_s184]; } in let { sat_s18f = NO_CCS GHC.ForeignPtr.ForeignPtr! [ww1_s180 ww2_s181]; } in write'_s17W w_s17Z sat_s18f sat_s18h sat_s18j sat_s18l; }; In the False case it looks like there is an excuse for calling the wrapper, because we're calling the wrapper for both Put.flushOld and Put.newBuffer. For Put.flushOld it's just to construct the continuation which is a boxed arg anyway, so that's not stopping us calling the worker. The Put.newBuffer returns a ForeignPtr and the write' worker takes the components of the ForeignPtr unboxed. However we are doing a case analysis on the result of Put.newBuffer so we could easily extract the components and pass them on to the write' worker. In the True case there is no excuse at all as far as I can figure out. We are explicitly boxing up exactly the arguments that the wrapper unboxes. So we really should be able to call the worker directly with the appropriate unboxed values as args. So what is going on here? Does NOINLINE prevent calling the worker or something? >From a quick experiment it would appear so: {-# OPTIONS_GHC -fbang-patterns #-} module Foo (foo, bar) where foo :: Int -> Int foo n = bar n bar :: Int -> Int bar !n = bar (n+1) As is, bar will not be inlined because it's recursive and looking at the STG code we see that foo makes a call to bar's worker: Foo.$wbar = \r [ww_sc8] case +# [ww_sc8 1] of sat_sca { __DEFAULT -> Foo.$wbar sat_sca; }; Foo.bar = \r [w_scd] case w_scd of w1_scl { GHC.Base.I# ww_scg -> Foo.$wbar ww_scg; }; Foo.foo = \r [eta_sck] Foo.bar eta_sck; however if we add in what you'd think is a redundant pragma {-# NOINLINE bar #-} and look at the stg code again: Foo.bar = \r [w_sc9] case w_sc9 of w1_sco { GHC.Base.I# ww_scc -> Foo.$wbar ww_scc; }; Foo.$wbar = \r [ww_scf] case +# [ww_scf 1] of sat_sch { __DEFAULT -> let { sat_scj = NO_CCS GHC.Base.I#! [sat_sch]; } in Foo.bar sat_scj; }; Foo.foo = \r [eta_scn] Foo.bar eta_scn; then we see that foo is now calling bar's wrapper, and what's worse, bar is calling it's wrapper in the recursive call! Oh noes! So it seems to me that NOINLINE should prevent inlining but not prevent calling the worker rather than the wrapper. I don't fully understand how NOINLINE interacts with the worker/wrapper transform (or I wouldn't have been surprised by this behaviour). I'm guessing that it works by doing the worker/wrapper split and then trying to inline the wrapper into as many call sites as possible. If this is indeed how it works then it'd explain why attaching NOINLINE to the function causes the observed behaviour since looking at the .hi file we see that the NOINLINE is attached to the wrapper function and not the worker. So perhaps the solution is to attach the NOINLINE to the worker rather than the wrapper when doing the worker/wrapper split. Would that work or cause other problems? Seems otherwise I'm stuck. I thought I could use NOINLINE to control the creation of join points like in my original example. Duncan

2 5