November 2002 - Glasgow-haskell-users

RE: first stab at -ffunction-sections
by Simon Marlow 19 Nov '02

19 Nov '02

> I noticed a lot of not-obviously-used stuff brought in from various > libraries and wanted to nuke some of the unneeded things. Step 1 was > trying to compile the libraries with the option, which didn't quite > fly... it looks like ghc-asm is the primary sufferer, and I'm not sure > the compiler option is needed... Yes, indeed you could do this in ghc-asm. But don't forget the native code generator too... > -split-objs I didn't really realize was there. I see (tracing > through ghc5, whatever debian's latest shipping version is): > > ghc/compiler/main/DriverFlags.hs:250 > , ( "split-objs" , NoArg (if can_split > then do writeIORef > v_Split_object_files True > add v_Opt_C > "-fglobalise-toplev-name > s" > else hPutStrLn stderr > "warning: don't > know how to split \ > \object files on > this architecture" > ) ) -split-objs has to go to some trouble to make more symbols global so that they can still be resolved after the assembler file has been split into chunks. This is one rather ugly hack that it would be nice to get rid of. > Then in ghc/driver/split/ghc-split.lprl:287 (there's actually > one per arch): > > # strip the marker > > $str =~ s/(\.text\n\t\.align > .(,0x90)?\n)\.globl\s+.*_?__stg_split_marker.*\ > n/$1/; > $str =~ s/(\t\.align > .(,0x90)?\n)\.globl\s+.*_?__stg_split_marker.*\n/$1/; > > ... Yes, this is all particularly horrible too. The splitter has to go to some trouble to make sure that "local constants" (strings and the like) get duplicated in each assembler chunk that refers to them, subverting the commoning up of constants that gcc does. > So to me it looks feasible to figure out who's fooling with these > things, though it's probably not necessary to do any of this > within the > compiler except for whatever might circumvent ghc-asm, if anything. > > At any rate, I am finding the amount of unused code/data linked into > the generated executables significant... for instance, in a non- > concurrent program: > > 080a1c64 D MVar_modifyMVarzu_closure > 0805aeb8 T MVar_modifyMVarzu_entry > 0805aece T MVar_modifyMVarzu_fast3 > 0805aeb8 T MVar_modifyMVarzu_info The IO library requires MVars (it's thread-safe) so most programs will have some MVar bits linked in. I'm not sure why modifyMVar in particular is being included though. > ... and as it's a 9-line script to mangle patches, it's certainly not > using this: > > 0805b140 T __stginit_PosixDB > > The idea with -ffunction-sections or brewing up an equivalent is to > build the libraries with it so when the final executable is linked, it > imports only the code and statically-allocated data it uses from them. Ok, I'm convinced :) Let us know if you need any more help. Cheers, Simon

1 0

RE: unsafePerformIO and IORefs
by Simon Marlow 19 Nov '02

19 Nov '02

> Hal Daume III wrote: > > You can't. [...] > > Well, you can, but only for CAFs. This idiom/hack is used > quite happily throughout GHC, HOpenGL, H/Direct, ... I think "quite happily" is a bit strong ;-) We'd much rather have a safe way to do what is really quite a reasonable thing. Cheers, Simon

1 0

RE: unsafePerformIO and IORefs
by Simon Marlow 19 Nov '02

19 Nov '02

> At 2002-11-18 11:05, Sven Panne wrote: > > >global :: a -> IORef a > >global a = unsafePerformIO (newIORef a) > > This is useful, you can do this with it: > > ref = global Nothing > > convert :: a -> IO b > convert a = do > writeIORef ref (Just a) > Just b <- readIORef ref > return b This particular "flexibility" provided by unsafePerformIO is actually documented... http://www.haskell.org/ghc/docs/latest/html/base/System.IO.Unsafe.html#u nsafePerformIO Cheers, Simon

1 0

Strings are slow
by Lauri Alanko 19 Nov '02

19 Nov '02

Hello. Here's a grep program: module Main where import Text.Regex import System.IO import System.Environment import Control.Monad import Data.Maybe main = do [re] <- getArgs let rx = mkRegex re let loop = do line <- getLine when (isJust (matchRegex rx line)) (putStrLn line) eof <- isEOF unless eof loop loop It turned out that this is remarkably slow. The first problem was with inlining. If this is compiled with ghc-5.04 -O -ddump-simpl, I get: case GHC.IOBase.unsafePerformIO @ (Data.Maybe.Maybe (GHC.Base.String, GHC.Base.String, GHC.Base.String, [GHC.Base.String])) (Text.Regex.Posix.regexec (Text.Regex.mkRegex re) a731) of wild4 { Ie. the regex is compiled anew every time a string is matched. A bug? Anyway, without optimization the code produced is reasonable, but still horrendously slow. Testing with a simple word as a pattern from a 7.3MB, 800kline file, the running time was 37.5 seconds. For comparison, a similar program in mzscheme (interpreted!) took 7.3 seconds while the system grep, of course, took 0.4 seconds. I did some profiling by creating new top-level bindings for matchRegex and getLine (is there a better way?): total time = 53.34 secs (2667 ticks @ 20 ms) total alloc = 1,172,482,496 bytes (excludes profiling overheads) COST CENTRE MODULE %time %alloc match Main 69.7 56.8 getl Main 23.9 40.2 main Main 6.3 3.0 So it seems like all the time is spent just converting ByteArrays to char lists to C arrays. This makes me wonder how sensible it really is to represent strings by char lists. Yes, it's nice and uniform and lazy, but... How can I get this faster, then? PackedStrings are not very useful because they just don't support enough operations (getline, matchregex) alone, and having to convert them to Strings sort of defeats their purpose. _Any_ operation that provides only a String interface condemns us to a gazillion allocations. And by default all char*-based foreign interfaces are represented with Strings on the Haskell side. Maybe a generic Textual class with at least String and PackedString (and ByteArray?) as instances would help? Then the common string-based operations could all have separate implementations for separate representations. With heavy specialization, of course. This would be especially useful if the FFI (especially withCString) supported it. Or alternatively, maybe the foldr/build rewriting trick could be used to eliminate some redundant conversions between representations? Just throwing ideas in the air here. Lauri Alanko la(a)iki.fi

1 0

Native Threads in the RTS
by Wolfgang Thaller 19 Nov '02

19 Nov '02

Hello All, A while ago there was a discussion on the shortcomings of the threaded RTS (in short, it doesn't work with foreign APIs that use thread-local state, and that breaks HOpenGL). Back then, it was decided to just keep the threaded RTS off by default and to do something about it some time after 5.04. I believe it's time to think about it again, so I'll take the liberty of proposing an extension to the RTS that might solve the problem. I propose adding something like forkNativeThread :: IO () -> IO () which forks a new Haskell thread that has its own OS thread to execute in. Note that the fact that only one Haskell thread may execute at a time remains unchanged. Whenever the scheduler determines that a "native" haskell thread is next, it sends the OS worker thread to sleep and wakes up the OS thread corresponding to the "native" haskell thread. When the "native" haskell thread yields again, so does the corresponding OS thread. Foreign calls from "normal" (non-native) haskell threads should be handled in exactly the same way as they are currently. If a callback is entered and the current OS thread corresponds to a native haskell thread, the callback should be executed in the current OS thread. Other haskell threads continue to run in the worker thread or in their own dedicated OS thread. Programs that don't use forkNativeThread won't be affected by the change. Thread switching to and from native threads will be slower, but not painfully slow. Wrapping an entire HOpenGL program in forkNativeThread should solve the OpenGL/GLUT thread-local-state problem, for example, and who knows what else it is good for. Any comments? Opinions? Wolfgang

5 8

RE: first stab at -ffunction-sections
by Simon Marlow 18 Nov '02

18 Nov '02

> On Mon, Nov 18, 2002 at 11:55:27AM -0000, Simon Marlow wrote: > > What exactly are you trying to use -ffunction-sections for? > I'm pretty > > sure it won't work as things stand currently, unless you > can guarantee > > to be able to find a text/data boundary symbol for the > garbage collector > > (currently it has to be able to distinguish text from data, > we're in the > > process of lifting the restriction but it's a lot of work). > > Trying to garbage-collect unused functions to reduce the size of the > .text section. You should be aware that -split-objs (the trick we use to build our libraries in lots of little bits) gets most of the benefit that you'd get from using -ffunction-sections. You might get slightly more fine-grained discarding of code with -ffunction-sections, but the effect won't be dramatic (I'm guessing). Also there's the issues of telling the garbage collector and the mangler about it. However, it would be nice to be able to use -ffunction-sections/--gc-sections instead of -split-objs. It's been at the back of my mind to have a go at this someday... Cheers, Simon

2 1

Re: unsafePerformIO and IORefs
by Ashley Yakeley 18 Nov '02

18 Nov '02

At 2002-11-18 11:05, Sven Panne wrote: >global :: a -> IORef a >global a = unsafePerformIO (newIORef a) This is useful, you can do this with it: ref = global Nothing convert :: a -> IO b convert a = do writeIORef ref (Just a) Just b <- readIORef ref return b -- Ashley Yakeley, Seattle WA

2 1

unsafePerformIO and IORefs
by Nicolas Oury 18 Nov '02

18 Nov '02

I want to write something like type State a = IORef a newState :: a -> State a newState v = unsafePerformIO (newIORef v) But I don't want the compileer to inline this nor to inline any application of this. {#NOINLINE newState#} But how can I stop this function to be inlined when applied for example : .... let x = newState 0 in {... code where x is used twice ...} How to be sure that x isn't inlined and that all occurences of x are pointing to the same memory place ? Best regards, Nicolas Oury

4 3

RE: Class/Implicit Param/Module Interface bug
by Simon Peyton-Jones 18 Nov '02

18 Nov '02

These bugs are now fixed in the HEAD. I hope the fixes will make it into 5.04.2 as well. Thanks for identifying them. It was all horribly bogus before. Simon | -----Original Message----- | From: Ashley Yakeley [mailto:ashley@semantic.org] | Sent: 13 November 2002 13:46 | To: Simon Peyton-Jones | Cc: GHC List | Subject: Re: Class/Implicit Param/Module Interface bug | | Actually it's worse than that. This crashes: | | module Main where | { | import A; | | main = let {?p="test"} in putStrLn f; -- BUG: this crashes | -- main = let {?p="test"} in putStrLn g; -- this works | } | | | -- | Ashley Yakeley, Seattle WA

1 0

RE: Bug? [was: Implicit params]
by Simon Peyton-Jones 18 Nov '02

18 Nov '02

Now fixed in the HEAD, and will be in 5.04.2 Thanks for pointing it out. Simon | -----Original Message----- | From: Jorge Adriano [mailto:jadrian@mat.uc.pt] | Sent: 14 November 2002 21:10 | To: Iavor S. Diatchki | Cc: Haskell Cafe; glasgow-haskell-users(a)haskell.org | Subject: Bug? [was: Implicit params] | | On Thursday 14 November 2002 18:47, Iavor S. Diatchki wrote: | > hello, | > | > > Well, actually you must be right since the pure field defines a pure | > > (projection) function... Hmmm, ok, can someone explain this to me, | > > | > > data E s = E{ | > > refi :: STRef s Int, | > > refc :: STRef s Char, | > > m :: Int | > > } | > > | > > -- this is fine, obviously... | > > pure :: E s -> Int | > > pure e = m e | > > | > > -- but this is not... | > > pure2 :: (?e :: E s) => Int | > > pure2 = m (?e) | > > | > > Why exactly isn't this allowed? What is the workaround? | > > Error msg: | > > ------------------------------------------------------------------------ | > > Ambiguous constraint `?e :: E s' | > > At least one of the forall'd type variables mentioned by the | > > constraint | > > must be reachable from the type after the '=>' | > > In the type: forall s. (?e :: E s) => Int | > > While checking the type signature for `pure2' | > > Failed, modules loaded: none. | > > ------------------------------------------------------------------------ - | > >- | > | > this seems like a bug in GHC. Hugs is happy with it. The "s" in the | > pure2 signature is not ambiguous because it is determined when you give | > the value of the implicit parameter. in fact the way i think about | > implicit parameters is simply as a nice notation for computations in the | > environment monad, so in my mind the above two definitions are pretty | > much the same. | | Thanks Iavor! | GHC people, can you confirm this, Is it a bug? I'm using Ghc 5.0.4 SuSE rpms. | Is there a workaround? | | Thanks, | J.A. | | | | _______________________________________________ | Glasgow-haskell-users mailing list | Glasgow-haskell-users(a)haskell.org | http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

1 0