August 2001 - Glasgow-haskell-users

shared libraries in sun solaris
by Siemens Entwicklungsteam 09 Aug '01

09 Aug '01

Hello, we are writing a haskell program that does a calculation for an intranet application. We assume that at peak times, several thousand users are going to use the program at the same time. Our problem is, that our binary file has several MBs and we don't know how to compile it (we are using ghc 5.01) in a way, that all users can share ONE binary file in memory. If every user has one file, we would need several GBs of memory. At the moment we are using Microsoft Windows NT but planning to use Sun Solaris later. Every hint would be a big help, thank you very much, Kilian _________________________________________________________________ Downloaden Sie MSN Explorer kostenlos unter http://explorer.msn.de/intl.asp

1 0

Re: FWD: Re: GHC FFI Return Type Bug
by Fergus Henderson 08 Aug '01

08 Aug '01

Sigbjorn Finne <sof(a)galconn.com> wrote: > > "Julian Seward (Intl Vendor)" <v-julsew(a)microsoft.com> writes: > > > > | > char fooble ( ... ) > > | > { > > | > return 'z'; > > | > } > > | > > > | > on an x86, 'z' will be returned at the lowest 8 bits in %eax. What I > > | > don't know is, is the C compiler obliged to clear the upper 24 bits of > > | > %eax, or does that onus fall on the callee? The upper 24 bits of eax must be cleared in the callee. The caller can assume that they are zero. I don't know if this is documented anywhere, but that is the convention which GNU C follows (on all architectures -- the C front-end internally promotes the return type from "char" to "int"). I can point you to the exact line of code in the GNU C front-end if you really want. I think this is required by traditional K&R C code, which does things like calling such functions without declaring them. -- Fergus Henderson <fjh(a)cs.mu.oz.au> | "I have always known that the pursuit The University of Melbourne | of excellence is a lethal habit" WWW: <http://www.cs.mu.oz.au/~fjh> | -- the last words of T. S. Garp.

1 1

--RTS
by Marcin 'Qrczak' Kowalczyk 08 Aug '01

08 Aug '01

Currently --RTS option for a program compiled with ghc means that all the following arguments are passed to the program, even if they are +RTS and -RTS. IMHO -- should act like --RTS too. Imagine a utility like Unix ls written in Haskell. If someone invokes it ls -- "$FILENAME" he means that "$FILENAME" is a filename. He shouldn't care that it's written in Haskell and be forced to write ls --RTS -- "$FILENAME" as currently. If it's not an option for the program, it's certainly not an option for its runtime. --RTS can be retained in case one really wants to invoke the program with arguments +RTS (and not -- +RTS). PS. +RTS -h has an out of date description of -M. -- __("< Marcin Kowalczyk * qrczak(a)knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZASTÊPCZA QRCZAK

1 0

RE: GC options. Reply
by Simon Marlow 08 Aug '01

08 Aug '01

Ok, I've done the following: - disabled the maximum heap size by default. It can still be enabled with +RTS -M<size> - RTS options are taken from the GHCRTS environment variable in addition to the command line (the command line has precedence). - Compacting garbage collection is enabled when the residency reaches a certain percentage of the maximum heap size (if there is one). Cheers, Simon

2 1

RE: "foreign export dynamic" in GHC
by Simon Marlow 08 Aug '01

08 Aug '01

[ moved to glasgow-haskell-users ] > When I make a function generator with "foreign export > dynamic", GHC spits > out stub files which it apparently then needs for the link. What are > these for? The stub files contain the helper function which invokes the Haskell code when called from C. Each time the foreign export dynamic is invoked to create a new callback function, a small piece of code has to be dynamically generated. It is the address of this dynamically generated bit of code that is returned as the Addr (or Ptr). When called from C, the dynamically generated code must somehow invoke the Haskell function which was originally passed to the f.e.d. function - it does this by invoking the helper function, passing it a StablePtr to the Haskell function. It's split this way for two reasons: the same helper function can be used each time the f.e.d. function is called, and to keep the amount of dynamically generated code to a minimum. Cheers, Simon

1 0

RE: adding isWHNF primop to 5.00.2 native code generator
by Julian Seward (Intl Vendor) 08 Aug '01

08 Aug '01

| In all the cases I tried, the info pointer in the closure | header pointed to the end of the info table. Although a | comment in ghc/includes/ClosureMacros.h worries me: | | "info pointer" The first word of the closure. Might point | to either the end or the beginning of the | info table, depending on whether we're using | the mini interpretter or not. GET_INFO(c) | retrieves the info pointer of a closure. | | Can someone please clarify when the info pointer will point | to the beginning or end of the table? Normally, the info pointer points at the first word following the info table. This word is the start of the closure's entry code. That means that, given an info pointer, we can either evaluate the closure by executing from that address, or look up info about it by indexing backward from it into the info table proper. However, for portable (-unreg) compilation, we can't control the relative placement of code and data. In this case the first word of the itbl points to the entry code. For nativeGen hacking you only need to consider the fast (pointer to end) case. | My problem is that this is a dodgy assumption since the design of the | info table may change in the future, and you get a nasty | unchecked dependency between the primops in the NCG and the | design of the rts. This is a major design problem (IMO) with the NCG. It afflicts every single macro and structure layout which the NCG has to deal with, and has already cost us some hours tracing bugs to do with divergences between the two implementations. I had a plan to derive all the structure layouts and macro defs from a single source to avoid this kind of breakage, but I don't know if it will ever be implemented. | It seems hard to get the kind of automatic offset calculation in the | NCG as you get with the C version of primops. My current idea | is to make some C primops that do the structure and array | indexing for me, and then use them in the NCG. i.e. primops | that essentially export 'closure_flags[type]' and | 'get_itbl(c)->type' to the NCG. It's a problem. StIndex is sometimes helpful, particularly since you can specify a PrimRep which indicates how the index should be scaled. Exactly like adding an integer to a pointer in C. J

1 0

Re: GHC FFI Return Type Bug
by Ashley Yakeley 07 Aug '01

07 Aug '01

At 2001-08-07 11:13, Carl R. Witty wrote: >You can see that the code for f is: > call g > andl $255,%eax > ret >So gcc believes that a function which returns a value of type unsigned >char is not responsible for clearing the high 3 bytes of %eax. This is to be expected; in most cases the caller will not be doing any kind of extension and so the high three bytes won't need to be masked at all. Only the caller knows whether extension is necessary, and (as Sigbjorn points out) whether it needs to do signed or unsigned extension. -- Ashley Yakeley, Seattle WA

1 0

Re: GHC FFI Return Type Bug
by cwitty＠newtonlabs.com 07 Aug '01

07 Aug '01

"Sigbjorn Finne" <sof(a)galconn.com> writes: > "Julian Seward (Intl Vendor)" <v-julsew(a)microsoft.com> writes: > > > > Hmm, we're looking at this. However, I don't really know what > > C is or is not supposed to do here. Given > > > > char fooble ( ... ) > > { > > return 'z'; > > } > > > > on an x86, 'z' will be returned at the lowest 8 bits in %eax. > > What I don't know is, is the C compiler obliged to clear the > > upper 24 bits of %eax, or does that onus fall on the callee? > > Yes, the callee is required to narrow <expr> in 'return <expr>;' to > fit that of the specified return type -- see 9.8 of Harbison and > Steele. So, a C compiler that cause f() to return 0x7fffffff for > the following, > > unsigned char g() > { > return 0x7fffffff; > } > > unsigned int f() > { > return g(); > } > > is in the wrong. [Notice that narrowing for signed integral types > is undefined in ISO C, but most current-day compilers implement > such narrowing ops the same way, i.e., by masking off excess bits.] It's certainly true that an ISO C compiler on a typical machine must return 255 from f() (on an atypical machine, it's possible to have unsigned char be a 32-bit type). However, this is essentially unrelated to the question of whether the x86 ABI allows g() to return a value in %eax that has the upper 3 bytes non-zero. When I compile the following file: -------------------- abitest.c -------------------- unsigned int g_val; unsigned char g() { return g_val; } unsigned int f() { return g(); } --------------------------------------------------- with the command gcc -Wall -O2 -fomit-frame-pointer -S abitest.c I get the output: -------------------- abitest.S -------------------- .file "abitest.c" .version "01.01" gcc2_compiled.: .text .align 16 .globl g .type g,@function g: movzbl g_val,%eax ret .Lfe1: .size g,.Lfe1-g .align 16 .globl f .type f,@function f: call g andl $255,%eax ret .Lfe2: .size f,.Lfe2-f .comm g_val,4,4 .ident "GCC: (GNU) 2.7.2.3" --------------------------------------------------- You can see that the code for f is: call g andl $255,%eax ret So gcc believes that a function which returns a value of type unsigned char is not responsible for clearing the high 3 bytes of %eax. Carl Witty

1 0

RE: POLL: GC options
by Simon Marlow 07 Aug '01

07 Aug '01

> In local.glasgow-haskell-users, you wrote: > > Issue 1: should the maximum heap size be unbounded by default? > > Currently the maximum heap size is bounded at 64M. > Arguments for: this > > stops programs with a space leak eating all your swap > space. Arguments > > against: it's annoying to have to raise the limit when you > legitimately > > need more space. > > I'm for boundless wasting of memory. If I'd cared, I'd set the > ulimits correspondingly. You really don't want to get to work > on Monday morning finding that your favourite prime-finder stopped > Friday five minutes after you left :-) The discussion is somewhat moot, since I removed the default limit anyway, but I just thought I'd point out that there's an important difference between GHC's max heap size and the ulimit: GHC will adjust the sizes of the generations to try to stay within the max heap size (thus trading performance for space), whereas it doesn't do that for the ulimit. Anyway, the upshot is that you'll get more failures setting the ulimit to X than when setting the max heap size to X. I certainly recommend setting either a ulimit or a maximum heap size, especially when doing development. In my experience, modern OS's which do memory overcommit tend to behave very badly when something eats all the swap: if you're lucky, your Haskell process will get killed, but if you're unlucky, some other random process(es) will get killed or the machine will crash altogether. Besides, if you're on a multi-user box using up all the swap isn't very sociable :-) An interesting idea is for GHC to read the current ulimit settings and set the max heap size accordingly; however this doesn't seem to be straightforward. There are four settings: the data seg size, the resident set size, the virtual memory size and the locked-in memory size. The data seg size doesn't affect GHC-compiled programs, because we mmap() the heap. The resident set size similarly doesn't seem to have any effect on Linux. The locked-in memory size doesn't affect us (we don't use mlock()), which leaves only the virtual memory size. This one sets a bound on the whole process size, including the size of the binary itself any memory that has been malloc()ed, so it isn't easy to figure out how much is left for the heap. Cheers, Simon

1 0

POLL: GC options
by Simon Marlow 07 Aug '01

07 Aug '01

Folks, There is some disagreement over how the GC options should be specified for Haskell programs. I've identified a couple of issues below, comments and opinions are greatly appreciated. If there's a concensus that things should be changed, then I'll make the changes for the next release. Issue 1: should the maximum heap size be unbounded by default? Currently the maximum heap size is bounded at 64M. Arguments for: this stops programs with a space leak eating all your swap space. Arguments against: it's annoying to have to raise the limit when you legitimately need more space. Options: 1. remove the default limit altogether 2. raise the default limit 3. no change (any others?) Issue 2: Should -M be renamed to -H, and -H renamed to something else? The argument for this change is that GHC's -M option is closer to the traditional meaning of -H. Issue 3: (suggestion from Julian S.) Perhaps there should be two options to specify "optimise for memory use" or "optimise for performance", which have the effect of setting the defaults for various GC options to appropriate values. Optimising for memory use might enable the compacting collector all the time, for instance. Optimising for performance is hard - we may be able to change some of the defaults to trade space for time, but it's unlikely to be entirely reliable (eg. turning on the compacting collector sometimes improves performance, sometimes reduces it). Cheers, Simon > -----Original Message----- > From: Julian Seward (Intl Vendor) > Sent: Monday, August 06, 2001 10:54 AM > To: Simon Marlow; Marcin 'Qrczak' Kowalczyk; cvs-ghc(a)haskell.org > Subject: RE: cvs commit: fptools/ghc/compiler/stranal DmdAnal.lhs > > > | It appears that no-one understands the GC options except me. > | Perhpas they should be redesigned, but here's the current story: > | > | -H<size> is the *minimum* heap size. The heap will be grown > | as necessary, starting with the minimum, and up to the > | maximum heap size. > | > | -M<size> is the *maximum* heap size, by default 64M. The > | heap is not allowed to grow beyond this size [...] > > In days long since gone (before the 4.XX RTS) -H simply set > the heap size, and because the heap was fixed-size, that was > the max heap size too. > > How about: > > * Renaming current -M to -H, and current -H to -HS. > * Fixing up the sizing calculations a bit so that the > max heap size is more closely observed. > > Result: -H means what it meant originally, but you can still > set an initial heap size with -HS if you want. That means > for the most part we can forget about -M and use -H instead. > > Also having an auto-fallback to compacting collection when > heap gets full. Overall aim is to reduce, ideally to zero, > the number of flags users have to give to the RTS in order to > get reasonable performance yet efficient use of memory. > People simply won't use the compacting collector if you have > to ask for it specially. > > J >

5 4